03 · What You Need to Know
The analysis plan lets you examine the decisions behind the numbers
A statistical analysis plan is more detailed than "we used regression"
A statistical analysis plan specifies how study data are intended to be analyzed. Its precise content varies by study and research tradition, but it can address analytical populations, outcome definitions, statistical models, covariates, transformations, handling of missing data, multiplicity, subgroup analyses, sensitivity analyses, interim analyses, and other decisions required to turn collected data into reported results.
The important feature is not simply that statistical procedures are named. A useful SAP can specify analytical decisions precisely enough that there is less room to choose among plausible approaches after seeing how each affects the results.
The protocol and statistical analysis plan are related but not interchangeable
Study protocol
Describes the broader planned study, including objectives, design, participants, interventions or exposures, outcomes, procedures, and usually the general analytical approach.
Statistical analysis plan
Provides more detailed instructions for how the collected data are intended to be processed and statistically analyzed.
If your question concerns changes to the overall study design, outcomes, eligibility criteria, or procedures, you may need to compare the protocol with the final paper. If your question concerns precisely how the numbers were produced, the SAP may be more informative.
Prespecification matters because many analyses involve legitimate choices
Real datasets rarely come with one inevitable analytical pathway. Researchers may need to decide which observations belong in an analysis, how variables are represented, which covariates enter a model, how missing observations are handled, whether transformations are needed, which interactions or subgroups are examined, and which sensitivity analyses are performed.
Many alternative choices can be statistically defensible. The concern arises when choices are influenced by the results they produce and are then presented as though they were always intended.
A prospectively finalized analysis plan can help distinguish planned analytical decisions from later exploratory or responsive ones.
The timing and version of the SAP matter
Finding an SAP is not enough. Ask when the relevant version was finalized and whether amendments occurred.
A document finalized before investigators had access to information capable of influencing analytical decisions provides stronger evidence of prespecification than one created after relevant results were already available. The exact timing that matters can depend on the design and how the study was conducted.
Version histories, dates, protocol records, repositories, trial registries, and amendments can therefore be important parts of the appraisal.
Compare the SAP with the analysis actually reported
Current CONSORT guidance for randomized trials explicitly recognizes the importance of this comparison. CONSORT 2025 notes that detailed statistical analyses are frequently prespecified in an SAP and recommends that trial reports identify and justify deviations from the SAP or, when no SAP exists, from the protocol. It also calls for distinguishing prespecified analyses from post hoc analyses.
That distinction matters because a statistically interesting result discovered during exploration can still be scientifically valuable. It simply carries a different evidential status from an analysis specified before the result was known.
Look beyond the name of the statistical test
Suppose both the SAP and paper say that linear regression was used. That does not necessarily mean the analyses match.
You may still need to compare which outcome definition was analyzed, which observations were included, which covariates were adjusted for, how missing data were treated, whether continuous variables were transformed or categorized, what interaction terms were used, and how multiple analyses were handled.
This is one reason close reading of the Methods section remains necessary even when an SAP exists.
Pay particular attention to analysis populations
In randomized trials, terms such as intention-to-treat, modified intention-to-treat, per-protocol, and safety population can correspond to different rules about which participants contribute to particular analyses.
Do not infer those rules from the label alone. Determine how the population was actually defined in the SAP and final report. Excluding participants or observations can sometimes be appropriate, but it can also change an estimate.
Check how missing data were supposed to be handled
Missing observations are common in empirical research, and different approaches can produce different estimates under different assumptions. An SAP may specify whether complete-case analysis, imputation, likelihood-based methods, sensitivity analyses, or another strategy will be used.
If the final paper handles missing data differently, ask why. The change may be entirely justified by the data encountered during the study, but it should be understood rather than silently treated as equivalent.
Subgroup and sensitivity analyses deserve special attention
Subgroup analyses can be compelling because they appear to identify who benefits, who is harmed, or under what conditions an effect occurs. But examining many subgroups also creates opportunities for chance patterns.
An SAP can help establish which subgroup analyses were prespecified and which emerged later. Similarly, sensitivity analyses may show whether a principal result survives reasonable alternative assumptions or analytical decisions.
Do not dismiss post hoc analyses merely because they were post hoc. Interpret them according to what they are: potentially informative analyses that may require greater caution and, sometimes, confirmation in other data.
An SAP does not guarantee a good analysis
Prespecifying a weak method does not make it strong. A flawed model remains flawed when planned six months in advance.
The SAP helps answer one question: what analysis was intended, and how does that compare with what was eventually done? You still need to judge whether the analytical approach itself was appropriate. If the method is unfamiliar, first work through what you do not understand about the analysis before drawing conclusions from the discrepancy.
Watch Out
Do not treat every deviation from an SAP as evidence of questionable practice. Real data can expose problems that could not reasonably have been anticipated. The important questions are what changed, why it changed, when the decision was made, whether the change was disclosed, and how it affects interpretation.
Not every field routinely provides a separate SAP
Detailed prospective SAPs are especially established in clinical trials and some other confirmatory research settings. Practices differ substantially across disciplines and study designs.
For many observational, exploratory, qualitative, computational, or secondary-data studies, there may be no document called a statistical analysis plan at all. Depending on the research context, similar information might instead appear in a preregistration, protocol, registered report, repository, or detailed Methods section.
Do not apply expectations developed for randomized clinical trials mechanically to every form of research.
07 · A Quick Checklist
What to compare between the SAP and final paper
When reading a statistical analysis plan, check:
Which SAP version you are reading and when it was finalized or amended
How primary and secondary outcomes were defined for analysis
Which participants or observations were supposed to enter each analysis
Which statistical models, covariates, transformations, and effect measures were planned
How missing data, exclusions, and protocol deviations were intended to be handled
Which subgroup, interaction, sensitivity, or secondary analyses were prespecified
Whether multiplicity or repeated analyses required adjustment or another planned strategy
Which analyses in the final paper differ from the plan and whether the differences are explained
Whether the paper clearly distinguishes prespecified analyses from exploratory or post hoc analyses
