03 · What You Need to Know
Why the Timing of Primary Analysis Specification Matters
What Is the Primary Analysis?
The primary analysis is the principal analytical procedure used to address a study's main research question or confirmatory objective.
In a randomized intervention study, it may estimate the difference in a prespecified outcome between treatment conditions.
In an observational study, it may estimate a defined association or causal contrast under stated assumptions.
The primary analysis is not necessarily the most complicated statistical model in the study.
Its importance comes from its role in supporting the central scientific conclusion.
When a study has multiple primary objectives, researchers should clarify how the corresponding analyses and inferential claims will be handled.
Why Should the Primary Analysis Be Planned Before Recruitment?
Recruitment is part of the design that determines which population the analysis will represent.
Eligibility criteria, recruitment procedures, allocation methods, and expected sample composition can affect the estimand and analytical requirements.
For example, a study designed to estimate an intervention effect among all randomized participants requires different planning from one focused on participants who complete a specified level of intervention exposure.
The intended analysis also influences sample-size planning and the information that must be collected.
If researchers wait until recruitment is underway to determine the primary comparison, they may discover that essential baseline variables, clusters, or follow-up measurements were not included in the protocol.
Early specification therefore supports design coherence as well as protection against result-driven analytical choices.
What Is the Difference Between Planning Before Recruitment and Finalizing Before Unblinding?
Planning before recruitment
Establishing the essential primary analytical strategy early enough to inform eligibility, sample size, measurement, allocation, and the research protocol.
Finalizing before unblinding or comparative outcome review
Completing remaining analytical specifications before investigators obtain information that could encourage choices favoring particular treatment results.
These stages serve related but different purposes.
In some clinical trials, a detailed SAP may be finalized before database lock and unblinding, even though recruitment began earlier.
However, this does not justify leaving the primary outcome, comparison, or sample-size assumptions undefined during study design.
Researchers should verify the applicable protocol, registration, regulatory, sponsor, and institutional requirements for their study.
Which Primary Analysis Decisions Should Be Made Before Recruitment?
The essential decisions should be sufficiently clear to determine what evidence the study needs and how its main result will be interpreted.
| Decision |
Why early specification matters |
Illustrative requirement |
| Primary research question |
Determines the intended scientific inference. |
Define the principal comparison or target quantity. |
| Primary outcome |
Determines what must be measured. |
Specify the construct, instrument, and assessment time. |
| Target population |
Defines to whom the inference applies. |
Align eligibility and recruitment with the intended population. |
| Estimand |
Clarifies the effect or quantity being estimated. |
Define the treatment contrast and relevant handling of intercurrent events. |
| Primary analytical approach |
Determines model and data requirements. |
Specify an appropriate comparison or estimation strategy. |
| Sample-size assumptions |
Determines recruitment requirements. |
Account for effect size, variability, clustering, or other relevant parameters. |
| Essential covariates |
Determines which baseline data are required. |
Identify planned adjustment variables or confounders. |
| Missing-data strategy |
Influences follow-up and measurement planning. |
Identify anticipated missingness and a defensible primary approach. |
Not every item requires the same level of technical detail before recruitment, but none should be ignored when it materially affects the design.
Must the Exact Statistical Model Be Finalized Before the First Participant?
Not universally. The required timing depends on the study's design and applicable requirements.
For a simple confirmatory study, the primary model can often be fully specified before recruitment.
For a complex trial, certain implementation details may be finalized in a separate SAP after recruitment begins, provided the process remains protected from inappropriate outcome-informed decisions.
However, the principal analytical strategy should already be clear enough to support sample-size planning and data collection.
Researchers should distinguish technical refinement from changing the scientific target or selecting a model after examining which one produces the desired result.
Should the Primary Outcome Be Fixed Before Recruitment?
Generally, yes, for confirmatory research.
The primary outcome should be defined before participants are enrolled so that measurement procedures, assessment timing, sample-size calculations, and analytical objectives are aligned.
For example, a study of an educational intervention should establish whether its primary outcome is final examination performance, an independently scored writing task, or another defined measure.
These outcomes are not interchangeable merely because they concern academic learning.
Changing the primary outcome after recruitment may be justified in unusual circumstances, but the timing, rationale, approvals, and implications require transparent documentation.
Changes made after investigators know comparative outcome results raise particular concerns about selective reporting.
How Does Primary Analysis Planning Affect Sample Size?
Sample-size planning depends on the intended analytical target and assumptions about variability, effect magnitude, precision, allocation, and dependence among observations.
For example, a cluster randomized trial must account for the number of clusters and within-cluster correlation rather than treat all individuals as independent allocation units.
A repeated-measures study may require assumptions about the correlation among observations from the same participant.
These requirements should be considered before recruitment targets are established.
Otherwise, a study may recruit the intended number of participants while remaining inadequately designed for its primary analysis.
Sample-size calculations should correspond to the planned analysis and intended inference rather than an unrelated statistical test.
Should the Analysis Population Be Specified Before Recruitment?
Yes, at least at the level necessary to establish the primary target and design.
For randomized studies, the intended analysis may concern participants according to their assigned treatment conditions, with appropriate handling of missing outcomes and intercurrent events.
However, terms such as intention-to-treat should be accompanied by clear definitions rather than used as substitutes for a complete analysis strategy.
For observational studies, the target population, eligibility criteria, observation period, and relevant exclusions should be established prospectively.
Changing the analytical population after examining results can alter the estimand and introduce selection bias.
Researchers should distinguish justified data-quality exclusions from exclusions selected because they change the estimated effect.
Should Covariates Be Selected Before Recruitment?
Important covariates should generally be identified during design when they affect data collection or the primary inferential strategy.
For example, an observational study examining the association between digital learning access and academic performance may require baseline socioeconomic and educational variables to address plausible confounding.
If these variables are not collected, they cannot ordinarily be recovered through a later statistical decision.
In randomized studies, prespecified baseline adjustment can improve precision under appropriate conditions.
Researchers should avoid selecting adjustment variables solely according to their observed statistical significance or their effect on the preferred conclusion.
Can Researchers Use Baseline Data to Refine the Analysis?
Potentially, provided the refinement is scientifically justified and appropriately protected from outcome-informed selection.
For example, blinded review of baseline distributions may identify data coding problems or reveal that a planned categorical variable contains very sparse categories.
However, changes based on baseline information can still affect the analysis and should not automatically be considered harmless.
The team should document what information was available, why the refinement was necessary, and whether the change affects the intended estimand or confirmatory interpretation.
Where applicable, independent or blinded review may reduce the risk that treatment-effect knowledge influences analytical decisions.
What If the Data Do Not Meet the Planned Model Assumptions?
The analysis plan should anticipate important model assumptions and establish how potential problems will be assessed.
For example, researchers may plan to examine residual patterns, influential observations, or model convergence.
However, assumption diagnostics do not always establish whether an analysis is valid, and some causal assumptions cannot be tested directly from the observed data.
Alternative methods should be selected according to their suitability for the research question and data structure.
Researchers should not automatically choose whichever alternative produces the smallest p-value or most favorable estimate.
Consequential departures from the original analysis should be documented and interpreted transparently.
What If Recruitment Reveals Unexpected Population Characteristics?
Recruitment may reveal that the enrolled population differs from what researchers anticipated.
For example, a study may recruit fewer participants with limited digital access than expected, affecting the precision of planned subgroup analyses.
Researchers should assess whether the recruitment process remains consistent with the target population and whether the intended primary analysis is still feasible.
However, changing the primary question solely to match whichever subgroup is most numerous can alter the study's intended inference.
Where changes are necessary, the reasons and implications should be documented.
Researchers should distinguish recruitment feasibility problems from opportunities to select a more favorable analytical population.
Can the Primary Analysis Be Changed After Recruitment Begins?
Yes, under some circumstances, but the justification and timing matter.
For example, an unforeseen data structure or a discovered error in the original specification may require revision.
A change made before investigators have access to relevant comparative outcome information is generally less vulnerable to result-driven selection than one made after examining treatment effects.
However, absence of unblinding does not automatically establish that every change is appropriate.
Researchers should evaluate whether the modification changes the estimand, primary outcome, analytical population, or intended inferential claim.
Relevant approvals, registration updates, and reporting requirements should be followed.
Does Preregistration Have to Occur Before Recruitment?
Requirements depend on the study type, registry, journal, funder, and applicable regulatory framework.
For example, prospective registration of applicable clinical trials is required under certain regulatory and publication policies.
Other quantitative studies may use voluntary preregistration to document hypotheses and planned analyses.
Researchers should not assume that uploading a plan after observing results provides the same evidentiary protection as genuine prospective registration.
Likewise, preregistration and a detailed SAP serve related but distinct functions.
Researchers should verify the specific requirements that apply to their study before recruitment begins.
What Is the Difference Between a Protocol Amendment and an SAP Refinement?
A protocol amendment changes an approved aspect of the study, potentially including objectives, procedures, outcomes, or other substantive features.
An SAP refinement may provide additional technical detail about how an already specified analysis will be implemented.
However, the labels do not determine whether a change is consequential.
For example, changing the primary outcome cannot be treated as a minor technical refinement merely because the change is recorded in the SAP.
Researchers should evaluate the substance of the modification and follow applicable approval procedures.
Changes should be dated, justified, and retained in an auditable version history.
How Does This Differ for Observational Research?
Observational studies may have different analytical objectives and sources of uncertainty from randomized trials.
For confirmatory observational causal research, early specification of the target population, exposure, outcome, confounding strategy, and identification assumptions is particularly valuable.
For descriptive studies, the principal estimates, sampling design, weighting, and uncertainty procedures may be central.
Exploratory observational studies may legitimately develop analyses through examination of the data, provided their exploratory status is transparent.
The absence of randomization does not make prospective analytical planning unnecessary.
However, the degree of prespecification should remain appropriate to the intended inferential claims.
How Does This Differ for Exploratory Research?
Exploratory studies often seek patterns, generate hypotheses, or develop analytical understanding rather than test one fixed confirmatory hypothesis.
It may therefore be inappropriate to require every possible analysis to be predetermined.
However, researchers should still define the study's purpose, data collection requirements, and principles for transparent reporting.
Exploratory analyses should not later be described as though they were prespecified confirmatory tests.
Researchers should distinguish flexibility that serves discovery from flexibility used to conceal result-driven selection.
How Should the Primary Analysis Be Protected From Outcome-Informed Decisions?
Relevant safeguards may include prospective documentation, restricted access to comparative outcome data, blinded analytical review, independent methodological oversight, and clear amendment procedures.
Not every safeguard is necessary or feasible in every study.
For example, blinding may be difficult in some educational or behavioral interventions, but independent outcome assessment or separation of analytical responsibilities may still be possible.
The objective is to reduce opportunities for knowledge of emerging results to influence the primary analytical strategy.
Researchers should also establish procedures for reporting deviations from the original plan.
What If the Primary Analysis Is Not Feasible?
Analytical planning may reveal that the proposed design lacks sufficient clusters, essential covariates, appropriate measurements, or adequate follow-up.
These problems should be addressed before recruitment where possible.
If discovered during implementation, the research team should assess whether a defensible modification can preserve the intended inference.
Where the necessary information cannot be obtained, the study may require redesign or a more limited research question.
Researchers should not substitute an inappropriate test merely to produce a numerical result.
These decisions may require redesigning a study because the required analysis is infeasible.
How Should Researchers Report Changes Made After Recruitment?
Reports should identify the original analytical plan, consequential modifications, the reasons for change, and when the changes occurred relative to access to outcome information.
Where appropriate, researchers should distinguish results from the original analysis and the revised analysis.
Changes to registered outcomes or primary analytical methods may require corresponding updates and explanations under applicable requirements.
Transparent reporting enables readers to evaluate whether modifications were justified or potentially influenced by observed results.
However, disclosure does not automatically eliminate bias introduced by an inappropriate change.
Watch Out
Do not interpret permission to finalize some SAP details before unblinding as permission to begin recruitment without a defensible primary analytical strategy. Essential decisions about outcomes, target populations, comparisons, and sample-size requirements belong in study design.
