Should You Preregister Sample Size and Data Collection Stopping Rules?

Should You Decide Your Sample Size and Stopping Point Before Seeing Results?

You plan to recruit 200 participants for an experiment. After collecting data from 150, the treatment effect is statistically significant. Could you stop early? Alternatively, suppose you reach 200 participants but the result is just above your significance threshold. Would collecting another 50 participants be acceptable?

Both situations raise a question about how decisions to continue or stop collecting data influence statistical inference. The issue is not simply whether the final sample is large enough. It is whether the amount of information collected was determined independently of the results being evaluated, or through a statistical design that explicitly accommodates interim decisions.

Preregistration can clarify these commitments, but researchers need to distinguish sample-size justification, recruitment targets, stopping rules, and legitimate adaptations when circumstances change.

Prespecify Both the Intended Sample and the Conditions for Stopping

How Sample Size and Stopping Rules Should Be Preregistered

Sample Size and Stopping Rules Are Related but Different Decisions

Sample size concerns how many participants, observations, clusters, events, or other units the study intends to obtain. A stopping rule specifies the circumstances under which data collection will end.

A study might target 300 participants and stop when that number has completed the primary assessment. Another might recruit until a specified calendar date, even if fewer than 300 participants are obtained. A third might use a sequential statistical procedure with predetermined interim evaluations.

These designs can produce different amounts of information and different inferential properties.

Specifying one without the other can leave important discretion unresolved. For example, a target of 200 participants does not explain whether recruitment will continue if some participants withdraw or whether data collection may end early because the observed result appears favorable.

Why Does Outcome-Dependent Stopping Matter?

Under conventional fixed-sample null-hypothesis significance testing, the analysis is ordinarily calibrated for a particular sampling and testing procedure. Repeatedly examining the accumulating results and stopping when a conventional significance threshold is crossed can change the probability of a false-positive conclusion.

This practice is often called optional stopping. Its consequences depend on the statistical framework, the frequency and nature of interim examinations, and the rule used to make decisions.

Consider a researcher who tests the same null hypothesis after every additional group of participants and stops as soon as the p-value falls below 0.05. The final p-value cannot automatically be interpreted as though the researcher conducted one prespecified fixed-sample test.

Not every form of interim monitoring is inappropriate. Group-sequential designs, alpha-spending approaches, and appropriately constructed anytime-valid methods can accommodate planned interim decisions. The important distinction is whether the inferential procedure accounts for the stopping process.

What Exactly Should Be Included in the Sample-Size Specification?

A meaningful preregistration should explain more than the intended number of participants.

For clustered or longitudinal designs, the number of clusters, observations per cluster, repeated measurements, and assumptions about dependence may be more informative than a participant count alone.

How Should You Justify a Sample-Size Target?

The appropriate justification depends on the research objective and inferential framework.

In a confirmatory experiment, an a priori power analysis may be suitable. It typically considers the target effect size, significance threshold, desired power, variability, and design characteristics. These inputs should be scientifically justified rather than selected merely to produce a convenient sample size.

For estimation-focused studies, a precision-based calculation may be more appropriate. Instead of asking how many observations are needed to reject a null hypothesis, researchers may ask how many are needed to estimate a parameter with a desired level of uncertainty.

Some studies are constrained by the number of eligible participants or available cases. A feasibility-based justification can be transparent, but researchers should explain the resulting limitations rather than claim that a convenient sample is automatically sufficient.

In qualitative research, sample planning may depend on the analytic approach, research scope, heterogeneity, information power, or evolving assessment of the material collected. These considerations should not be forced into an inappropriate statistical power calculation.

How Do You Calculate Sample Size for a Simple Comparison?

For illustration, consider a two-group comparison of independent means with equal allocation, a common outcome standard deviation, a two-sided significance threshold, and a normal-approximation sample-size formula.

This approximation is useful for explaining the logic of sample-size planning, not as a universal calculation. A real study may require an exact or software-based procedure, different assumptions, unequal allocation, adjustment for clustering, or a simulation-based calculation.

The result does not establish that a sample of 126 guarantees detection of an effect, that the assumed effect size is correct, or that a nonsignificant result would demonstrate the absence of a meaningful effect.

Should Recruitment Stop at the Target Number or Continue Until Enough Data Are Complete?

The preregistration should distinguish the number recruited, the number enrolled or randomized, and the number expected to contribute to the primary analysis.

For example, if the objective is to obtain 180 completed assessments and the anticipated completion rate is 90%, researchers might plan to recruit approximately 200 participants. This allowance should be documented in advance.

However, replacing participants after randomization or excluding those with missing outcomes can create methodological problems. In randomized research, the analysis population and missing-data strategy should be consistent with the intended estimand and the prespecified participant eligibility rules.

Researchers should avoid making replacement decisions based on observed treatment effects. If recruitment continues to compensate for missing data, the procedure and relevant limits should be defined clearly.

Can You Preregister Sequential or Adaptive Stopping?

Yes. Preregistration does not require every study to use a fixed sample size.

A group-sequential design may specify several interim analyses and stopping boundaries for efficacy, futility, or safety. An adaptive design may permit particular changes based on accumulating information, provided the design and statistical analysis appropriately account for those adaptations.

For such studies, prespecification should address the maximum sample size, timing of interim analyses, information used for decisions, stopping boundaries, and final inferential method.

Safety monitoring may also require action when unexpected risks emerge, regardless of whether a planned statistical boundary has been reached. Participant protection obligations take precedence over maintaining an unchanged research plan.

Sequential designs require appropriate statistical expertise. Merely stating that recruitment will stop when results become significant is not a sufficient sequential analysis plan.

Can You Increase or Decrease the Sample Size After Preregistration?

Changes may be justified by recruitment feasibility, funding, unexpected attrition, revised measurement assumptions, or other methodological considerations.

The critical issue is whether the change was influenced by observed outcomes and whether the intended inferential procedure remains valid.

An increase made before examining treatment differences, based on a documented feasibility problem or blinded nuisance-parameter reassessment, raises different concerns from an increase motivated by a nearly significant treatment effect.

When a revision is necessary, record the original target, revised target, reason, timing, information available, and implications for analysis. Preserve the original registration and follow the procedures for documenting preregistration amendments.

Do Qualitative Studies Need Fixed Numerical Stopping Rules?

Not universally. In qualitative inquiry, the appropriate stopping rationale depends on the methodology and analytic purpose.

Some studies use a planned sample range informed by information power, variation across participants, or practical constraints. Others use iterative sampling and reassess whether further data are likely to contribute meaningfully to the analysis.

Researchers should explain the intended sampling strategy, the basis for deciding whether additional data are needed, and the circumstances under which recruitment may end.

The concept of saturation is used in some qualitative traditions but is not universally appropriate. Reflexive thematic analysis, for example, may involve different assumptions about coding and theme development from approaches that treat saturation as a measurable endpoint.

The relevant question is addressed further through preregistration for qualitative research.

Planning Recruitment and Stopping for an Educational Experiment

Ending with fewer participants than intended does not automatically invalidate the study. However, the resulting uncertainty may be greater than originally anticipated, and the researchers should avoid interpreting an inconclusive result as evidence that the intervention has no effect.

If the team instead wanted to extend recruitment, it would need to document the amendment and explain whether relevant outcome information influenced the decision.

Common Misunderstandings About Sample Size and Stopping Rules

How to Select a Defensible Sample-Size and Stopping Strategy

Begin with the primary research objective and the type of inference you intend to make. Then select a sample-size justification and stopping strategy compatible with the design.

Coordinate the stopping plan with the primary outcome, intended analysis population, and statistical model. These decisions are interdependent and should not be justified in isolation.

Where uncertainty about assumptions is substantial, simulation or sensitivity calculations may be more informative than a single sample-size estimate. The aim is to make the design's operating characteristics understandable before the results become available.

Before Preregistering Sample Size and Stopping Rules

Frequently Asked Questions About Preregistering Sample Size

Decide How Data Collection Will End Before Results Influence the Decision

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.