01 · The Question
Should You Power Your Study for the Effect You Expect or the Smallest Effect That Matters?
Suppose previous research suggests that an intervention may produce a moderate improvement, but a considerably smaller improvement would still justify adopting it. Which effect size should determine your sample size?
Using the expected effect may produce a smaller, more manageable study. Planning around the smallest effect worth detecting may require substantially more participants. The choice is consequential because it determines which effects your study is adequately powered to detect, and which potentially meaningful effects it might miss.
The distinction is not simply between an optimistic and a conservative estimate. The two quantities answer different questions.
03 · What You Need to Know
How the Two Effect Sizes Lead to Different Sample-Size Decisions
The Expected Effect Size Is a Prediction
The expected effect size represents the magnitude of the effect that researchers anticipate observing in the population. It may be informed by previous studies, meta-analyses, substantive theory, preliminary data, or other relevant evidence.
For example, a researcher might expect a standardized mean difference of 0.50 between an intervention group and a control group because comparable studies have reported effects of approximately that magnitude.
This is a prediction, not a guarantee. Published estimates may be affected by sampling error, differences in populations, selective reporting, or methodological limitations. Even a well-supported expectation can be wrong.
When the anticipated effect is uncertain, researchers should distinguish the evidence supporting their estimate from the precision with which that estimate is known. An effect observed in a different study population may not transfer directly to the proposed investigation.
The Smallest Effect Size of Interest Is a Decision Threshold
The SESOI is the smallest effect magnitude that researchers consider scientifically, practically, clinically, educationally, or otherwise substantively important for a specified research purpose.
For example, an educational intervention might be expected to improve test scores by eight points, while an improvement of four points would already justify serious consideration of its implementation.
Here, eight points is the expected effect and four points is a candidate SESOI. The smaller threshold reflects a judgment about meaningfulness rather than a prediction about what will happen.
The threshold should be grounded in the consequences of the effect, the measurement scale, relevant evidence, and the intended use of the findings. It should not be selected simply because it produces an affordable sample size.
A detailed explanation of what constitutes a defensible SESOI is particularly useful before choosing the target effect for power analysis.
Expected effect size
What magnitude of effect do we anticipate?
Smallest effect size of interest
What is the smallest magnitude that would matter for our research objective?
Why Does the Choice Change the Required Sample Size?
In a conventional superiority test, smaller effects are harder to detect reliably than larger effects when other design features remain unchanged. A study powered for a large effect can therefore have inadequate power for a smaller but meaningful effect.
For an illustrative two-sided comparison of two independent means with equal allocation, a common normal-approximation formula is:
The calculation illustrates an important relationship: for this design, the approximate required sample size is inversely proportional to the square of the standardized effect size.
It does not establish that either sample size is appropriate for a particular study. Actual planning must reflect the statistical test, outcome distribution, allocation ratio, variance assumptions, dependence among observations, and intended analysis.
Why Can Planning Around the Expected Effect Be Risky?
Imagine that a researcher expects d = 0.50 but considers d = 0.30 substantively worthwhile. A study powered at 80% for d = 0.50 will have substantially less than 80% power if the true effect is d = 0.30.
Failure to reject the null hypothesis would then be difficult to interpret. The study might have missed an effect that the researcher had already considered meaningful.
This does not make the original power calculation mathematically incorrect. It means that the calculation answered a narrower question: how often would the planned test reject the null hypothesis if the true effect were the larger target?
Researchers sometimes overlook this distinction when they treat the effect size used in power analysis as though it were merely a technical input.
Does Using the SESOI Guarantee That the Study Will Detect a Meaningful Effect?
No. If a study has 80% power at its SESOI, it still has a 20% probability of failing to reject the null hypothesis when the true effect equals that threshold, under the assumed model.
Power also does not guarantee that the observed estimate will exceed the SESOI, that the confidence interval will exclude effects considered trivial, or that the intervention will prove worthwhile in practice.
Those are distinct inferential objectives. If the aim is to demonstrate that an effect exceeds a meaningful threshold, the statistical hypotheses and sample-size calculation should explicitly address that threshold rather than merely test whether the effect differs from zero.
Watch Out
A conventional test of a zero effect, powered using the SESOI, is not equivalent to a test demonstrating that the effect is larger than the SESOI. The latter requires a different hypothesis formulation and potentially a different sample-size calculation.
When Might the Expected Effect Be the Better Planning Target?
The expected effect may be appropriate when the primary objective is to evaluate a clearly specified theoretical prediction, when reliable prior evidence supports the target, or when a design must be evaluated under a particular anticipated scenario.
For example, a confirmatory experiment might be designed to test a specific predicted effect magnitude while separately acknowledging that smaller effects are not its principal target.
There are also studies in which the research objective is not conventional superiority testing. Equivalence, noninferiority, estimation, prediction-model development, and some adaptive designs require planning quantities that cannot simply be substituted with a SESOI in a standard superiority formula.
For estimation-focused research, planning around confidence-interval precision may better match the research objective.
Can You Use Both Effect Sizes?
Yes. In many studies, using both is preferable to treating the planning decision as an either-or choice.
The SESOI can define the effect the study should be capable of detecting, while the expected effect can describe the most plausible outcome under current evidence.
A power curve or sensitivity table can then show how power changes across several plausible true effects. This makes the consequences of uncertainty visible rather than concealing them within a single assumption.
When resources are limited, a range of plausible planning assumptions may reveal whether the proposed study remains informative under less favorable conditions.
The distinction also matters when the expected effect size is unknown. Uncertainty about what will happen does not necessarily prevent researchers from specifying what would matter.
04 · A Practical Example
Planning an Educational Intervention When a Smaller Improvement Would Still Matter
Hypothetical Example
Comparing Two Approaches to Teaching Statistics
A researcher plans a randomized study comparing a new statistics-learning intervention with conventional instruction. The primary outcome is a continuous achievement score, and the planned analysis is a two-sided comparison of independent group means.
Previous evidence suggests an expected standardized mean difference of d = 0.50. However, discussions about instructional costs and educational benefits indicate that an effect of d = 0.30 would already be worth detecting.
Step 1: Specify the two quantities
Expected effect: d = 0.50. Smallest effect of interest: d = 0.30. Both require justification, but they represent different judgments.
Step 2: Calculate alternative sample sizes
Using the normal approximation with 80% power and two-sided α = 0.05 gives approximately 63 participants per group for d = 0.50 and 175 per group for d = 0.30.
Step 3: Examine the consequence of the smaller design
If the researcher recruits approximately 63 participants per group, the study is planned to detect the larger effect with roughly 80% power, but its power at d = 0.30 is only about 39% under the same approximation.
Step 4: Match the design to the scientific objective
If detecting d = 0.30 is essential, the larger design is the more appropriate starting point. If the larger design is infeasible, the researcher must reconsider the study objective, resources, or design rather than silently redefine the meaningful effect.
The example assumes independent observations, equal allocation, and an appropriate continuous-outcome model. In actual educational research, students may be clustered within classes or institutions. Such dependence can materially increase the required sample size and must be considered before recruitment targets are finalized.
06 · What This Means for You
Choose the Planning Effect by Identifying What Your Study Must Establish
Before calculating sample size, write down the scientific or practical consequence that would make a particular effect important. Then ask whether the planned statistical analysis is designed to detect that effect, estimate it precisely, or establish that it exceeds a decision threshold.
These objectives may require different calculations even when they concern the same outcome.
A simple decision framework
If detecting the smallest meaningful nonzero effect is essential
Consider powering the planned superiority test at a substantively justified SESOI.
If a specific effect is predicted by strong prior evidence
Evaluate power at the expected effect while examining the consequences of smaller plausible effects.
If you must demonstrate that an effect exceeds a meaningful threshold
Formulate the hypothesis around that threshold and calculate sample size for the corresponding test.
If estimation is the primary objective
Consider a precision-based design using a justified target confidence-interval width.
If the SESOI-based sample is infeasible
Reassess the design and the study's achievable objectives, and disclose the resulting limitations.
A defensible protocol should report the chosen effect size, the rationale for selecting it, the statistical test, the target power and significance level, and the other assumptions affecting the calculation.
When uncertainty is substantial, sample-size sensitivity analysis can show how dependent the proposed design is on the assumptions.
07 · A Quick Checklist
Before Selecting an Effect Size for Sample-Size Planning
Before finalizing your power calculation, check:
Define whether the primary objective is detection, estimation, equivalence, noninferiority, or demonstrating an effect beyond a threshold.
State the expected effect and identify the evidence supporting that prediction.
Determine whether a substantively justified SESOI can be specified independently of sample-size convenience.
Use an effect-size scale consistent with the outcome, statistical model, and substantive interpretation.
Calculate power or sample size under both the expected effect and the SESOI when relevant.
Account for clustering, unequal allocation, attrition, and other design features where applicable.
Document what meaningful effects the proposed sample may be unable to detect reliably.
Record the assumptions and rationale before observing the study's primary results.
