Can You Use a Range of Plausible Assumptions for Sample-Size Planning?

What If You Cannot Justify One Exact Value for Every Sample-Size Assumption?

You are planning a study and need to calculate the required sample size. Previous research suggests that the intervention effect could be modest or moderately large. Outcome variability also differs across published studies, and the proportion of participants who will complete the study remains uncertain.

Your sample-size software asks for specific numerical inputs. Yet choosing one value for each parameter may give an impression of certainty that the available evidence does not support.

Can you use a range of plausible assumptions instead? And if those assumptions produce substantially different sample sizes, how should you decide which number to use?

Yes, Plausible Ranges Can Make Sample-Size Planning More Defensible

How to Build and Use Plausible Sample-Size Assumptions

Why Is a Single Sample-Size Calculation Often Insufficient?

Sample-size calculations are conditional. Their results depend on the statistical model, research objective, and numerical assumptions entered into the calculation.

For a conventional superiority study, these assumptions may include the target effect, outcome variability, significance level, desired power, allocation ratio, and dependence among observations. Estimation studies instead require assumptions related to the desired precision and the variability of the estimator.

Some quantities, such as the nominal significance level or target power, are design choices. Others, including the population standard deviation and likely intervention effect, are unknown population characteristics.

Reporting one calculated sample size without explaining uncertainty in the underlying parameters can create false precision. A recommendation of 146 participants may look exact even when a reasonable change in one assumption would increase the requirement to 250.

The calculation is not necessarily wrong. Its interpretation becomes misleading when a conditional result is presented as an unconditional requirement.

What Makes an Assumption Plausible?

A plausible assumption is a value that can reasonably be defended using available evidence, substantive knowledge, or explicitly stated design considerations.

Sources may include comparable previous studies, systematic reviews, existing datasets, pilot investigations, measurement characteristics, or carefully elicited expert judgments.

Plausibility is not equivalent to mathematical possibility. An outcome standard deviation of 40 points may be mathematically possible on a 100-point scale, but it may be implausible for a relatively homogeneous student population using a particular assessment.

Conversely, an assumption should not be excluded merely because it produces an inconvenient sample size.

When prior evidence is sparse, researchers can acknowledge that a proposed range is exploratory or judgment-based rather than presenting it as empirically established.

Which Parameters Should Be Varied?

Vary parameters that are both uncertain and consequential for the design. Not every input requires a range.

The relevant parameters depend on the intended analysis. A paired study may require the standard deviation of within-person differences, while a clustered study may require information about the number and sizes of clusters.

Researchers should first identify the correct statistical model, then determine which uncertain inputs materially affect the sample-size result.

How Should You Choose the Lower and Upper Values?

Begin by reviewing the strongest relevant evidence. If comparable studies report standardized effects between 0.25 and 0.45, that interval may provide a starting point for considering plausible effects.

However, the minimum and maximum published estimates should not automatically become the planning bounds. Published estimates contain sampling error and may be affected by selective reporting, population differences, or methodological limitations.

A systematic review may provide information about between-study heterogeneity, but that heterogeneity should not be confused with uncertainty about the effect in the proposed population.

Researchers may also use a smallest effect size of interest to establish a substantively meaningful lower target for conventional superiority power analysis.

The range should be broad enough to represent credible uncertainty, but not so broad that it combines implausible conditions without explanation.

Should You Use Optimistic, Central, and Pessimistic Scenarios?

These labels can be useful when their meanings are explicitly defined.

An optimistic scenario might assume a relatively large intervention effect, lower outcome variability, and favorable retention. A less favorable scenario might assume a smaller effect, greater variability, and higher attrition.

However, scenarios should not be treated as equally likely simply because they appear in the same table. Nor should a central scenario automatically be described as the most probable without evidence.

It is often preferable to label scenarios according to their numerical assumptions, such as d = 0.30 or d = 0.40, rather than relying exclusively on qualitative descriptors.

How Do Plausible Effects Change Sample-Size Requirements?

Consider an equal-allocation, two-sided comparison of two independent means. Under a normal approximation, the sample size per group can be expressed in terms of a target standardized mean difference.

The range produces substantially different recruitment requirements. Rather than concealing that difference, the researcher can explain what each design would be capable of detecting under the assumed conditions.

Should You Vary Several Parameters at the Same Time?

Yes, when several uncertain parameters materially influence the calculation. However, their relationships require attention.

For example, a researcher might vary both the raw mean difference and outcome standard deviation. Combining the smallest plausible difference with the largest plausible standard deviation produces a demanding scenario.

That combination is useful only if it represents a credible joint possibility. Some parameters may be correlated, and independently combining their extremes may create unrealistic scenarios.

Researchers can use a scenario matrix to evaluate credible combinations or simulation to investigate more complex relationships.

For a standardized effect, avoid varying both d and its underlying raw difference and standard deviation independently without recognizing that d = Δ / σ. Doing so can inadvertently count the same uncertainty more than once.

What Is the Difference Between Scenario Analysis and Sensitivity Analysis?

Scenario analysis evaluates a set of specified combinations of assumptions. Sensitivity analysis examines how results change when assumptions vary.

In practice, the approaches overlap. A table of required sample sizes across plausible effects is a simple sensitivity analysis, while a comparison of several joint configurations is a scenario analysis.

The distinction is less important than making the assumptions and resulting consequences transparent.

A more systematic treatment of sample-size sensitivity analysis can help researchers decide which parameters deserve attention and how to interpret the results.

Can You Assign Probabilities to Different Assumptions?

Yes, but doing so introduces a different level of modeling.

Instead of considering only several fixed values, researchers can specify a probability distribution for an uncertain parameter and evaluate the performance of candidate sample sizes across that distribution.

For example, assurance-based planning may calculate the expected probability of meeting a statistical success criterion by averaging conditional power over a distribution of plausible effects.

This can be informative when credible prior information is available. Nevertheless, the resulting assurance depends on the chosen distribution and its assumptions.

A simple scenario table does not provide assurance merely because it includes several values. Without probabilities, it cannot establish how likely each scenario is.

Should You Always Choose the Largest Sample Size Across All Scenarios?

No. Selecting the maximum requirement can be appropriate when the least favorable scenario is credible and the consequences of insufficient power are serious.

However, it may also create an unnecessarily demanding study if the extreme scenario is unlikely, poorly justified, or inconsistent with the research objective.

Researchers should consider the value of detecting smaller effects, the ethical and financial costs of recruitment, the feasibility of alternative designs, and the consequences of uncertainty.

If the study cannot achieve its preferred operating characteristics under every plausible scenario, the protocol should explain which effects or conditions the proposed design adequately addresses.

How Should You Document Plausible Assumptions?

A transparent sample-size justification should identify the primary planning scenario, alternative values examined, sources supporting the assumptions, and the resulting differences in sample-size requirements or power.

It should also explain why the selected sample size is appropriate for the intended inference.

Reporting a sensitivity range does not require presenting every calculation in the main manuscript. Extensive tables or simulation results may be included in a protocol or supplementary material.

The key is that the final number should emerge from a defensible decision rather than an unexplained preference for whichever scenario is easiest to recruit.

Planning an Educational Experiment Under Several Plausible Effects

Completed sample sizes use the normal approximation, and recruitment targets are calculated by dividing the rounded completed sample by 0.90 and rounding upward. These targets are planning expectations, not guarantees of achieving the desired completed sample.

The example assumes independent observations. If students are assigned or taught in intact classes, the researcher must account for clustering. Similarly, an adjusted analysis may require different variance assumptions.

The purpose of examining several scenarios is not to discover a mathematically privileged sample size. It is to make the consequences of competing assumptions explicit before recruitment begins.

Misconceptions About Using Multiple Planning Assumptions

Use Plausible Scenarios to Make a Deliberate Sample-Size Decision

Start by distinguishing fixed design choices from uncertain population parameters. Then identify which uncertainties could materially change the study's ability to answer its research question.

Prioritize credible evidence and substantive importance over numerical convenience.

Where the main uncertainty concerns the effect size, distinguishing the expected effect from the smallest meaningful effect can prevent an optimistic prediction from becoming the sole justification for recruitment.

If outcome variability is uncertain, a separate assessment of plausible standard deviations may be warranted.

Ultimately, the chosen sample size should correspond to a clearly stated research objective and an acceptable level of uncertainty, not simply to the most convenient result in the scenario table.

Before Using Plausible Assumptions in Sample-Size Planning

Frequently Asked Questions About Plausible Sample-Size Assumptions

Uncertain Assumptions Should Be Examined, Not Hidden

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.