What Should You Do When Outcome Variability Is Unknown for Sample-Size Planning?

How Can You Calculate Sample Size When You Do Not Know the Outcome's Standard Deviation?

You are planning a study that will compare average examination scores, measure changes in psychological well-being, or estimate the mean value of a continuous outcome. You know what difference would matter, and you have selected a significance level or desired confidence-interval width.

However, the sample-size calculation also requires information about how much the outcome varies among participants. You do not yet have data from the population you intend to study.

Should you borrow a standard deviation from another study, estimate one through a pilot, assume a convenient value, or calculate sample sizes under several plausible scenarios?

The answer matters because an incorrect variability assumption can substantially change the number of participants needed to achieve the intended power or precision.

Use the Best Available Variability Evidence and Examine Its Uncertainty

Understanding Variability Assumptions in Sample-Size Calculations

What Does Outcome Variability Mean?

Outcome variability describes how much the values of a measured variable differ across observations in a population or within a statistical model.

For continuous outcomes, variability is commonly summarized using the variance or standard deviation. The variance measures average squared deviation from the population mean, while the standard deviation is its square root and is expressed in the outcome's original units.

For example, two groups of students might have the same average examination score of 75 points but different standard deviations. Scores in one group may be concentrated near 75, whereas scores in the other may be much more dispersed.

The same five-point difference between group means will generally be easier to detect statistically when the relevant outcome variability is smaller, provided other design features remain unchanged.

Why Does the Standard Deviation Affect Required Sample Size?

In many conventional calculations involving continuous outcomes, the required sample size increases with the variance of the outcome.

Consider a two-sided comparison of two independent population means with equal group sizes and an assumed common standard deviation.

The squared relationship explains why uncertainty about standard deviation deserves serious attention. A modest error in the assumed standard deviation can have a larger proportional effect on the calculated sample size.

Which Standard Deviation Does Your Study Actually Need?

The relevant variability parameter depends on the design and statistical analysis. Researchers should not assume that every sample-size formula requires the overall standard deviation of the outcome.

For example, in a paired study, using the standard deviation of baseline scores instead of the standard deviation of change scores may substantially misrepresent the sample-size requirement.

Likewise, the total variation among students across an entire university may not represent the residual variability relevant to an analysis that adjusts for baseline performance and institutional clustering.

Where Can You Obtain a Reasonable Standard Deviation?

Begin with research that measures the same or a closely comparable outcome in a relevant population.

Potential sources include previous published studies, systematic reviews, institutional datasets, established measurement studies, and pilot investigations.

When evaluating a reported standard deviation, examine the outcome scale, population characteristics, measurement timing, eligibility criteria, and analytical design.

A standard deviation from an intervention group after treatment may differ from the baseline standard deviation or the pooled within-group standard deviation required for a planned comparison.

Do not assume that a value is suitable merely because the study used the same questionnaire or examination.

Can You Use a Standard Deviation From a Previous Study?

Yes, provided that the previous study offers sufficiently comparable information.

For example, a researcher planning a study of undergraduate achievement may use variability estimates from a previous investigation involving the same assessment instrument and a similar student population.

However, differences in academic preparation, score distributions, assessment difficulty, and administration conditions may affect variability.

If several comparable studies report standard deviations, researchers can examine their consistency and consider an appropriate synthesis. A pooled estimate should account for relevant differences in study design and measurement rather than simply average unrelated numbers.

When prior evidence is heterogeneous, retaining a plausible range may be more defensible than choosing one apparently precise value.

Can a Pilot Study Estimate Outcome Variability?

A pilot study can provide an estimate of outcome standard deviation, particularly when comparable external data are unavailable.

However, the estimate itself is uncertain. Small pilot samples can produce standard deviations that are noticeably smaller or larger than the underlying population value.

Browne (1995) examined the use of pilot samples for sample-size determination and highlighted the need to account for uncertainty in preliminary variance estimates.

Julious (2005) discussed a commonly cited pilot sample-size rule of thumb, but such rules should not be interpreted as guaranteeing precise estimation of variance or treatment effects.

The distinction between estimating variability and estimating an intervention effect is important. Although pilot studies may be useful for both, the statistical properties and planning implications differ.

The limitations of using pilot effect sizes for main-study planning should not be assumed to apply identically to every nuisance parameter, but neither should pilot variance estimates be treated as known quantities.

How Uncertain Is a Pilot Standard Deviation?

Under independent sampling from a normally distributed population, the sample variance has a chi-square sampling distribution. This provides a basis for confidence intervals around the population variance.

For a sample of n observations, the conventional confidence interval for the population variance is:

This example demonstrates why a single pilot standard deviation may be an inadequate basis for a definitive sample-size decision.

For markedly nonnormal data, the chi-square interval may not have its nominal coverage. Alternative uncertainty methods should be considered where the model assumptions are inappropriate.

Should You Use the Upper Confidence Limit of the Standard Deviation?

Using an upper confidence limit for the standard deviation can be a conservative planning strategy in some settings because larger assumed variability generally increases the required sample size.

However, it is not a universal requirement. The appropriate allowance depends on the pilot size, desired assurance, distributional assumptions, and consequences of an underpowered study.

A very small pilot can produce an extremely wide upper confidence limit, resulting in a sample-size recommendation that may be impractical.

Researchers should explain the chosen uncertainty allowance and consider whether the resulting design is scientifically and operationally justified.

Can You Estimate Standard Deviation From a Measurement Scale's Range?

Knowing the possible minimum and maximum scores can sometimes provide mathematical bounds on variability, but it rarely establishes a reliable expected standard deviation.

For a variable restricted to the interval from a to b, the population variance cannot exceed (b − a)² / 4. Consequently, the population standard deviation cannot exceed (b − a) / 2.

This is a distribution-free upper bound, not a prediction of typical variability.

For example, a test scored from 0 to 100 has a maximum possible population standard deviation of 50 points. Actual score variability may be much smaller.

Using the maximum possible standard deviation as the planning assumption could therefore produce a very large sample-size requirement.

It is generally preferable to use empirical information about the score distribution when available.

Can Standardized Effect Sizes Eliminate the Need to Estimate Variability?

Standardized effect sizes express a raw effect relative to a measure of variability. For a common-standard-deviation mean comparison, the population standardized difference is:

d = Δ / σ.

If a researcher can justify a standardized target effect directly, a conventional power calculation may be performed without separately entering a raw mean difference and standard deviation.

However, standardization does not make variability irrelevant. The assumed standardized effect already contains an implicit relationship between the meaningful raw difference and the population standard deviation.

For example, a five-point improvement corresponds to d = 0.50 when σ = 10 but d = 0.25 when σ = 20.

Thus, substituting an arbitrary standardized benchmark does not resolve uncertainty about what a meaningful raw effect represents.

The smallest effect size of interest should be defined with attention to substantive meaning rather than chosen solely to avoid estimating variance.

How Can Sensitivity Analysis Address Unknown Variability?

When the standard deviation is uncertain, calculate the required sample size using several defensible values.

Suppose previous evidence suggests that σ might reasonably lie between 8 and 14 points. Rather than selecting 10 without explanation, calculate the sample size at 8, 10, 12, and 14.

This reveals how strongly the design depends on the variability assumption and helps determine whether the study remains feasible under less favorable conditions.

A sample-size sensitivity analysis is particularly valuable when the available variability evidence is sparse or inconsistent.

It may also be appropriate to evaluate uncertainty in the target effect, attrition, and other parameters rather than varying the standard deviation in isolation.

What If Outcome Variability Cannot Be Estimated Credibly?

Sometimes no suitable previous study, pilot dataset, or measurement information exists.

In that situation, researchers should avoid presenting a speculative standard deviation as an established fact.

Possible strategies include conducting a feasibility study, using a defensible range of assumptions, planning a study around an alternative outcome, or considering an adaptive design with prespecified nuisance-parameter reassessment.

Blinded sample-size reassessment can be useful in some randomized studies when outcome variability is initially uncertain. Such procedures require appropriate statistical planning and should not be confused with unrestricted changes to sample size after examining interim treatment effects.

If a conventional power calculation cannot be justified, researchers may need a different sample-size rationale that accurately reflects the exploratory or resource-constrained nature of the study.

Planning a Study When Examination Score Variability Is Uncertain

The table uses the approximate expression n ≈ 15.70σ² / 25, with values rounded upward. Exact sample-size calculations may differ.

If students are randomized within existing classes, the researcher must additionally examine classroom dependence. If the primary analysis adjusts for baseline examination scores, the relevant residual variability may differ from the unadjusted within-group standard deviation used in this illustration.

The example shows that uncertainty about variability is not merely a technical inconvenience. It can materially affect the scientific and logistical feasibility of a study.

Common Mistakes When the Outcome Standard Deviation Is Unknown

Build a Variability Assumption That Matches the Planned Analysis

The first task is not to find any available standard deviation. It is to identify the variability parameter required by the actual sample-size method.

For an independent-groups comparison, this may be a pooled within-group standard deviation. For a paired design, it may be the standard deviation of within-person changes. For an adjusted or multilevel model, other variance components may be relevant.

When variability uncertainty is substantial, the use of multiple plausible planning assumptions may be more informative than reporting one apparently precise calculation.

For estimation studies, the desired confidence-interval width should also be justified. Variability and precision jointly determine the required sample size.

The final protocol should make clear which standard deviation was assumed, where it came from, why it applies to the planned analysis, and how alternative values would affect the design.

Before Finalizing a Sample Size With Unknown Outcome Variability

Frequently Asked Questions About Unknown Outcome Variability

Unknown Variability Requires Transparent Assumptions, Not Arbitrary Numbers

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.