Should Sample Size Be Based on Precision Rather Than Statistical Power?

Should You Calculate Sample Size to Detect an Effect or Estimate It Precisely?

A researcher wants to estimate the prevalence of academic burnout among university students. Another wants to determine whether an instructional intervention improves examination performance. Both need to justify their sample sizes, but do they need the same type of calculation?

Statistical power analysis is frequently treated as the default approach to sample-size planning. Yet not every study is designed primarily to reject a null hypothesis. Some studies seek to estimate a population characteristic, quantify a relationship, or determine the likely magnitude of an effect with sufficient precision to support interpretation.

The distinction matters because a study can have adequate power for a particular hypothesis test while producing an estimate too imprecise for a practical decision. Conversely, a study designed for a sufficiently narrow confidence interval may have no need for a conventional superiority-test power calculation.

Base Sample Size on the Study's Primary Inferential Objective

How Precision-Based and Power-Based Sample-Size Planning Differ

What Does Power-Based Sample-Size Planning Ask?

Power-based planning determines the sample size required for a statistical test to reject a specified null hypothesis with a chosen probability when a particular alternative is true.

Statistical power is the probability of rejecting the null hypothesis under the assumed alternative and statistical model. It is commonly expressed as 1 − β, where β represents the probability of a Type II error for that alternative.

For example, a randomized study might be designed to have 80% power to detect a mean difference of five examination points using a two-sided significance level of 0.05.

This does not mean that the study has an 80% probability of finding the true answer, or that it has 80% power for every nonzero effect. Power changes with the true effect magnitude and other population parameters.

Power analysis is particularly useful when a hypothesis test is central to the study's intended inference.

What Does Precision-Based Sample-Size Planning Ask?

Precision-based planning determines the sample size required to estimate a parameter with a specified degree of uncertainty.

The target is often expressed as a confidence-interval width or margin of error. For example, a researcher might want to estimate the prevalence of a characteristic with a 95% confidence interval having an anticipated half-width of no more than five percentage points.

Instead of asking whether a test will reject a null hypothesis, the researcher asks whether the resulting interval is expected to be sufficiently informative.

The desired precision should be justified by the research question. A five-percentage-point margin of error may be useful for one institutional survey but too broad for a rare condition or a consequential policy threshold.

The question of how narrow a confidence interval needs to be is therefore substantive as well as statistical.

When Is Precision the More Appropriate Primary Target?

Precision-based planning is particularly suitable for descriptive and estimation-focused studies. Examples include estimating a population mean, prevalence, correlation, or another parameter whose numerical magnitude is itself the principal finding.

Consider a survey estimating the proportion of teachers who use a particular educational technology. The researcher may not have a scientifically meaningful null hypothesis that the proportion equals a particular arbitrary value.

Instead, the study's usefulness may depend on whether the estimate is sufficiently precise to inform institutional planning.

Precision can also be the primary objective when a study aims to estimate an intervention effect rather than make a binary superiority decision. This may be particularly appropriate in exploratory or early-stage research, provided the target interval width is justified.

However, a precise estimate is not automatically unbiased or generalizable. Sampling quality, measurement validity, and model assumptions remain essential.

When Is Statistical Power the More Appropriate Primary Target?

Power-based planning is commonly appropriate for confirmatory studies that test a prespecified hypothesis.

For example, a randomized experiment may seek to determine whether an intervention improves an outcome relative to a control condition. The study can be designed to achieve a specified power at a scientifically meaningful alternative effect.

The effect selected for planning should be justified. It might reflect an anticipated effect supported by previous research or a smallest effect size of interest that the study must be capable of detecting.

Power calculations remain conditional on assumptions about effect magnitude, variability, allocation, and the statistical model.

Consequently, a study with nominally adequate power may perform less favorably when the actual population parameters differ from those assumed.

How Do the Two Approaches Compare?

How Is Sample Size Calculated for Precision?

For estimating a population mean from independent observations, with an assumed population standard deviation and a normal approximation, the required sample size can be expressed as:

The required sample increases as the desired margin of error decreases. Halving the target half-width approximately quadruples the required sample size under this formula.

For a population proportion, the calculation instead incorporates p(1 − p), where p represents anticipated prevalence. Other estimands require different variance expressions and interval procedures.

How Is Sample Size Calculated for Statistical Power?

For a two-sided comparison of two independent means with equal allocation and a common standard deviation, a conventional normal-approximation formula is:

Can a Study Have Adequate Power but Poor Precision?

Yes. Power and estimation precision are related through sample size and variability, but they are not interchangeable.

A study may be designed to reject a zero-effect null hypothesis with 80% power when the true effect equals a specified target. Yet the resulting confidence interval may remain broad enough to include effects with substantially different practical consequences.

For example, an intervention may show a statistically significant improvement while its confidence interval includes both modest and substantial benefits.

Statistical significance establishes neither the exact magnitude of the effect nor whether the improvement is sufficient to justify implementation.

This is particularly important when decisions depend on whether the effect exceeds a substantive threshold rather than merely differs from zero.

Can a Precision-Based Study Also Test Hypotheses?

Yes. A study designed for estimation may also report hypothesis tests, provided those analyses are appropriate and transparently described.

However, the study's sample size was not necessarily selected to achieve a particular power for those tests.

Similarly, a study designed primarily for power should ordinarily report effect estimates and confidence intervals, even though its recruitment target was determined through hypothesis-testing criteria.

The planning framework determines the principal design justification, not every statistical analysis that may be conducted.

Should Both Power and Precision Be Evaluated?

Sometimes both are important. A study may need sufficient power to test its primary hypothesis while also requiring a confidence interval narrow enough to inform a practical decision.

In such cases, researchers can calculate the sample-size requirement under each criterion and consider a sample that satisfies both, assuming the criteria are appropriate and feasible.

However, the objectives must be clearly defined. A conventional superiority calculation and a precision calculation do not necessarily address the same scientific question.

Researchers should also distinguish expected confidence-interval width from a probability guarantee that the realized width will remain below a target. The latter requires an assurance-type or other appropriate probabilistic calculation.

What About Equivalence and Noninferiority Studies?

Equivalence and noninferiority studies typically use prespecified margins representing differences considered acceptable or negligible for a particular purpose.

Their sample-size calculations often use power-based methods, but the hypotheses differ from conventional superiority testing.

For example, equivalence testing may seek evidence that a population effect lies within prespecified lower and upper bounds. A nonsignificant superiority test does not establish equivalence.

These designs illustrate why choosing between precision and power cannot be reduced to a universal rule. The appropriate calculation follows the intended inference.

How Do Uncertain Planning Assumptions Affect Both Approaches?

Precision-based and power-based calculations both depend on assumptions that may be uncertain before data collection.

For a mean-estimation study, the population standard deviation may be unknown. For a prevalence survey, the anticipated population proportion may be uncertain. For a superiority test, both the target effect and variability may require justification.

Researchers can examine plausible alternative assumptions rather than relying on one unsupported value.

A sample-size sensitivity analysis can then show how the proposed design performs when important assumptions change.

Such analysis is useful for both planning frameworks, although the performance measure being examined may differ.

Two Educational Studies That Need Different Sample-Size Justifications

Neither calculation includes attrition or complex sampling adjustments. The first assumes independent observations from the target population, while the second assumes independent groups with equal allocation and a common standard deviation.

If the first study requires estimates within individual academic programs, or the second randomizes entire classrooms, additional design considerations would change the calculations.

Misconceptions About Precision and Statistical Power

Choose the Planning Criterion Before Opening Sample-Size Software

Begin by writing a sentence that describes what the primary analysis must establish. Avoid choosing a sample-size method simply because a familiar calculator provides it.

If the study is intended to estimate a parameter, specify the degree of precision needed for the estimate to be useful. If the study is intended to test a hypothesis, specify the relevant alternative and desired operating characteristics.

For a prevalence survey, determining how precise the prevalence estimate needs to be may be more useful than asking which difference from an arbitrary null proportion the study should detect.

For a continuous-outcome estimation study, the relevant calculation may involve the sample size required to estimate a population mean.

Whatever the approach, the final sample-size justification should identify the estimand, the statistical method, the planning assumptions, and the reason the resulting information is sufficient for the research objective.

Before Choosing Between Precision and Power

Frequently Asked Questions About Precision-Based and Power-Based Planning

Plan the Sample Around the Information the Study Must Produce

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.