03 · What You Need to Know
How to Evaluate AI-Recommended Sample Size Formulas
Why Are There Different Sample Size Formulas?
Different statistical problems require different quantities of information.
Estimating a population proportion within a specified margin of error is not the same as detecting a difference between two means, estimating a correlation, or testing a regression coefficient.
Each objective has a different sampling distribution, variance structure, and relationship between sample size and uncertainty.
For example, a two-group experiment requires consideration of the allocation ratio and the variability of the outcome, while a descriptive survey may require an anticipated proportion and desired confidence interval width.
AI should identify the statistical objective before recommending a calculation method.
What Information Should Be Established Before Selecting a Formula?
| Information |
Why it matters |
| Primary research objective |
Determines whether the study seeks estimation, comparison, prediction, or another form of inference. |
| Outcome type |
Distinguishes continuous, binary, count, time-to-event, and other data structures. |
| Primary estimand or hypothesis |
Identifies the parameter or effect that the calculation must address. |
| Planned statistical analysis |
Determines which statistical model or testing framework is relevant. |
| Sampling or assignment design |
Identifies clustering, stratification, repeated measurements, and allocation requirements. |
| Planning assumptions |
Provides the effect size, variability, precision, power, or other values required by the calculation. |
Without this information, AI may produce a mathematically complete calculation that does not correspond to the intended study.
Which Sample Size Methods Are Commonly Used for Different Research Objectives?
| Research objective |
Common calculation approach |
Important inputs |
| Estimate one population proportion |
Precision-based proportion formula. |
Confidence level, anticipated proportion, desired margin of error, and sampling design. |
| Estimate one population mean |
Precision-based mean formula. |
Confidence level, anticipated standard deviation, and desired margin of error. |
| Compare two independent means |
Power- or precision-based two-group comparison. |
Target difference, outcome variability, allocation ratio, and relevant statistical criteria. |
| Compare two proportions |
Two-proportion power or precision calculation. |
Anticipated proportions, target difference, allocation ratio, and testing assumptions. |
| Estimate or test a correlation |
Correlation-specific precision or power calculation. |
Target correlation, desired precision or power, and relevant assumptions. |
| Test a regression effect |
Model-specific power analysis or simulation. |
Target coefficient or effect, predictor structure, residual variability, and model assumptions. |
| Analyze clustered or longitudinal data |
Design-specific analytical or simulation-based calculation. |
Cluster structure, correlations, timing, attrition, and intended analysis. |
These are methodological categories rather than universal formulas. Each category may contain several valid procedures depending on the exact study design.
What Formula Is Used to Estimate a Population Proportion?
For a conventional large-sample confidence interval for a population proportion under simple random sampling assumptions, a common starting formula is:
Single Proportion Formula
n₀ = Z² × p(1 − p) / e²
Where:
n₀ = initial sample size.
Z = normal critical value for the desired confidence level.
p = anticipated population proportion.
e = desired absolute margin of error.
For a 95% confidence level, p = 0.50, and e = 0.05, the initial result is approximately 384.16, conventionally rounded upward to 385.
The formula may be appropriate for a descriptive prevalence estimate under the stated assumptions.
It is not automatically appropriate for comparing two intervention groups or testing relationships between constructs.
When Is a Finite Population Correction Appropriate?
For a defined finite population sampled without replacement under appropriate probability sampling conditions, a common adjustment is:
Finite Population Adjustment
n = n₀ / [1 + (n₀ − 1) / N]
Where:
n = adjusted sample size.
n₀ = initial sample size.
N = finite population size.
For example, with N = 5,000 and n₀ approximately 384.16, the adjusted result is approximately 357.
However, the correction is not appropriate merely because the researcher knows how many people are enrolled at an institution.
Its use depends on the sampling design and intended population estimate.
AI should explain the justification rather than apply the correction automatically.
What Formula Is Used to Estimate a Population Mean?
For a conventional large-sample precision calculation under independent observations and an anticipated population standard deviation, a common starting expression is:
Single Mean Formula
n₀ = Z² × σ² / e²
Where:
σ = anticipated population standard deviation.
e = desired absolute margin of error for the mean.
The calculation depends on the variability of the outcome and the desired precision.
For example, estimating the average academic self-efficacy score requires an anticipated standard deviation or another defensible planning estimate.
AI should not substitute an arbitrary standard deviation simply because the researcher provides the scale's minimum and maximum possible scores.
Finite population and design adjustments may be necessary depending on the sampling procedure.
What Formula Is Used to Compare Two Independent Means?
For an illustrative two-sided comparison of independent group means with equal allocation, common variance, and a normal-approximation power calculation, a simplified expression is:
Two Independent Means: Approximate Per-Group Sample Size
n ≈ 2(Z₁₋α/₂ + Z₁₋β)²σ² / Δ²
Where:
n = required sample size per group.
α = significance level.
1 − β = desired statistical power.
σ = anticipated common outcome standard deviation.
Δ = target difference between group means.
The expression assumes a particular independent-group comparison and uses a normal approximation. Exact or alternative procedures may produce different requirements.
It should not be applied unchanged to paired measurements, cluster-randomized trials, or other designs with different variance structures.
Researchers should verify whether the planned analysis corresponds to the calculation.
Why Does the Allocation Ratio Matter?
In comparative studies, the number of participants assigned to each condition influences the precision of the estimated difference.
Equal allocation is often statistically efficient for comparing two groups when observations have similar costs and outcome variances.
However, unequal allocation may be justified by ethical, operational, or other design considerations.
A formula assuming equal group sizes may be inappropriate when the planned ratio is 2:1 or another unequal distribution.
AI should therefore request the allocation ratio rather than assume that every experimental study uses equal-sized groups.
What Formula Is Used for Correlation Studies?
Sample-size planning for correlation may involve testing whether a population correlation differs from a specified value or estimating a correlation with desired precision.
Under common bivariate normal assumptions, Fisher's z transformation provides a basis for approximate calculations.
For testing a correlation against zero, an illustrative approximation is:
Correlation Power Approximation
n ≈ 3 + (Z₁₋α/₂ + Z₁₋β)² / [atanh(ρ)]²
Where:
ρ = anticipated population correlation.
atanh(ρ) = Fisher's z transformation of the correlation.
The approximation is not a universal correlation sample-size formula. Its suitability depends on the hypothesis, assumptions, and selected testing procedure.
For example, planning to estimate a correlation within a desired confidence interval width requires a different precision-oriented calculation.
AI should distinguish correlation estimation from hypothesis testing before selecting the method.
Can AI Select a Formula for Multiple Regression?
AI may identify model-specific power or precision methods for regression analysis.
However, multiple regression does not have one universal sample-size formula suitable for every research question.
For example, testing whether a group of predictors explains additional variance may involve an F-test-based power analysis.
Testing one regression coefficient may require assumptions about the predictor's relationship with other covariates and the residual variability of the outcome.
Detecting a small interaction effect may require substantially more information than estimating a larger main effect.
Researchers should specify the primary regression hypothesis rather than request a sample size for multiple regression in general.
Why Are Fixed Respondent-to-Variable Rules Problematic?
AI may recommend collecting five, ten, or twenty participants per predictor or questionnaire item.
Such rules of thumb can be misleading when treated as universal requirements.
For regression, sample adequacy depends on the intended effects, predictor relationships, outcome variability, estimation method, and other assumptions.
For factor analysis and structural equation modeling, the measurement model, factor loadings, communalities, identification, and estimator may substantially affect sample requirements.
Wolf and colleagues (2013) demonstrate through simulation that sample-size requirements in structural equation modeling vary considerably with model characteristics.
AI should therefore avoid presenting a fixed participant-to-item ratio as proof of adequate sample size.
Can AI Select a Formula for Structural Equation Modeling?
AI can explain candidate approaches, including simulation-based sample-size planning.
However, structural equation modeling involves different models, estimation methods, parameters, and inferential objectives.
A model involving a small number of well-measured latent variables may have different requirements from a complex model with weak factor loadings and numerous parameters.
Monte Carlo simulation can be useful when researchers need to evaluate power, precision, convergence, or other performance characteristics under a specified model.
However, simulation results are only as defensible as the assumptions used to generate them.
AI may assist with simulation code, but the model specification and implementation require independent verification.
Can AI Choose a Formula for Cluster-Randomized Studies?
Cluster-randomized studies require attention to the number of independent clusters, cluster sizes, intraclass correlation, allocation, and the intended analytical method.
AI may suggest multiplying an individually randomized sample-size estimate by a design effect.
Such approximations can be useful under particular assumptions, but they are not universally adequate.
For example, a study with only two classrooms cannot be made into a conventionally well-powered cluster-randomized trial simply by recruiting more students within those classrooms.
The number of independent allocation units remains a critical constraint.
Researchers should use a design-specific calculation and verify whether the proposed design effect is appropriate.
Can AI Select a Formula for Repeated-Measures Studies?
Repeated-measures studies require consideration of within-participant correlation, number and timing of measurements, expected missingness, and the planned analytical model.
For example, comparing pretest and posttest scores within participants differs from comparing repeated trajectories across intervention groups.
The two designs may require different calculations because the relevant estimands and variance structures differ.
AI should not assume that adding more measurement occasions automatically reduces the required number of participants.
The benefit of repeated observations depends on their correlation structure and the intended analysis.
What About Qualitative Research?
Qualitative research generally does not use the same sample-size formulas as conventional quantitative estimation or hypothesis testing.
For example, a qualitative interview study may justify its sample through information power, methodological purpose, participant specificity, or an appropriate approach to saturation.
Malterud, Siersma, and Guassora (2016) describe information power as one framework for evaluating sample adequacy in qualitative interview studies.
AI should not select a survey formula for qualitative research merely because the researcher provides the total population size.
The appropriate approach depends on the qualitative tradition and the kind of evidence required.
Can AI Determine the Correct Formula From the Statistical Test Name Alone?
Not always. A statistical test name may leave important details unresolved.
For example, a researcher may state that the analysis will use a t-test without specifying whether the comparison involves independent groups, paired observations, or another structure.
Similarly, regression may refer to different models and hypotheses.
AI should ask clarifying questions about the outcome, estimand, design, and assumptions before selecting a calculation method.
Choosing a formula based only on a familiar statistical term risks matching the wrong calculation to the study.
Can the Same Research Question Have Different Valid Sample Size Calculations?
Yes. Different planning approaches may be defensible when they address different inferential objectives.
For example, a researcher comparing two instructional conditions might plan the sample to achieve a specified statistical power for a meaningful effect.
Alternatively, the researcher might prioritize estimating the intervention effect with a desired confidence interval width.
Both approaches may be appropriate, but they answer different planning questions and may produce different required sample sizes.
Lakens (2022) discusses several approaches to sample size justification, including power analysis, accuracy in parameter estimation, and other considerations.
AI should explain which justification is being used rather than imply that only one formula can ever be correct.
How Can You Verify an AI-Selected Formula?
1. Identify the primary estimand. Specify the parameter or effect the study intends to estimate or test.
2. Confirm the outcome and data structure. Determine whether observations are continuous, binary, paired, clustered, repeated, or otherwise structured.
3. Examine the planned analysis. Establish which statistical model or inferential procedure will address the primary question.
4. Identify candidate calculation methods. Compare formulas or computational approaches that correspond to the objective.
5. Review assumptions. Verify effect sizes, variability, precision, power, allocation, and dependence as applicable.
6. Check design adjustments. Determine whether clustering, finite population correction, attrition, or other modifications are necessary.
7. Reproduce the calculation. Use appropriate statistical software or a transparent computational procedure.
8. Document the rationale. Explain why the selected method is appropriate and what assumptions determine the result.
The correct formula should follow from the statistical problem rather than from the convenience of a familiar equation.
Watch Out
Do not use a sample-size formula merely because it is widely cited, easy to calculate, or commonly recommended by AI. A formula is appropriate only when its statistical objective and assumptions correspond to the actual research design.
