Should You Use an Effect Size From a Previous Study With a Different Population?

Can You Borrow an Effect Size From Research Conducted in Another Population?

You are planning a study and find a previous article investigating almost the same research question. It reports an effect size that appears suitable for your sample-size calculation.

There is one complication: the previous study involved a different population.

Perhaps its participants were university students, while yours will be secondary school students. Perhaps the earlier intervention was evaluated in a high-income country, while your study will take place in a lower-resource setting. Or perhaps the previous study involved experienced professionals, while your participants will be beginners.

Can you use the published effect size anyway? The answer depends on more than whether the two studies investigate similar topics.

A Previous Effect Size May Be Useful, but Its Transferability Must Be Justified

How Population Differences Affect the Use of Published Effect Sizes

Why Does the Population Matter When Borrowing an Effect Size?

An effect size summarizes the magnitude of an effect or association under particular study conditions. Its relevance to another study depends partly on whether those conditions are sufficiently comparable.

Consider a digital learning intervention that improves achievement among university students. The same intervention might produce a different effect among younger students because of differences in prior knowledge, self-regulation, digital literacy, instructional support, or access to technology.

These differences do not establish that the effect will change. They identify reasons why it might.

More generally, an effect observed in one population may not represent the effect that would occur in another. This concern relates to external validity and transportability: whether findings obtained under one set of conditions can support conclusions about another target population.

For sample-size planning, the immediate concern is narrower. You need to determine whether the published effect is a defensible numerical assumption for the effect your proposed study is intended to detect.

Which Population Differences Are Most Important?

Not every demographic or contextual difference is consequential. The relevant question is whether a difference could plausibly alter the magnitude of the effect, the outcome variability, or the relationship being investigated.

Potentially important differences include:

• Baseline characteristics: prior achievement, disease severity, existing skills, or other characteristics that influence the outcome.
• Effect modifiers: characteristics that change the magnitude or direction of an intervention effect or association.
• Intervention implementation: differences in intensity, duration, fidelity, instructional support, or access to resources.
• Outcome measurement: differences in instruments, measurement timing, scoring, or operational definitions.
• Study context: institutional practices, service availability, educational conditions, or other contextual influences.
• Sampling and selection: differences in eligibility criteria, recruitment processes, and the composition of participants.

For example, a difference in participants' average age may be relatively unimportant for one intervention but central to another. Researchers should identify a plausible mechanism linking the population difference to the expected effect rather than assuming every difference is equally serious.

Distinguish Effect Modification From Differences in Baseline Risk

A particularly important distinction concerns baseline outcome levels and effect modification.

Two populations may have different baseline risks or average outcomes without necessarily having different treatment effects on every measurement scale.

Conversely, an intervention may genuinely work differently across populations because a characteristic modifies its effect.

For binary outcomes, the distinction becomes especially important because risk differences, risk ratios, and odds ratios may behave differently across populations.

For example, even if a risk ratio remains constant, the corresponding absolute risk difference can change when baseline risk changes. Odds ratios introduce additional complications because they are noncollapsible, meaning marginal and conditional odds ratios can differ even without conventional confounding.

Consequently, a previous study's reported effect metric must be examined before deciding whether its numerical value can be transferred.

Why Standardized Effect Sizes May Not Transfer Directly

Researchers often assume that standardized effect sizes are inherently comparable because they do not use the original measurement units.

That assumption can be misleading.

For two independent groups, Cohen's d is commonly expressed as the difference between group means divided by a standard deviation, typically a pooled within-group standard deviation under the relevant assumptions.

Now consider a second population in which the same six-point difference occurs but the pooled standard deviation is 20 points. Its standardized difference would be d = 0.30.

The raw effect is identical, but the standardized effect differs because the populations have different outcome variability.

This matters for sample-size calculations because the standardized effect combines information about the expected difference and variability. A value borrowed from another population may misrepresent one or both components.

Whenever possible, examine the original mean difference and standard deviation separately instead of relying exclusively on a reported standardized effect.

Does Using the Same Instrument Make the Effect Size Transferable?

Using the same instrument can improve comparability, but it does not guarantee equivalent measurement or outcome variability.

A questionnaire may function differently across age groups, languages, cultures, or levels of prior experience. Even when the instrument measures the same construct, score distributions may differ.

For instance, an achievement test that distinguishes effectively among novice learners may produce ceiling effects among advanced students. Such restrictions can affect observed variability and potentially the estimated intervention effect.

Researchers should therefore examine whether the instrument has suitable measurement properties in the target population and whether the reported outcome statistics are relevant to their proposed analysis.

Measurement comparability is especially important when the borrowed effect concerns a latent construct such as motivation, anxiety, engagement, or self-efficacy.

How Much Confidence Should You Place in a Single Previous Study?

Even a study involving a highly comparable population may provide an imprecise effect estimate.

A point estimate is only one value from a sampling distribution. Its uncertainty matters, particularly when the original sample was small.

Before using the published effect, examine the study's confidence interval, sample size, research design, risk of bias, and outcome measurement.

Be cautious when the earlier study was selected primarily because it reported a statistically significant or unusually large effect. Such estimates may be exaggerated relative to the underlying population effect, particularly in small studies and selectively reported literatures.

Borrowing an estimate from a different population adds another layer of uncertainty to the uncertainty already present in the original estimate.

If the expected effect remains uncertain, the appropriate response is to acknowledge and investigate that uncertainty rather than treating the published point estimate as established truth.

Can a Meta-Analysis Provide a Better Planning Estimate?

A systematic review or meta-analysis may provide useful evidence about the distribution of effects across studies, particularly when multiple studies examine related interventions and outcomes.

However, a pooled effect is not necessarily the effect expected in your specific population.

Meta-analyses may combine studies with different participant characteristics, interventions, outcome measures, and methodological quality. A pooled estimate summarizes effects under the assumptions of the chosen synthesis model.

When between-study heterogeneity is substantial, the average effect may be a poor representation of what a new study will observe.

A prediction interval, when appropriately estimated and interpreted, can help describe the range of underlying effects that might occur in a comparable future study under a random-effects model. Such intervals can be wide and are sensitive to the available evidence and modeling assumptions.

Subgroup analyses and meta-regression may also inform judgments about population differences, but they can be underpowered and susceptible to ecological bias or confounding. They should not automatically be interpreted as demonstrating causal effect modification.

The strongest evidence for planning is not necessarily the largest or most precise pooled estimate. It is the evidence most relevant to the intended study and sufficiently credible for the planning decision.

When Should You Adjust a Borrowed Effect Size?

Sometimes researchers reduce a previously reported effect to create a more conservative planning assumption.

For example, a published standardized effect of 0.60 might be accompanied by sample-size calculations for 0.40 and 0.30.

This can be useful as sensitivity analysis, but the adjustment should not be presented as a statistically validated correction unless an appropriate method and evidence support it.

There is no universal rule requiring researchers to reduce every borrowed effect by a fixed percentage. Nor is there a general formula that converts an effect from one population into the correct effect for another.

Where transportability methods are applicable, individual-level data and information about relevant effect modifiers may support formal adjustment or standardization to a target population. Such approaches require assumptions, including sufficient overlap between the source and target populations, and may not be feasible when only published aggregate results are available.

For ordinary sample-size planning, it may be more practical to use several explicitly justified effect-size scenarios rather than claiming to have precisely corrected a borrowed estimate.

Should You Use the Smallest Meaningful Effect Instead?

If a substantively meaningful threshold can be established, it may provide a more defensible basis for planning than an uncertain estimate from a different population.

For example, a university might regard a four-point improvement in a validated achievement measure as sufficient to justify an instructional intervention, regardless of whether a previous study in another setting reported an eight-point improvement.

The choice between an expected effect and the smallest effect of interest should follow the intended inference.

If the study is designed to detect an effect that would justify implementation, planning around that threshold may be appropriate. If the objective is to evaluate a specific theoretical prediction about effect magnitude, an expected-effect approach may instead be relevant.

Either approach requires justification. A meaningful threshold is not automatically objective merely because it is expressed numerically.

Why Study Design Must Also Be Comparable

Population similarity is only one part of effect-size transferability.

A previous study may report a difference between independent groups, while the proposed study uses a pretest-posttest design, repeated measures, clustered sampling, or covariate adjustment.

The effect-size metric and its relationship to statistical power may differ across these designs.

For instance, a paired comparison depends partly on the variability of within-person differences. A cluster-randomized study must account for dependence among participants within clusters. An analysis of covariance may depend on the predictive contribution of baseline measurements.

Even if the anticipated intervention effect is comparable, the sample-size calculation must reflect the proposed design and analysis rather than reproduce the previous study's calculation uncritically.

Borrowing an Educational Intervention Effect Across Student Populations

Before selecting the effect size, the researcher identifies several uncertainties: differences in learners' prior knowledge, independence in completing online activities, institutional support, and achievement-score variability.

None proves that the effect will be smaller. Nevertheless, the previous estimate cannot be assumed to apply unchanged.

The researcher calculates sample-size requirements for three scenarios, assuming an independent two-sample, two-sided t-test, equal allocation, a significance level of 0.05, and 80% power.

These figures are rounded illustrations of conventional power calculations under the stated assumptions. They do not include attrition, clustering, or other adjustments. The smaller effect scenarios are hypothetical, not empirically established corrections for the population difference.

The calculations reveal that relying on d = 0.60 could lead to a considerably smaller planned sample than would be required to detect d = 0.30 with the same target power.

Suppose stakeholders independently determine that an improvement corresponding to d = 0.30 would be educationally meaningful, based on a defensible estimate of outcome variability.

The researcher could then use d = 0.30 as the primary planning target, while documenting the published estimate and presenting the larger-effect scenarios as supporting sensitivity analyses.

Alternatively, if the available sample is restricted to 200 participants, the researcher could report the design's approximate 80% power for d = 0.40 and evaluate its lower power for d = 0.30.

The result is a transparent planning decision rather than an unsupported claim that the earlier study's effect will necessarily recur.

Common Misinterpretations When Borrowing Effect Sizes

How to Decide Whether a Published Effect Size Is Suitable for Your Study

Evaluate a borrowed effect size as a piece of evidence rather than a ready-made numerical input. Its suitability depends on the source study's credibility, the relevance of its population, and the inferential purpose of your planned research.

In the research protocol, identify the original study and explain why its effect estimate was considered relevant. Report the actual numerical value, its metric, and any transformation used to match the proposed analysis.

If alternative scenarios were examined, explain how the primary planning assumption was selected. The final justification should allow another researcher to reproduce the calculation and understand the uncertainty surrounding it.

Before Using an Effect Size From a Different Population, Check:

Frequently Asked Questions About Transferring Effect Sizes

Borrow the Evidence, Not the Assumption of Identical Effects

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.