How Can Regression to the Mean Threaten Internal Validity?

Could an Apparent Improvement Occur Even If Your Intervention Had No Effect?

A university identifies students who performed exceptionally poorly on a diagnostic examination and enrolls them in an academic support program. Six weeks later, the students complete another assessment. Their average scores are substantially higher.

The improvement appears to demonstrate that the program worked. Yet some students might have scored unusually poorly the first time because of temporary illness, fatigue, unfamiliarity with the assessment, or ordinary measurement variation. Their subsequent scores could be closer to their typical performance even without the program.

This possibility is known as regression to the mean. It becomes particularly important when researchers select participants because of extreme initial scores and then interpret subsequent movement toward more typical values as evidence of an intervention effect.

Why Does Regression to the Mean Threaten Internal Validity?

Why Extreme Measurements Tend to Become Less Extreme

What Is Regression to the Mean?

Regression to the mean is a statistical phenomenon in which observations that are unusually high or low on one measurement tend, on average, to be closer to the population mean on a subsequent measurement, provided the measurements are not perfectly correlated and appropriate distributional conditions hold.

The principle is most easily understood by recognizing that an observed score may reflect a relatively stable underlying characteristic together with temporary influences.

A student's examination score, for example, may reflect academic proficiency, but it may also be affected by concentration, sleep, anxiety, question sampling, or other factors that vary between assessments.

A student with an exceptionally low observed score may have genuinely low proficiency, an unusually unfavorable combination of temporary influences, or both.

When the assessment is repeated, those temporary influences will not necessarily recur in the same direction or magnitude. Consequently, the second score may be less extreme.

The reverse applies to exceptionally high scores. A participant whose initial performance was unusually favorable may obtain a lower score when measured again.

Regression to the mean is a property of repeated observations and their statistical relationship. It is not evidence that a person is inevitably improving or deteriorating.

Why Does Selecting Participants With Extreme Scores Create a Problem?

Researchers frequently select participants because their baseline measurements exceed or fall below a threshold.

Examples include students with very low achievement scores, patients with unusually high symptom severity, employees with exceptionally poor performance ratings, or individuals with extreme physiological measurements.

These eligibility decisions may be appropriate for practical or ethical reasons. However, selection based on an extreme measurement creates a predictable interpretive challenge.

If the selected participants are measured again, some of the initial extremity may not persist. An apparent improvement can occur even if no effective intervention was delivered.

This is particularly problematic when the study has only one group and compares its baseline and follow-up means.

The researcher observes change but lacks an appropriate estimate of how much change would have occurred without the intervention.

Regression to the mean is therefore one of the alternative explanations that can undermine a causal comparison.

How Is Regression to the Mean Related to Measurement Reliability?

The degree of regression depends partly on how strongly repeated measurements are correlated.

When measurements are highly stable and strongly correlated, extreme observations tend to remain relatively extreme. When their correlation is weaker, greater movement toward the mean is expected on average.

Measurement error is one reason repeated observations may be imperfectly correlated, but it is not the only reason. Genuine short-term fluctuations can also contribute.

For example, blood pressure varies naturally across occasions, even when measured with a well-calibrated device. Similarly, academic performance may fluctuate because different assessments sample different knowledge or skills.

Improving measurement reliability may reduce some regression-to-the-mean effects, but it does not necessarily eliminate the phenomenon.

Researchers should also distinguish the reliability of an instrument from the stability of the underlying construct. A technically reliable instrument can measure a characteristic that genuinely fluctuates over time.

How Can Regression to the Mean Be Demonstrated Mathematically?

Under a bivariate normal model for two measurements with the same population mean and standard deviation, the expected second measurement conditional on the first is determined by their correlation.

The formula illustrates why an extreme initial score need not be repeated. The second measurement is expected to be less extreme when the correlation is positive but below one.

It does not predict every individual's score. Some participants will remain extreme, others may become more extreme, and some may move substantially toward or beyond the mean.

Nor does the calculation establish how much change in a real intervention study is attributable to regression to the mean. That requires evidence about the measurement process, untreated outcomes, and the research design.

Does Regression to the Mean Require an Unreliable Instrument?

No. Regression to the mean can occur even when researchers use carefully developed instruments.

Repeated measurements may differ because the underlying phenomenon fluctuates naturally. A reliable assessment can still capture temporary variation in performance, symptoms, or other characteristics.

However, greater measurement error can increase the degree of regression when other conditions remain comparable.

Researchers should therefore consider both instrument quality and the natural stability of the measured characteristic.

How Is Regression to the Mean Different From Maturation?

Regression to the mean and maturation may both produce apparent changes between baseline and follow-up, but their mechanisms differ.

A student may score higher on a second examination because the initial score was unusually low relative to their typical performance. That is consistent with regression to the mean.

The student may also improve because their knowledge genuinely develops during the intervening weeks. That is a possible maturation-related explanation.

Both processes can operate simultaneously. Researchers should not assume that all observed improvement belongs to one category.

How Is Regression to the Mean Different From a History Effect?

A history effect involves an event or circumstance outside the intervention that influences the outcome during the study period.

For example, students selected for academic remediation may also receive additional tutoring from another program. Their improvement could partly reflect that separate intervention.

Regression to the mean does not require such an event. It can occur because of the statistical relationship between repeated measurements.

In practice, events occurring between baseline and follow-up may combine with regression to the mean, making a single-group comparison especially difficult to interpret.

Can Regression to the Mean Affect Studies With Comparison Groups?

Yes. A comparison group does not automatically eliminate regression-to-the-mean concerns.

Suppose an intervention group is selected because its baseline scores are exceptionally low, while a comparison group is selected without the same criterion.

The intervention group may show greater subsequent improvement partly because its initial measurements were more extreme.

Comparing changes between these groups could therefore produce a misleading estimate.

A more credible design may apply comparable eligibility criteria to both groups and assign eligible participants randomly where feasible.

When groups are formed nonrandomly, researchers must examine whether selection differences between the groups could generate different regression-to-the-mean patterns.

Does Random Assignment Protect Against Regression to the Mean?

Random assignment can provide strong protection when participants are selected using the same eligibility procedure before randomization and then allocated appropriately to intervention and comparison conditions.

Both groups may experience regression to the mean, but randomization makes their potential outcome distributions comparable in expectation.

Consequently, the randomized between-group contrast can estimate the effect of assignment without mistaking the shared tendency toward the mean for an intervention effect, subject to the other requirements of a valid trial.

Randomization does not guarantee identical observed changes in every finite sample. Chance differences, missing outcomes, and postrandomization selection may still affect results.

The important distinction is between interpreting improvement within one group and comparing outcomes across appropriately randomized groups.

Can Multiple Baseline Measurements Help?

Collecting repeated baseline measurements may help researchers characterize participants' typical performance and identify unusually extreme observations.

For example, averaging several appropriately timed assessments may reduce the influence of temporary measurement variation when estimating a relatively stable underlying characteristic.

However, repeated baseline measurements are not automatically a solution. The underlying outcome may change during the baseline period, repeated testing may influence performance, and the average may still contain measurement error.

Researchers should consider the measurement schedule, stability of the construct, and purpose of the intervention before deciding how baseline scores will be used.

When a study recruits participants based on a single extreme score, obtaining additional preintervention observations may be particularly informative.

Can ANCOVA or Change-Score Analysis Correct Regression to the Mean?

Baseline-adjusted analysis of covariance, change-score analysis, and other longitudinal methods can produce different estimates when groups have different baseline distributions.

None should be treated as a universal statistical correction.

In randomized trials, prespecified baseline adjustment can improve precision under appropriate assumptions. In nonrandomized studies, the choice between adjusting for baseline outcomes and analyzing change requires attention to the causal structure, measurement error, and the processes determining group assignment.

Simply subtracting baseline scores from follow-up scores does not remove regression to the mean. Indeed, extreme baseline scores can be mechanically associated with subsequent change scores.

Researchers should prioritize a credible design and choose analytical methods that match the estimand and assumptions.

When Is Regression to the Mean Most Consequential?

The threat deserves particular attention when participants are selected because of extreme measurements, repeated scores are imperfectly correlated, and the study lacks a suitable comparison condition.

It may also be important when groups have substantially different baseline distributions or when interventions are introduced in response to unusually poor performance.

The latter situation is common in practice. Schools introduce remediation after examination results decline, organizations implement corrective programs after performance deteriorates, and clinicians intensify treatment when symptoms become unusually severe.

Subsequent improvement may be encouraging, but the timing of the intervention alone does not establish its effectiveness.

How Regression to the Mean Can Make an Academic Intervention Appear Effective

Now suppose the researchers conduct a more informative evaluation in which eligible students are randomly assigned to receive the additional program or continue with the existing support available to all students.

The example does not establish that regression to the mean caused exactly ten points of improvement. The comparison group's change may have several explanations.

Nor does the three-point difference automatically establish a statistically significant or practically important effect. Sampling uncertainty, outcome measurement, adherence, and other design features still require evaluation.

The central lesson is that selecting participants because of extreme scores makes a single-group improvement particularly vulnerable to misleading causal interpretation.

Common Misconceptions About Regression to the Mean

How to Protect Your Study From Misleading Regression-to-the-Mean Effects

Start by examining how participants entered the study. If eligibility depends on an unusually high or low score, consider whether temporary variation could influence that score.

Next, determine whether the design provides a credible estimate of what would happen to similarly selected participants without the additional intervention.

Where feasible, address internal validity threats through the design before selecting a statistical method.

For an existing study, describe the eligibility threshold, examine the distribution of baseline scores, assess repeated-measurement stability, and identify whether a suitable comparison is available.

If regression to the mean remains a plausible explanation, acknowledge it specifically rather than making a generic statement that the study has limitations.

Regression-to-the-Mean Checks Before Interpreting Improvement

Frequently Asked Questions About Regression to the Mean

Extreme Baseline Scores Can Produce Misleading Evidence of Improvement

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.