What Is a Natural Experiment?

What Makes a Naturally Occurring Event a Natural Experiment?

A government introduces an education policy in selected regions. A university changes its scholarship eligibility rules. A natural disaster unexpectedly closes some schools while others remain open. Researchers did not assign these exposures, yet the resulting differences may create opportunities to investigate causal effects.

Are these situations natural experiments simply because they occurred outside a laboratory or without researcher intervention?

Not necessarily. Many exposures occur naturally, but their occurrence alone does not provide a credible basis for causal inference. Students choose schools, institutions adopt policies, and communities experience events for reasons that may also influence the outcomes researchers want to explain.

The defining methodological question is whether the circumstances generating exposure create a sufficiently credible comparison between exposed and unexposed conditions.

A Natural Experiment Uses Externally Generated Exposure Variation to Investigate Causal Effects

How Natural Experiments Work and What Makes Them Credible

What Does Natural Mean in a Natural Experiment?

The word natural refers to how the exposure variation arises, not necessarily to nature, environmental conditions, or events occurring without human involvement.

A natural experiment may arise from a government policy, administrative decision, eligibility rule, infrastructure change, institutional reform, or unexpected external event.

For example, a government may introduce free public transportation for students in selected jurisdictions. Researchers could investigate whether the policy affects school attendance, provided the assignment mechanism and available comparisons support a credible causal analysis.

Although the policy is deliberately implemented by government officials, it is external to the researcher's assignment process. It may therefore create a natural-experiment opportunity.

Conversely, observing students who independently choose to use a digital learning platform does not automatically constitute a natural experiment. Their choices may reflect motivation, access to technology, academic ability, or other determinants of learning outcomes.

How Is a Natural Experiment Different From a Randomized Experiment?

In a randomized experiment, researchers or an authorized implementing body assign treatment according to a random allocation procedure. When properly implemented, randomization provides a known mechanism supporting comparability between treatment groups in expectation.

In a natural experiment, the researcher does not control the exposure assignment. Instead, the researcher investigates whether an external process produces variation that can support causal identification.

Natural experiments are sometimes described as approximating randomized experiments. This analogy can be useful, but only when the assignment process genuinely supports it.

The absence of researcher control does not establish random assignment.

Does a Natural Experiment Require As-If Random Assignment?

Terminology varies across disciplines and methodological traditions.

In a relatively strict interpretation, a natural experiment involves an external assignment mechanism that plausibly produces exposure variation resembling random allocation, at least within a defined population or conditional on relevant characteristics.

Other researchers use the term more broadly for externally generated policy changes or events evaluated through quasi-experimental methods, even when identification depends on assumptions such as parallel trends rather than literal as-if random exposure.

These uses should not be treated as methodologically equivalent.

A claim of as-if random assignment requires evidence that exposure was not systematically determined by factors related to the potential outcomes. A policy introduced in economically disadvantaged districts, for example, is not automatically as-if random merely because researchers did not choose those districts.

Researchers should explicitly explain how the exposure assignment mechanism supports or fails to support as-if random variation.

What Types of Events Can Create Natural Experiments?

Several kinds of externally generated variation may create useful research opportunities.

Policy and Regulatory Changes

Governments may introduce, repeal, or modify policies affecting particular populations, locations, or time periods.

For example, a scholarship policy may change eligibility rules for students entering university after a specified date. Researchers might compare outcomes across affected and unaffected cohorts, but they must consider other changes associated with cohort membership.

The policy's introduction creates exposure variation. Whether that variation identifies a causal effect depends on the assignment mechanism and available counterfactual.

Administrative and Institutional Rules

Eligibility thresholds, school assignment procedures, administrative boundaries, and allocation rules can sometimes create relatively sharp differences in treatment exposure.

For example, a remedial education program may be assigned according to a diagnostic score cutoff. Students immediately around the cutoff could provide a credible local comparison if relevant continuity assumptions hold.

Such settings may support regression discontinuity, although a threshold does not automatically guarantee a valid design.

Unexpected External Events

Natural disasters, infrastructure disruptions, and other unexpected events may affect some populations more than others.

However, exposure to such events is often geographically or socially patterned. Communities experiencing a disruption may differ systematically in resources, vulnerability, infrastructure, and institutional capacity.

Researchers must therefore distinguish the unexpected timing of an event from the potentially nonrandom distribution of its consequences.

Staggered Implementation and Resource Allocation

Policies or interventions may reach institutions at different times because of administrative schedules, funding cycles, or logistical constraints.

Such variation may create opportunities for causal evaluation, particularly when implementation timing is plausibly unrelated to anticipated outcome changes.

Yet early adopters may differ from later adopters. They may have stronger leadership, better infrastructure, or greater readiness to implement the intervention.

Researchers should examine whether naturally staged implementation provides a credible research opportunity rather than treating staggered adoption as automatically exogenous.

How Does a Natural Experiment Identify a Causal Effect?

A natural experiment requires a defensible comparison between observed outcomes and the outcomes that would have occurred under an alternative exposure condition.

This unobserved alternative is the counterfactual.

Suppose a new education subsidy is introduced in one jurisdiction but not another. Researchers observe an improvement in enrollment in the treated jurisdiction.

The observed improvement alone does not identify the subsidy effect. Enrollment might have increased because of demographic changes, economic conditions, or other education policies.

A credible causal analysis must establish why the untreated jurisdiction, an earlier trajectory, an eligibility threshold, or another source of variation provides useful information about the treated population's counterfactual.

The appropriate analytical strategy depends on how the external event created exposure variation.

Which Research Designs Can Analyze a Natural Experiment?

Natural experiment is not a single statistical estimator. Different identification strategies may be used depending on the setting.

These strategies are not interchangeable. Each exploits a different feature of the assignment process and depends on distinct assumptions.

The decision should follow the logic of selecting a causal identification strategy, rather than choosing a method because it is commonly associated with natural experiments.

What Makes a Naturally Occurring Exposure Useful for Causal Research?

Three considerations are particularly important: the exposure-generating mechanism, the available comparison, and the credibility of the assumptions linking that comparison to the intended counterfactual.

The researcher should establish how exposure occurred, who was affected, when it occurred, and whether the process could be influenced by characteristics related to the outcome.

For example, a government may allocate educational technology grants according to an administrative funding formula. The formula could create useful variation, but researchers must examine whether it is associated with school disadvantage, prior achievement, or other determinants of educational outcomes.

Even when the assignment mechanism appears promising, researchers must consider interference, anticipation, noncompliance, and differences in treatment intensity.

The broader question of whether a naturally occurring exposure or policy change supports causal research requires examining these features in the specific setting.

Can Researchers Design a Natural Experiment After the Event?

Yes. Researchers may identify an external event retrospectively and investigate whether its assignment mechanism supports a credible causal analysis.

For example, a university may have changed scholarship eligibility several years earlier. Historical administrative records could allow researchers to reconstruct the eligibility rule, identify affected cohorts, and evaluate subsequent outcomes.

Retrospective design does not inherently invalidate the analysis. However, researchers must address incomplete records, selective availability of outcomes, changes in measurement, and the possibility of choosing analyses after observing favorable results.

Clear documentation of the assignment mechanism and transparent reporting of analytical decisions are especially important.

What Are the Main Threats to Natural-Experiment Validity?

The most consequential threats arise when the external event does not create the comparison assumed by the researcher.

Confounded exposure: Exposed and unexposed populations differ in factors that also influence outcomes.

Anticipation: Individuals or institutions change behavior before formal implementation because they expect the event or policy.

Concurrent events: Other changes occur around the same time and affect the outcome.

Selective migration or participation: Individuals enter, leave, or select exposure conditions in ways related to potential outcomes.

Spillovers: The intervention affects individuals or institutions classified as unexposed.

Measurement changes: Outcome definitions, reporting systems, or data completeness change across exposure conditions.

These threats should be evaluated through the assignment mechanism, contextual evidence, and appropriate diagnostics or sensitivity analyses.

Can Natural Experiments Establish Causation?

Natural experiments can support causal inference when the exposure-generating process and analytical strategy justify the intended counterfactual comparison.

Some natural experiments provide unusually persuasive identification opportunities. Others remain vulnerable to substantial confounding or uncertainty about the assignment mechanism.

The appropriate conclusion should reflect the strength of the evidence rather than the attractiveness of the natural-experiment label.

Researchers should also distinguish internal validity from generalizability. A natural experiment may identify a credible local effect for a particular population or policy context without establishing that the same effect would occur elsewhere.

Evaluating a Scholarship Policy Introduced Through an Administrative Rule

Suppose enrollment is estimated to be 6 percentage points higher immediately on the eligible side of the cutoff than on the ineligible side, after an appropriate discontinuity analysis.

Under the required assumptions, this difference could be interpreted as a local causal effect of eligibility. Statistical uncertainty and design diagnostics would still need to be reported.

However, if applicants can manipulate their scores precisely to qualify, or if another educational benefit changes at the same threshold, the causal interpretation may be compromised.

The example illustrates why a natural experiment is not defined merely by the existence of a policy. Its usefulness depends on the assignment process and the causal comparison that process makes possible.

Common Misconceptions About Natural Experiments

How to Recognize a Credible Natural-Experiment Opportunity

Begin by investigating how exposure was generated. The relevant evidence may be found in policy documents, administrative rules, implementation schedules, eligibility records, or institutional decisions.

Next, determine which individuals or institutions were exposed and why. Ask whether the same factors that determined exposure could also influence the outcome.

For educational technology research, promising opportunities may arise when software access, digital infrastructure, or institutional support is allocated through documented external rules. Yet technology readiness and resource availability often correlate with educational outcomes, making careful identification particularly important.

A credible natural experiment should be recognizable from the assignment process before the researcher examines whether the estimated effect is statistically significant.

Before Calling a Study a Natural Experiment, Verify These Conditions

Frequently Asked Questions About Natural Experiments

A Natural Experiment Is Defined by Its Exposure Mechanism, Not Merely by an External Event

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.