Does Measuring Before and After an Intervention Prove the Intervention Caused the Change?

If Participants Improve After an Intervention, Can You Say the Intervention Worked?

You measure participants before an intervention, implement a program, and measure them again. Their average scores increase. A statistical test shows that the difference is significant.

It is tempting to conclude that the intervention caused the improvement. After all, the change occurred after the intervention, and the researchers deliberately introduced the program.

But the timing of two events does not, by itself, establish a causal relationship. The central methodological question is whether participants would have experienced a similar change even if the intervention had never occurred.

That question determines the difference between demonstrating an observed pretest-posttest change and estimating an intervention's causal effect.

Observed Change Is Not Automatically a Causal Effect

Why Before-and-After Differences Cannot Establish Causation Alone

What Is the Difference Between Observed Change and a Causal Effect?

Observed change is the difference between measurements taken at two points in time. A causal effect concerns the difference between outcomes under two alternative conditions, such as receiving an intervention and not receiving it, for the same target unit and relevant time.

These quantities are not interchangeable.

Suppose a group of students scores higher on a research-methods assessment after completing a workshop. The observed increase may reflect learning from the workshop. However, students might also have learned through regular coursework, independent study, or repeated exposure to the assessment.

The pretest tells us where students started. It does not tell us where they would have ended without the workshop.

Consequently, the appropriateness of a pretest-posttest design depends partly on whether the research objective is to describe change or identify its cause.

What Is the Counterfactual Problem?

Modern causal inference often defines an intervention effect by comparing potential outcomes. For a particular participant, one potential outcome represents what would happen under the intervention. Another represents what would happen under a specified alternative condition.

At the same time and under the same relevant circumstances, researchers ordinarily observe only one of these potential outcomes. The other is counterfactual: it describes an outcome that was not observed because the alternative condition did not occur.

Because the individual counterfactual is generally unobservable, research designs seek credible ways to estimate average effects across participants or other units.

A comparison group, for example, may provide information about outcomes without the intervention. Whether it provides a valid counterfactual depends on the design and its assumptions.

Why Is the Pretest Not the Counterfactual Outcome?

A pretest is an observation from an earlier time. The counterfactual is an outcome under an alternative intervention condition at the relevant follow-up time.

The distinction matters because participants and their environments can change independently of the intervention.

Consider an educational program introduced at the beginning of a semester. By the end of the semester, students may have gained knowledge through ordinary instruction, completed assignments, or developed familiarity with assessment tasks.

The baseline score does not account for these developments. Treating it as the outcome students would have achieved without the intervention assumes that nothing else would have changed the outcome during the study period.

That assumption is often implausible and cannot generally be verified using only one pretest and one posttest.

Which Alternative Explanations Can Produce Before-and-After Change?

Several well-established threats to internal validity can generate differences between measurements even when an intervention has no causal effect.

These are plausible competing explanations, not automatic diagnoses of bias in every study. Their importance depends on the research setting, intervention, measurement process, and observed data.

Researchers should evaluate the processes that can change outcomes without an intervention rather than assuming that every observed difference reflects the intended treatment.

Does Statistical Significance Resolve the Causal Question?

No. Statistical significance and causal identification address different problems.

A paired-samples t-test, for example, can assess whether the mean paired difference is compatible with a specified null hypothesis under the test's assumptions. It does not determine why that difference occurred.

Even a very small p-value cannot rule out a systematic external influence that affected participants between measurements.

Increasing the sample size can improve precision in estimating the observed mean change, but greater precision does not automatically reduce bias arising from an inadequate counterfactual.

The two forms of inference are complementary. A credible causal design still requires appropriate statistical analysis, while sophisticated statistical analysis cannot automatically repair a fundamentally weak causal design.

How Does a Comparison Group Improve Causal Inference?

A comparison group provides observations about outcomes under an alternative condition. This can help distinguish changes associated with the intervention from changes occurring more generally over the same period.

For example, if both intervention and comparison groups improve substantially, some of the improvement may reflect influences shared by both groups.

However, the mere presence of a comparison group is not sufficient. Participants may differ in prior achievement, motivation, access to resources, or other factors related to their outcomes.

The appropriateness of a comparison group depends on whether its observed outcomes can credibly inform the counterfactual for the intervention group.

Random assignment can strengthen this comparison by making treatment assignment independent of potential outcomes in expectation, subject to correct implementation. It does not guarantee identical baseline characteristics in a particular sample or eliminate problems such as differential attrition and nonadherence.

Can Comparing Changes Between Groups Identify an Intervention Effect?

One common approach is to compare the change observed in an intervention group with the change observed in a comparison group. This is the basic idea behind difference-in-differences estimation.

For a causal interpretation in a nonrandomized setting, a central identifying assumption is that, without the intervention, the groups would have followed parallel outcome trends over the relevant period. Additional assumptions and safeguards may also be required, including appropriate treatment timing, measurement comparability, and consideration of spillovers and other concurrent changes.

With only one baseline measurement, the parallel-trends assumption cannot be meaningfully evaluated against an observed pre-intervention trend. Multiple pre-intervention measurements can provide useful diagnostic evidence, although similar past trends do not guarantee that future counterfactual trends would have remained parallel.

Difference-in-differences is therefore not simply a statistical formula that converts any two-group pretest-posttest study into a causal experiment.

What Can Randomization Establish That Before-and-After Measurement Cannot?

In a properly implemented randomized trial, assignment to the intervention is determined by a random mechanism rather than participants' characteristics or investigators' preferences.

Randomization can make the assigned groups comparable in expectation with respect to potential outcomes, supporting estimation of average causal effects of assignment under appropriate assumptions.

The post-intervention difference between randomized groups can therefore provide a defensible causal contrast even when pretest measurements are unavailable. Baseline measurements may still improve precision, support assessment of baseline characteristics, and help address particular analytical objectives.

Randomization is not a guarantee of validity. Poor allocation concealment, differential missing outcomes, deviations from assigned interventions, interference between participants, or inappropriate analysis can undermine the interpretation.

The relevant question is not merely whether participants were measured twice, but how the design makes the alternative outcome condition credible.

Can Stronger Quasi-Experimental Designs Support Causal Claims?

Yes, under defensible identifying assumptions. Randomization is a powerful approach to causal identification, but it is not the only one.

Some evaluations use natural experiments, regression discontinuity designs, interrupted time series, or carefully specified difference-in-differences approaches when random assignment is infeasible.

For example, an interrupted time-series design may estimate whether an intervention coincides with a departure from an established pre-intervention trajectory. Its interpretation still depends on assumptions about the counterfactual trend and the absence or adequate handling of competing events.

Different quasi-experimental methods address different sources of bias. Their credibility comes from the research setting, assignment mechanism, design features, and assumptions, not from the methodological label alone.

Can an Uncontrolled Before-and-After Study Ever Provide Useful Evidence?

Yes. Lack of strong causal identification does not make observed change meaningless.

An uncontrolled study can document outcome trajectories, assess feasibility, examine participant experiences, evaluate measurement procedures, or identify patterns worth investigating in a stronger design.

In some circumstances, an exceptionally large and immediate change closely linked to an intervention, together with strong substantive evidence against plausible alternatives, may contribute to a causal argument. The strength of that argument must be evaluated case by case.

Nevertheless, Cochrane's guidance identifies uncontrolled studies with one measurement before and one after intervention as generally presenting serious or critical concerns when used to estimate intervention effects.

The appropriate response is to report what the study demonstrates and make the remaining causal uncertainty explicit.

When Higher Posttest Scores Have More Than One Explanation

The observed result is a 16-point mean increase. But during the same six weeks, the students also completed writing-intensive coursework, received feedback from supervisors, and practiced academic writing independently.

Those activities are plausible alternative explanations for at least part of the improvement. The original design cannot determine how much change would have occurred without the AI-assisted program.

Now suppose the researchers had also followed a nonrandomized comparison group of 60 students who completed similar coursework but did not receive the program.

The intervention group improved by 7 points more than the comparison group. This contrast is more informative than the original 16-point within-group increase because it accounts descriptively for improvement also observed among students without the program.

Nevertheless, the 7-point difference is not automatically the program's causal effect. Students might have entered the groups with different motivations, received different supervisory support, or experienced other unequal influences. A causal difference-in-differences interpretation would require defensible assumptions about their untreated trends.

If the students had been randomly assigned to the program and comparison conditions, with assignment integrity and outcome ascertainment maintained, the between-group contrast could support a stronger causal interpretation.

The central lesson is that the same observed pretest-posttest increase can support different conclusions depending on how the counterfactual is established.

Common Errors When Interpreting Intervention-Related Change

How to Make Defensible Claims From Before-and-After Research

Before collecting data, decide whether your intended conclusion concerns change, comparative effectiveness, or a causal intervention effect. These objectives require different levels of evidence.

If you already have uncontrolled pretest-posttest data, do not discard them simply because they cannot establish causation. Report the observed change accurately, quantify uncertainty, and identify plausible competing explanations.

If you are planning a study intended to establish effectiveness, consider strengthening the design before implementation rather than relying on statistical adjustment afterward.

How Should You Phrase Your Findings?

Wording should reflect the actual strength of the design.

When reporting an intervention effect, distinguish the numerical estimate from the assumptions required to interpret it causally. This is particularly important in educational evaluations where students, teachers, and institutions may simultaneously experience multiple influences.

Researchers should also consider whether additional baseline measurements would help characterize existing trends, and whether including a control group in a before-and-after study would better address the intended causal question.

Before Claiming an Intervention Caused Change

Frequently Asked Questions About Pretest-Posttest Causality

Demonstrating Change Is Different From Demonstrating Its Cause

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.