What Can Change Over Time Even Without the Intervention?

If Outcomes Changed After an Intervention, What Else Could Explain It?

You introduce a new teaching strategy, measure students' performance before and afterward, and discover that their average scores improved. It is tempting to attribute the improvement to the strategy. But what if those same students would have improved anyway?

Time is not an empty interval between two measurements. Participants develop, circumstances change, and the act of measuring an outcome may influence subsequent performance. Some changes reflect genuine developments unrelated to the intervention. Others arise from how participants were selected, assessed, or retained.

Recognizing these possibilities is essential when interpreting whether a before-and-after difference supports a causal claim.

Several Processes Can Produce Change Without the Intervention

Why Outcomes May Change Independently of an Intervention

The central problem in evaluating an intervention is determining what would have happened to the same participants, under comparable circumstances, if they had not received it. This unobserved outcome is called the counterfactual.

A baseline measurement describes what was observed before the intervention. It does not necessarily describe what would have happened afterward without the intervention. The difference matters because the passage of time may influence outcomes through several distinct mechanisms.

History Effects: Something Else Happened

A history effect occurs when an event or contextual change during the study could influence the outcome independently of the intervention.

Suppose a university introduces an academic writing workshop. During the same semester, students also gain access to an institutional writing center. Their writing scores improve, but the workshop is not the only plausible explanation.

History effects can involve institutional policies, economic conditions, public events, curriculum revisions, technological changes, or other activities occurring alongside the intervention.

The crucial issue is not simply whether another event occurred. It is whether that event could plausibly affect the measured outcome and whether the design can distinguish its influence from that of the intervention.

Maturation: Participants Change Naturally

Maturation refers to changes within participants that occur as time passes, independently of the intervention being evaluated.

Children develop cognitively and physically. Students accumulate experience. Employees become familiar with their responsibilities. Patients may recover naturally or experience disease progression.

For example, first-year university students may become more confident in academic communication over a semester simply through participation in their ordinary coursework. A communication-training program introduced during that period might coincide with improvement that would have occurred without it.

Maturation is not restricted to biological aging. Learning through routine experience, fatigue, adaptation, and changes in motivation may also matter.

Secular Trends and Seasonal Patterns: The Outcome Was Already Moving

A secular trend is a longer-term pattern of change in a population or system. Seasonal patterns are recurring changes associated with particular periods.

Suppose an institution introduces a student-support initiative after its graduation rate has increased steadily for several years. A further increase afterward does not necessarily indicate that the initiative accelerated improvement.

Similarly, student engagement may vary across an academic semester because of examinations, holidays, or workload. Hospital admissions may fluctuate seasonally. These patterns can create misleading before-and-after contrasts when measurement timing is poorly chosen.

Existing trends are particularly important when studying institutions, communities, and populations rather than only individual participants.

Testing Effects: The First Measurement Changes the Second

A testing effect occurs when exposure to an earlier assessment influences later performance or responses.

Students may remember test questions, recognize the expected format, or investigate unfamiliar concepts after completing a pretest. Respondents may become more reflective about an attitude after answering questions about it.

Consequently, posttest improvement can reflect familiarity with the assessment rather than the intervention.

Using alternate but equivalent test forms may reduce some practice effects, although differences between forms can introduce new measurement problems. Researchers must consider both comparability and repeated-exposure effects.

Instrumentation: The Measurement Process Changed

Instrumentation effects occur when differences in instruments, scoring procedures, assessors, or administration conditions contribute to apparent outcome changes.

Imagine that students' research presentations are scored using a strict rubric before an intervention but a more generous rubric afterward. Higher posttest scores might partly reflect the revised scoring procedure.

Similar problems arise when survey wording changes, software updates alter calculated indicators, different assessors apply inconsistent standards, or equipment is recalibrated.

Using the same instrument does not automatically guarantee comparability. Its administration and interpretation must also remain sufficiently consistent, and the instrument should continue to measure the intended construct across occasions.

Regression to the Mean: Extreme Scores Become Less Extreme

Regression to the mean is a statistical phenomenon in which unusually high or low observations tend, on average, to be less extreme on subsequent measurement when the observations contain random variation.

Suppose researchers recruit students specifically because they obtained exceptionally low scores on a diagnostic assessment. Some students may score higher on a second assessment even without receiving an effective intervention.

The initial scores may have reflected both underlying ability and temporary influences such as fatigue, anxiety, or measurement error. Those temporary influences are unlikely to recur in precisely the same way.

Regression to the mean is especially relevant when eligibility depends on an extreme baseline score. Its magnitude depends partly on measurement reliability and the selection process. It is not evidence that all participants inevitably move toward an average.

Attrition: The People Measured Later May Be Different

Attrition occurs when participants initially included in a study do not contribute data at subsequent measurement occasions.

Suppose 100 students complete a baseline motivation survey, but only 65 complete the follow-up. If students with lower motivation are disproportionately absent, the follow-up average may rise even if motivation did not improve among the original group.

This is a compositional change rather than necessarily a genuine improvement within individuals.

Attrition can also bias estimated within-person change when the participants who remain differ systematically in their outcomes or trajectories from those who leave. The consequences depend on the missing-data mechanism and the analysis.

Concurrent Activities and Changing Exposure

Participants rarely experience only the intervention under investigation. Students continue attending classes, patients receive other care, and employees participate in routine professional development.

When these activities change during the evaluation period, they may influence the same outcome. For example, improved research-methods knowledge following an online tutorial could also reflect concurrent coursework, independent reading, or peer tutoring.

Researchers should document relevant co-interventions and contextual changes rather than assuming the intervention was the only meaningful exposure.

How These Explanations Differ

These explanations are not mutually exclusive. An evaluation may simultaneously experience maturation, external events, and testing effects. The challenge is determining which explanations are plausible, consequential, and distinguishable using the available evidence.

Did an AI-Assisted Learning Program Improve Research Skills?

The observed improvement is real as a descriptive difference in the recorded scores. What remains uncertain is how much of that improvement, if any, was caused by the AI-assisted program.

During the six weeks, students also attended regular research-methods classes. Some discussed pretest questions with classmates. Their instructors introduced additional research exercises, and the assessment reused several questions from the baseline test.

These circumstances create plausible alternative explanations. Regular coursework could produce maturation through learning. Additional exercises represent concurrent educational exposure. Reused questions may produce testing effects.

Now suppose a comparable group of students who did not receive the AI-assisted program improved from 60 to 70 over the same period. Their 10-point increase suggests that improvement was possible without the program.

The intervention group's 16-point increase exceeds the comparison group's 10-point increase by 6 points. This contrast is potentially more informative than the intervention group's change alone, but it is not automatically a causal estimate. The groups may have differed in prior achievement, motivation, instruction, or expected outcome trajectories.

A stronger evaluation would investigate these differences, document concurrent activities, and examine whether the groups had comparable pre-intervention trends where such data are available.

Common Mistakes When Explaining Change Over Time

How to Investigate Alternative Explanations Before Making Claims

Begin by identifying the outcome's expected trajectory without the intervention. Ask whether participants would naturally develop, whether the outcome fluctuates seasonally, and whether relevant institutional or environmental changes are likely during follow-up.

Next, examine the measurement process. Determine whether the baseline assessment itself could change participants' subsequent responses and whether instruments, scoring procedures, and administration conditions will remain comparable.

Finally, consider how the study design can provide evidence about the counterfactual. A suitable comparison group in a before-and-after study may help distinguish intervention-related change from changes occurring more generally, provided its selection and outcome trajectory are defensible.

When stronger causal identification is not feasible, a carefully conducted before-and-after study may still document meaningful changes, assess implementation, and generate hypotheses. The reporting should distinguish observed improvement from demonstrated intervention effectiveness.

Before Attributing a Change to an Intervention, Check:

Frequently Asked Questions About Changes Without an Intervention

Change Over Time Is Not Necessarily Change Caused by an Intervention

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.