03 · What You Need to Know
How Concurrent Events Undermine Natural-Experiment Identification
What Counts as a Concurrent Event?
A concurrent event is a development occurring during the relevant evaluation period that may influence the outcome independently of, or in combination with, the intervention of interest.
It does not have to occur on precisely the same date as the intervention. An event beginning shortly before implementation, developing gradually afterward, or affecting outcomes with a delay may also be consequential.
Examples include changes in government policy, institutional funding, curriculum, assessment standards, economic conditions, technology infrastructure, and population composition.
Suppose a university introduces a learning analytics platform in August. During September, it also establishes mandatory academic advising. Both interventions could influence student retention over the following semesters.
If retention improves, the researcher must determine whether the available design can distinguish the platform's effect from the advising policy's effect.
The presence of another event is not automatically disqualifying. Its importance depends on whether it changes the outcome or the counterfactual comparison relevant to the causal question.
Why Are Concurrent Events a Threat to Internal Validity?
Internal validity concerns whether the research design supports the intended causal interpretation for the population and conditions being studied.
A natural experiment relies on some source of exposure variation and an associated counterfactual. The researcher must explain why observed outcome differences can reasonably be attributed to the exposure rather than competing causes.
Concurrent events threaten that explanation when they produce outcome changes that the design attributes to the intervention.
For example, if a school district introduces a digital attendance system during a period of improved public transportation, a subsequent increase in attendance might reflect the technology, transportation improvements, or their combined influence.
In traditional experimental-design terminology, this concern is often described as a history threat. In causal inference, it may involve time-varying confounding, violations of comparison-trend assumptions, or other forms of counterfactual misspecification.
The precise problem depends on the design and the relationship between the competing event, treatment assignment, and outcome.
Does Every Concurrent Event Create Confounding?
No. Temporal coincidence alone does not establish confounding.
Suppose an education policy is implemented during a period of nationwide economic growth. If the economic change affects treated and untreated populations similarly, an appropriate comparison design may account for its influence.
However, if the economic change affects treated regions differently, the same design may fail to separate the policy effect from the differential economic response.
Researchers should distinguish events that are common to all relevant units from those that disproportionately affect particular exposure groups.
Shared concurrent event
A change affecting treated and comparison populations in a sufficiently comparable way, potentially allowing an appropriate comparative design to account for its influence.
Differential concurrent event
A change affecting treated and comparison populations differently, potentially distorting the estimated treatment contrast.
Even a shared event may interact with treatment or population characteristics. Researchers should not assume that common exposure guarantees identical outcome effects.
Why Does the Counterfactual Matter More Than the Number of Events?
The fundamental question is what would have happened to the treated population without the intervention of interest.
A concurrent event matters when it changes that counterfactual or prevents the researcher from estimating it credibly.
For example, suppose two universities experience the same nationwide examination reform, but only one introduces an AI tutoring program.
If the examination reform affects achievement measurement similarly in both institutions, an appropriately specified comparison may help distinguish the tutoring-associated change from the reform.
But if the reform interacts with differences in curricula or student composition, the comparison may not adequately capture its influence.
This is why selecting an appropriate comparison group requires considering how populations respond to external events, not merely whether they resemble each other before intervention.
How Do Concurrent Events Affect Difference-in-Differences?
Difference-in-differences (DiD) compares changes in outcomes between treated and comparison groups. Its causal interpretation depends on assumptions about how their outcomes would have evolved without treatment.
Suppose District A receives a digital connectivity subsidy while District B does not. During the same period, District A also introduces a mandatory online learning policy.
If the mandatory policy independently increases online participation, the observed difference between districts may reflect both interventions.
Even if the districts exhibited similar pre-intervention trends, that historical similarity does not eliminate a new differential event occurring after treatment begins.
The relevant parallel-trends assumption concerns untreated potential outcomes. A concurrent event that differentially changes those outcomes can undermine the assumption.
Researchers should investigate policies and contextual changes occurring in both treated and comparison groups, especially around treatment adoption.
How Do Concurrent Events Affect Interrupted Time-Series Designs?
Interrupted time-series (ITS) designs estimate whether an outcome changes relative to its earlier trajectory following an intervention.
An uncontrolled ITS may detect a convincing statistical interruption without establishing which event caused it.
Suppose a university introduces a new digital academic advising platform at the same time as a scholarship expansion. Withdrawal rates subsequently decline.
Segmented regression may identify an immediate level change or a change in slope. However, those coefficients cannot automatically be attributed to the platform.
A controlled interrupted time-series design may strengthen inference when a credible comparison series captures relevant background changes.
Nevertheless, a comparison series cannot necessarily separate two interventions introduced simultaneously in the treated population.
If both interventions always occur together and no additional identifying variation exists, their separate effects may remain unidentified.
Can Concurrent Events Threaten Regression Discontinuity?
Yes, particularly when another intervention or institutional rule changes at the same assignment threshold.
Suppose students scoring below a diagnostic cutoff qualify for AI tutoring. The same cutoff also determines eligibility for additional instructor consultations.
A discontinuity in achievement at the threshold may reflect tutoring, consultations, or their combined effects.
The problem is not simply that two interventions occurred during the same period. It is that both change at the same point used for causal identification.
Researchers must determine whether the intended treatment can be isolated or whether the design identifies the effect of a broader treatment package.
Other changes that are smooth around the cutoff may be less threatening under the usual continuity assumptions.
Can Concurrent Events Violate Instrumental-Variable Assumptions?
Yes. A concurrent event may undermine instrumental-variable identification if it creates an alternative pathway from the instrument to the outcome or compromises instrument independence.
For example, suppose a government allocation rule increases access to digital tutoring but also provides participating schools with improved internet infrastructure.
If the researcher uses allocation as an instrument for tutoring participation, the infrastructure improvement may affect achievement independently of tutoring.
This threatens the exclusion restriction for an analysis seeking the isolated effect of tutoring.
The instrument might still identify a different policy effect under another estimand, but the intended tutoring effect would require additional justification.
Researchers should evaluate the assumptions underlying instrumental-variable identification rather than assuming that externally generated assignment is sufficient.
What If Two Policies Are Introduced as a Package?
When multiple interventions are deliberately introduced together, researchers must decide whether the treatment of interest is the entire policy package or one component.
Suppose a university introduces AI tutoring, instructor training, and additional mathematics support sessions as one coordinated reform.
If all components begin together and affect the same students, the available comparison may identify the effect of the combined package rather than the separate effect of AI tutoring.
This is not necessarily a methodological failure. A policy-package effect may be substantively important for decision-makers.
However, researchers should avoid attributing the combined effect to a single component without additional identifying variation or assumptions.
Separating component effects may require different implementation timing, factorial assignment, additional comparison groups, or another suitable design.
How Do You Distinguish a Confounder From a Mediator?
Not every event occurring after intervention is an independent competing cause.
Some changes may be consequences of the intervention and part of the mechanism through which it affects the outcome.
Suppose a digital tutoring program increases student engagement, which subsequently improves examination performance. Engagement may be a mediator of the tutoring effect.
Adjusting for engagement when estimating the total effect of tutoring could remove part of the effect the researcher intends to measure.
By contrast, suppose an unrelated university policy independently increases engagement and examination performance during the same period. That policy may provide a competing explanation.
Researchers should establish the temporal and causal relationships among the intervention, concurrent changes, and outcomes before deciding what to adjust for.
Watch Out
Automatically controlling for every post-intervention variable can introduce bias. Some variables may be mediators, colliders, or consequences of treatment rather than pre-existing confounders.
How Can Researchers Identify Relevant Concurrent Events?
A credible evaluation should include a systematic investigation of the institutional and policy environment surrounding implementation.
Researchers can examine official policy announcements, administrative memoranda, funding records, implementation schedules, curriculum changes, and institutional reports.
For educational technology studies, software deployment records, teacher training schedules, assessment revisions, and changes in digital access may be particularly relevant.
The investigation should cover both the treated population and the comparison population.
Researchers should distinguish events that were planned before intervention, events triggered by the intervention, and independent events occurring afterward.
A documented timeline can help identify overlapping exposure periods and plausible mechanisms through which competing events might influence the outcome.
Can Additional Comparison Groups Help?
Sometimes. Multiple comparison populations may help determine whether an observed change is specific to the intervention or reflects a broader event.
Suppose a national curriculum reform affects all schools, while a digital learning subsidy affects only selected districts.
Comparing subsidized and unsubsidized districts may help account for the national reform if its influence is sufficiently comparable across them.
Additional comparison districts may provide evidence about the stability of this relationship and the sensitivity of the estimated effect.
However, adding more controls does not automatically solve the problem. If all available comparison populations respond differently to the concurrent event, the resulting estimates may remain biased.
Comparison selection should follow the causal identification argument rather than the availability of additional datasets.
Can Negative Control Outcomes Reveal Competing Explanations?
Negative control outcomes are outcomes that should not be causally affected by the intervention of interest but may share relevant sources of bias with the primary outcome.
For example, a researcher evaluating a policy intended to improve online learning participation might examine an outcome that is not plausibly affected by the policy but is sensitive to broader administrative reporting changes.
An apparent intervention-associated change in the negative control outcome could suggest confounding, measurement changes, or another problem with the causal interpretation.
However, selecting a valid negative control requires substantive knowledge. An outcome thought to be unaffected may actually respond indirectly to the intervention.
A null negative control result also does not prove that the primary estimate is unbiased.
Negative controls are diagnostic tools, not substitutes for a credible assignment mechanism and counterfactual.
What Can Placebo Dates and Sensitivity Analyses Tell You?
Researchers may examine whether similar outcome interruptions appear at dates when no intervention occurred.
For example, an interrupted time-series analysis might test a prespecified placebo date before the actual policy implementation.
If the model identifies similarly large changes at several unrelated dates, the apparent intervention effect may reflect unstable trends or model misspecification.
Researchers may also evaluate alternative trend specifications, observation windows, or credible comparison populations.
These analyses can reveal how sensitive conclusions are to modeling decisions.
However, placebo tests cannot independently establish causality. A plausible placebo date must be chosen carefully, and multiple exploratory tests should be reported transparently rather than selectively.
Can Statistical Adjustment Separate Simultaneous Policy Effects?
Only when the data contain sufficient identifying variation and the relevant assumptions are credible.
Suppose two policies are always introduced together in the same institutions at the same time. Their exposure indicators may be perfectly collinear.
In that setting, a regression cannot separately estimate both effects simply by including both policy indicators.
Even when the indicators are not perfectly collinear, separate causal interpretation may require assumptions about treatment selection, interactions, and time-varying confounding.
Researchers should distinguish statistical estimability from causal identifiability. A model may produce separate coefficients without providing a credible basis for interpreting them as separate causal effects.
When Should Researchers Limit Their Causal Claims?
Causal claims should be limited when a consequential concurrent event cannot be separated from the intervention of interest.
Researchers may be able to estimate the effect of a combined policy package, describe changes associated with implementation, or identify effects under additional assumptions.
However, they should not attribute an observed change exclusively to one intervention when the design cannot distinguish it from another influential event.
Transparent reporting should identify the competing event, explain its likely causal pathway, and state how it affects interpretation.
The appropriate conclusion depends on the evidence, not on whether the original study was described as a natural experiment or quasi-experiment.
