03 · What You Need to Know
What Makes Causal Inference From Cohort Data Defensible?
What Is the Difference Between Association and Causation?
An association describes a statistical relationship between variables. Causation concerns whether changing an exposure would change an outcome, under a specified intervention or causal contrast.
Suppose students who receive AI literacy training are less likely to withdraw from university. That observation establishes an association if the study's measurements and analysis are valid.
A causal interpretation makes a stronger claim: the students' withdrawal outcomes would have differed if their training exposure had been different, with other relevant conditions appropriately accounted for.
The difficulty is that researchers cannot ordinarily observe the same student's outcome under both training and no training in the same circumstances. Causal inference therefore involves comparing observed outcomes and reasoning about unobserved counterfactual outcomes.
The central question is whether the observed comparison can credibly represent the counterfactual contrast of interest.
Why Is Temporal Ordering Necessary but Insufficient?
A proposed cause must precede the outcome it is hypothesized to influence. Cohort studies can establish this sequence when exposure and outcome timing are recorded appropriately.
For example, documented training completion in January followed by academic withdrawal in November provides evidence that the training preceded the withdrawal.
However, temporal precedence does not establish that training caused or prevented the outcome.
Students who participate in training may already differ in motivation, academic preparation, institutional support, or access to technology. Those characteristics may influence their later outcomes regardless of training.
Temporal ordering addresses one requirement of causal reasoning. It does not establish that the exposure groups would otherwise have experienced comparable outcomes.
This distinction is one reason cohort studies are useful for examining subsequent outcomes without being automatically equivalent to randomized experiments.
What Is the Causal Effect You Want to Estimate?
Before selecting an analytical method, researchers should define the causal estimand: the specific causal quantity the study seeks to estimate.
For example, a study might ask about the difference in two-year withdrawal risk if all eligible students initiated an AI literacy program at enrollment compared with if none initiated it.
That question differs from asking whether students who eventually completed training had lower observed withdrawal rates.
The first concerns specified intervention strategies. The second classifies students according to an observed behavior that may occur after follow-up begins.
Other causal estimands may concern sustained participation, different training intensities, or effects within particular student populations.
Researchers should specify the population, exposure or intervention strategies, outcome, follow-up horizon, and effect measure. Without this clarity, the same dataset can produce estimates that answer substantially different questions.
How Do Potential Outcomes Help Define Causality?
The potential-outcomes framework represents causal effects through outcomes that would occur under alternative exposure or intervention conditions.
For a binary intervention, let Y(1) represent an individual's outcome under intervention and Y(0) the outcome under no intervention.
The individual causal effect is conceptually the difference between these potential outcomes, although both cannot ordinarily be observed for the same person under the same conditions.
At the population level, researchers may define an average causal effect.
The formula defines the causal quantity. It does not provide a method for identifying that quantity from observational data.
What Is Exchangeability?
Exchangeability concerns whether the groups being compared are sufficiently comparable with respect to their potential outcomes under the relevant exposure strategies.
In a randomized trial, appropriate randomization can support exchangeability at baseline in expectation.
In an observational cohort, exposure is not randomly assigned. Researchers may instead seek conditional exchangeability by measuring and appropriately adjusting for common causes of exposure and outcome.
For example, baseline academic performance and motivation might influence both participation in AI literacy training and subsequent withdrawal risk.
Adjustment may improve comparability if the relevant confounders are accurately measured and the causal model is appropriate.
However, unmeasured confounding may remain. Conditional exchangeability is generally an identifying assumption rather than something that can be established conclusively through statistical testing.
What Is Positivity, and Why Does It Matter?
Positivity requires that the exposure strategies being compared have a nonzero probability of occurring within relevant combinations of confounders in the target population.
Suppose AI literacy training is mandatory for all students in one academic program and unavailable to everyone in another. If program membership is an essential confounder, the data may contain no trained and untrained students within the same program.
Estimating a training contrast within those strata would then require extrapolation beyond the observed exposure support.
Researchers should examine whether the data contain meaningful exposure variation across relevant participant characteristics.
Extreme exposure probabilities can also produce unstable estimates, even when positivity is not strictly violated.
What Is Consistency?
Consistency connects the observed outcome under an individual's actual exposure history to the potential outcome under the corresponding specified intervention strategy.
It requires particular care when exposure labels represent substantially different versions of an intervention.
For example, "AI literacy training" might refer to a two-hour introductory seminar, a semester-long course, or individualized instruction. These are not necessarily interchangeable treatments.
If the intervention is poorly defined, the causal contrast may be ambiguous even when the statistical analysis is technically correct.
Researchers should specify what the intervention entails and whether relevant variations are compatible with the intended estimand.
How Does Confounding Threaten Causal Inference?
Confounding arises when factors associated with both exposure and outcome distort the observed exposure–outcome relationship.
For example, students who voluntarily enroll in AI literacy training may have stronger academic motivation than nonparticipants. Motivation may also influence retention.
An observed association between training and retention could therefore partly reflect differences in motivation.
Researchers may use regression adjustment, matching, weighting, standardization, or other methods to address measured confounding.
However, no method automatically removes confounding when important common causes are unmeasured or poorly measured.
Covariate selection should follow substantive causal reasoning. Adjusting indiscriminately for every available variable can introduce bias when some variables are mediators, colliders, or consequences of exposure.
Can Directed Acyclic Graphs Help?
Directed acyclic graphs, or DAGs, represent assumptions about causal relationships among variables.
They can help researchers identify potential confounding pathways, distinguish pre-exposure variables from mediators, and evaluate which adjustment strategies are consistent with a proposed causal structure.
For example, a DAG might represent academic motivation as a common cause of training participation and retention, while confidence developed after training might be represented as a mediator.
Adjusting for motivation may be appropriate for estimating a total training effect under that structure. Adjusting for post-training confidence could instead remove part of the pathway of interest.
A DAG does not prove that its assumed relationships are correct. Its value lies in making those assumptions explicit and subject to scrutiny.
What Is Selection Bias in a Cohort Study?
Selection bias can occur when inclusion or continued observation depends on variables related to exposure and outcome in ways that distort the intended comparison.
For example, a study that includes only students who remained enrolled long enough to complete an AI literacy program may systematically exclude early withdrawals.
Similarly, students with academic difficulties may be less likely to participate in follow-up assessments.
Selection mechanisms can create associations that differ from those in the intended target population.
Researchers should examine cohort eligibility, recruitment, loss to follow-up, and the consequences of conditioning on participation or survival.
The selection of an appropriate comparison group is particularly important when exposure groups differ in their opportunities to enter or remain in the cohort.
Why Is Time Zero Critical for Causal Comparisons?
Time zero is the beginning of follow-up for the intended comparison.
In causal cohort analyses, eligibility, exposure strategies, and follow-up should be appropriately aligned.
Suppose researchers classify students as trained if they complete an AI literacy course at any point during their first year, but begin counting withdrawal outcomes from enrollment.
Students classified as completers must remain enrolled long enough to complete the course. If the earlier observation period is incorrectly attributed to training, the analysis can introduce immortal time bias.
Possible solutions depend on the causal estimand and may include defining intervention strategies at eligibility, using appropriate time-varying exposure methods, or emulating a target trial with a specified initiation period.
The methodological principles of defining cohort time zero are therefore essential for credible causal comparisons.
What Is Target Trial Emulation?
Target trial emulation is an approach to observational causal inference that begins by specifying the hypothetical randomized trial corresponding to the causal question.
Researchers describe the target trial's eligibility criteria, intervention strategies, treatment assignment, follow-up, outcome, causal contrast, and analysis plan.
They then examine whether observational data can emulate the relevant features of that trial.
For example, a target trial might compare initiating a defined AI literacy program within the first month of enrollment against not initiating during that period, with withdrawal assessed over two years.
Historical administrative records might support an observational emulation if eligibility, exposure dates, outcomes, and relevant confounders are recorded adequately.
However, specifying a target trial does not make observational data randomized. Exchangeability, positivity, consistency, measurement validity, and other assumptions still require justification.
The approach is useful because it exposes ambiguities and design problems before statistical modeling.
What Happens When Confounders Change Over Time?
Time-varying confounding creates additional challenges when participants' characteristics influence later exposure and outcomes.
Suppose academic engagement affects whether students seek additional AI training, while previous training also affects engagement.
Engagement may then be both a predictor of later training and a consequence of earlier training.
Conventional regression adjustment for such a variable may not identify the intended total causal effect.
Depending on the estimand and available data, methods such as marginal structural models with inverse probability weighting or other g-methods may be appropriate.
These approaches require careful specification and assumptions about confounding, positivity, model estimation, and observation.
The problem is explored further through exposures that change during cohort follow-up.
Can Statistical Significance Establish Causality?
No. A small p-value or narrow confidence interval does not demonstrate that the estimated association is free from confounding or other systematic bias.
A large observational cohort may estimate a biased association with considerable precision.
Likewise, a nonsignificant estimate does not necessarily demonstrate the absence of a causal effect. The study may lack precision or have substantial measurement error.
Researchers should evaluate effect magnitude, uncertainty, design validity, and the credibility of the identifying assumptions rather than relying on significance thresholds alone.
Can Sensitivity Analysis Strengthen a Causal Argument?
Sensitivity analysis examines how conclusions change under alternative assumptions, analytical specifications, or plausible sources of bias.
For example, researchers may assess how strong an unmeasured confounder would need to be to explain an observed association, or whether estimates change under alternative exposure definitions.
Negative control exposures or outcomes may also provide useful evidence about certain biases when suitable controls can be identified.
However, sensitivity analyses do not automatically validate the causal interpretation. Their usefulness depends on whether they address realistic threats and whether the assumptions underlying them are appropriate.
A defensible causal argument should acknowledge uncertainty that remains after such analyses.
Does Prospective Cohort Research Provide Stronger Causal Evidence Than Retrospective Research?
Not necessarily. Prospective studies may allow researchers to plan measurements and follow-up procedures, while retrospective cohorts may use extensive historical records containing reliable exposure and outcome dates.
Both remain observational unless the exposure is assigned experimentally.
The choice between prospective and retrospective cohort designs influences data availability and measurement quality, but neither label independently establishes causal validity.
A carefully designed retrospective cohort with credible confounding control may support a stronger causal argument than a prospective cohort with substantial selection bias and poorly defined exposures.
