03 · What You Need to Know
Understanding How Unmeasured Variables Affect Causal Inference
What Is Confounding?
Confounding occurs when the association between an exposure and an outcome does not adequately represent the causal effect of interest because of other factors related to the causal structure.
For example, students with stronger prior academic achievement may be more likely to use an optional learning platform and may also achieve higher grades.
If prior achievement influences both platform use and performance, the observed association could partly reflect differences in prior achievement.
However, identifying confounding requires more than finding a variable correlated with both exposure and outcome.
The variable's causal position matters, including whether it precedes the exposure and whether conditioning on it would block or open relevant pathways.
What Is an Unmeasured Confounder?
An unmeasured confounder is a factor relevant to confounding control that is not adequately observed in the available data.
For example, an institutional dataset may lack a suitable measure of students' prior academic achievement.
Even if the dataset contains demographic characteristics and current grades, these variables may not adequately substitute for prior achievement.
Unmeasured confounding is particularly important in observational research intended to estimate causal effects.
Hernán and Robins explain that causal identification from observational data commonly depends on assumptions about exchangeability and the sufficiency of measured covariates.
These assumptions cannot be established simply by fitting a statistical model.
How Are Missing Values Different from Missing Variables?
A missing value occurs when a variable exists but has no recorded observation for a particular case.
A missing variable is a factor that was never collected, is unavailable in the released dataset, or cannot be measured adequately using the available information.
For example, prior achievement may be recorded for 80% of students. This creates a missing-value problem.
If prior achievement was never recorded for anyone, it represents an absent variable.
Multiple imputation may be appropriate for certain missing-value problems under defensible assumptions, but it cannot automatically reconstruct a wholly unobserved confounder.
Researchers should distinguish these situations before selecting analytical remedies.
Which Missing Variables Actually Matter?
Not every unavailable variable threatens causal inference.
The importance of a variable depends on its role in the causal relationship under investigation.
For example, a student's favorite learning application may be irrelevant to a particular causal comparison, while prior achievement could be important.
Some variables may be consequences of the exposure rather than confounders. Adjusting for them may remove part of the effect being investigated.
Other variables may be colliders, meaning that conditioning on them can create a noncausal association.
Researchers should use substantive theory and causal reasoning to identify an appropriate adjustment set rather than assume that every available predictor belongs in the model.
How Do Different Variable Roles Affect Adjustment?
| Variable role |
Illustrative example |
Implication for adjustment |
| Confounder |
Prior achievement influences platform use and grades |
May need adjustment for the intended causal effect |
| Mediator |
Study behavior changes after platform use and affects grades |
Adjustment may block part of the effect of interest |
| Collider |
A variable influenced by both exposure and another cause of the outcome |
Conditioning may introduce selection or collider bias |
| Instrumental variable candidate |
A factor influencing exposure through a specific mechanism |
Requires strong identification assumptions; not ordinary confounder adjustment |
| Proxy confounder |
Earlier grades used to approximate prior academic preparation |
May provide partial information but leave residual confounding |
| Irrelevant predictor |
A recorded characteristic unrelated to relevant causal pathways |
Not necessarily needed for confounding control |
The roles in this table are hypothetical. The same variable can occupy different causal positions under different research questions and temporal arrangements.
Why Is a Causal Diagram Useful?
A directed acyclic graph, or DAG, represents assumptions about causal relationships among variables.
For example, a researcher might propose that prior achievement influences both platform participation and final grades.
The diagram can help identify pathways that create confounding and clarify which variables should be measured for adjustment.
It can also identify situations in which conditioning on a variable may introduce bias.
However, a causal diagram is not proof that its assumed relationships are correct.
Researchers should develop it using relevant theory, empirical evidence, and substantive knowledge before relying on it to select adjustment variables.
What Is Conditional Exchangeability?
Conditional exchangeability is a causal identification assumption under which, after conditioning on an appropriate set of variables, the compared exposure groups are sufficiently comparable with respect to their potential outcomes.
For example, researchers estimating the causal effect of platform participation may assume that students with the same relevant baseline characteristics do not differ systematically in unmeasured causes of the outcome related to their exposure.
This assumption is demanding in observational data.
It cannot ordinarily be verified from observed variables alone because the necessary counterfactual outcomes are unavailable.
Missing essential confounders may make the assumption difficult to defend.
Even complete measurement of known confounders does not automatically guarantee that all causal identification assumptions hold.
Does Regression Automatically Control Confounding?
No. Regression can adjust for measured variables included appropriately in a statistical model.
However, it does not automatically eliminate confounding from variables that were not measured.
For example, adjusting for age, program, and course grades cannot necessarily replace an absent measure of prior achievement.
Regression also depends on model specification, measurement quality, and the causal roles of included variables.
Researchers should avoid describing a model as fully controlling confounding merely because it contains numerous covariates.
The adequacy of adjustment depends on whether the chosen variables and analytical assumptions address the relevant confounding pathways.
Does a Large Dataset Compensate for Missing Confounders?
No. Increasing sample size may reduce random sampling uncertainty without eliminating systematic bias.
For example, a dataset containing 100,000 student records may estimate an association between platform use and grades very precisely.
However, if students' prior achievement systematically influences both variables and is unmeasured, the estimate may remain biased for the intended causal effect.
Researchers should distinguish statistical precision from causal validity.
A narrow confidence interval does not establish that the estimated relationship has a causal interpretation.
What Is Residual Confounding?
Residual confounding refers to confounding that remains after the analytical adjustments have been applied.
It may arise because relevant variables were omitted, measured inaccurately, categorized inadequately, or modeled inappropriately.
For example, a researcher may adjust for a broad category of previous academic performance while meaningful differences remain within that category.
Residual confounding may also persist when available proxies inadequately represent underlying causes.
Researchers should not assume that including a variable automatically removes all confounding associated with the concept it is intended to measure.
Can Proxy Variables Replace Missing Confounders?
Proxy variables may provide partial information about an unmeasured factor.
For example, an earlier standardized assessment score may provide information about prior academic preparation.
However, a proxy does not necessarily capture every relevant dimension of the underlying confounder.
Adjustment may therefore leave residual confounding.
Furthermore, a poorly selected proxy could introduce bias if it has an inappropriate causal relationship with exposure or outcome.
Researchers should assess how proxy variables can be used in secondary data analysis before treating them as substitutes for missing confounders.
Can Propensity Scores Solve the Problem?
Propensity score methods use measured covariates to model the probability of receiving an exposure or treatment.
They can support matching, weighting, stratification, or adjustment under appropriate conditions.
However, conventional propensity score methods do not automatically account for unmeasured confounders.
If an essential confounder is absent from the dataset, balancing the observed variables does not guarantee that the groups are comparable on the unobserved factor.
Researchers should avoid interpreting good balance on measured covariates as evidence that all confounding has been eliminated.
Can Machine Learning Recover Unmeasured Confounders?
Machine learning methods may improve prediction or support flexible modeling of measured relationships.
However, predictive performance does not establish causal identification.
A model cannot generally recover an entirely unmeasured confounder simply by using a more sophisticated algorithm.
Some specialized causal approaches use proxy measurements or other structures to address unmeasured confounding, but they require additional assumptions and suitable observed information.
Researchers should distinguish flexible statistical estimation from the identification of causal effects.
Complexity does not compensate automatically for missing causal information.
What About Instrumental Variables?
Instrumental variable methods may identify certain causal effects despite unmeasured confounding when demanding assumptions are satisfied.
These assumptions typically concern the instrument's relationship with exposure, the absence of particular pathways to the outcome, and independence from relevant unmeasured causes.
For example, a researcher might investigate whether an institutional allocation rule creates plausibly exogenous variation in access to a learning intervention.
However, an administrative variable does not become a valid instrument merely because it predicts exposure.
Instrumental variable analysis requires a defensible identification argument and may identify an effect for a particular population or margin of treatment variation.
It should not be introduced as a routine statistical remedy for missing confounders.
What Is Sensitivity Analysis for Unmeasured Confounding?
Sensitivity analysis examines how conclusions would change under specified assumptions about unmeasured confounding.
For example, researchers may ask how strongly an unmeasured factor would need to be associated with exposure and outcome to explain an observed association.
VanderWeele and Ding (2017) introduced the E-value as one approach for evaluating sensitivity to unmeasured confounding on the risk-ratio scale.
However, the E-value is not universally appropriate for every outcome, model, or causal question.
Other quantitative bias analyses may be more suitable depending on the available evidence.
Sensitivity analysis evaluates dependence on assumptions; it does not reveal the actual values of unmeasured variables.
Can Negative Controls Help?
Negative control exposures or outcomes may help detect certain forms of residual bias when their causal properties are justified.
For example, a researcher may identify an outcome that should not plausibly be affected by the exposure but may share relevant sources of confounding.
An unexpected association could raise concerns about bias.
However, selecting a valid negative control requires substantive causal reasoning.
The absence of an association with a negative control does not guarantee that the primary analysis is free from confounding.
Negative controls are diagnostic tools under particular assumptions rather than universal solutions.
How Does the Research Question Affect the Importance of Missing Confounders?
Missing confounders are especially consequential when the study intends to estimate a causal effect.
For example, asking whether platform participation causes improvements in academic performance requires a defensible causal identification strategy.
Asking whether platform participation is associated with grades is a different question.
The associational analysis may remain meaningful even when a causal interpretation is unsupported.
However, researchers should still consider selection, measurement, and omitted-variable limitations when interpreting associations.
Changing causal language to associational language should reflect a genuine change in the inferential objective, not merely conceal a failed causal design.
When Should Additional Data Be Collected?
Additional data collection may be appropriate when essential confounders can be measured reliably and linked to the existing observations.
For example, an institution might have earlier assessment records that can be linked to student-level platform activity.
However, linkage must be authorized, technically accurate, and consistent with applicable privacy requirements.
Newly collected measurements may also fail to represent historical confounders accurately.
Researchers should determine whether additional information can genuinely address the identification problem before expanding collection.
When Does Missing Confounding Information Make a Study Infeasible?
A proposed causal study may be infeasible when the available data cannot support a credible identification strategy and essential assumptions cannot be defended.
For example, students' self-selection into an optional AI learning platform may depend heavily on unmeasured academic preparation and motivation.
If the dataset contains no appropriate baseline information or alternative source of identification, a causal claim about platform effectiveness may be unjustified.
However, the same dataset may still support descriptive or associational questions.
Researchers should distinguish infeasibility of a particular causal inference from complete unusability of the dataset.
How Should Missing Confounders Be Reported?
The methods section should explain which confounders were considered, how they were identified, and which were available for adjustment.
Researchers should justify the selected covariates using substantive reasoning rather than only statistical significance.
Important unavailable confounders should be identified explicitly, together with their plausible consequences for inference.
Where appropriate, report sensitivity analyses and their assumptions.
The STROBE reporting framework recommends transparent identification of potential confounders, adjustment methods, and limitations.
Researchers should avoid claiming that an adjusted association is necessarily causal or free from residual confounding.
