03 · What You Need to Know
Why More Covariates Do Not Necessarily Mean Better Causal Estimates
What is the difference between excessive measurement and overadjustment?
Excessive measurement concerns the collection of information whose expected scientific contribution may not justify its cost, burden, or risks. Overadjustment concerns the analytical decision to condition on variables that should not be controlled for the causal effect of interest.
The distinction matters because a variable may be worth measuring without belonging in the primary adjustment model.
For example, researchers studying the total effect of online tutoring on examination performance may measure students' confidence after tutoring begins. Confidence could be useful for investigating intervention mechanisms, but adjusting for it in the primary total-effect analysis may remove part of the effect that tutoring produces through confidence.
Conversely, a study may collect only a few variables and still suffer from overadjustment if one is an inappropriate conditioning variable.
Excessive variable measurement
Collecting variables without sufficient scientific, analytical, descriptive, or operational justification.
Unnecessary adjustment
Conditioning on variables that are not needed for the intended analysis and may contribute little benefit or impair estimation.
Overadjustment bias
Bias arising when adjustment inappropriately blocks relevant causal pathways or otherwise changes the causal contrast being estimated.
Terminology varies across methodological literature. Some authors use overadjustment broadly to describe several forms of inappropriate conditioning, while others distinguish overadjustment for mediators from collider-stratification bias and unnecessary adjustment for precision-irrelevant covariates.
For clarity, researchers should identify the specific causal or statistical problem rather than relying solely on the label overadjustment.
Why does including every available covariate seem attractive?
Researchers often associate extensive statistical adjustment with methodological rigor. A model containing many covariates may appear to account for more alternative explanations than a simpler model.
There is a legitimate concern behind this instinct: failing to measure or control important confounders can undermine causal inference. Yet the solution is not to include every recorded variable.
A covariate may predict the outcome without confounding the exposure-outcome relationship. Another may be a consequence of exposure. A third may be a collider, meaning that conditioning on it can create a noncausal pathway.
The appropriate question is whether the variable belongs in the adjustment strategy for the specific causal estimand.
This requires identifying plausible confounders before conducting the study rather than treating the available columns in a dataset as an automatic adjustment list.
How can adjusting for a mediator distort a total-effect estimate?
A mediator lies on a causal pathway from exposure to outcome. If an intervention changes a mediator, which then changes the outcome, that pathway contributes to the intervention's total effect.
Consider the proposed causal sequence:
Online tutoring → Academic confidence → Examination performance.
If tutoring increases confidence and confidence improves performance, controlling for post-tutoring confidence may block part of the causal pathway through which tutoring operates.
As a result, the adjusted estimate may no longer represent the total effect of tutoring.
This does not mean mediator adjustment is universally inappropriate. Researchers interested in direct effects or mediation may need to account for mediators using methods suited to those estimands and their identification assumptions.
However, a conventional regression coefficient obtained after adjusting for a mediator should not automatically be interpreted as a valid direct causal effect. Exposure-induced mediator-outcome confounding, measurement error, and other causal structures can complicate interpretation.
How can adjusting for a collider introduce bias?
A collider is a variable influenced by two or more other variables. Conditioning on it can create an association between its causes even when those causes were otherwise independent.
Suppose both tutoring participation and unmeasured academic motivation influence whether a student completes an optional follow-up assessment. Motivation also affects examination performance.
Restricting analysis to assessment completers, or otherwise conditioning on completion, may create an association between tutoring and motivation within the selected group. This can distort the tutoring-performance relationship.
Collider bias is not limited to regression adjustment. It can arise through eligibility criteria, participant selection, complete-case analysis, or matching on inappropriate variables.
The causal consequences of conditioning on mediators and colliders therefore deserve explicit consideration before the adjustment set is finalized.
Can unnecessary covariates reduce statistical precision?
Yes, although the consequences depend on the estimator, sample size, and relationships among variables.
In an ordinary regression model, adding covariates consumes model complexity and may increase uncertainty when the data contain limited information. Highly correlated predictors can make individual coefficients unstable, while numerous parameters relative to the available sample can encourage overfitting.
However, including a strong baseline outcome predictor may improve precision. In randomized trials, appropriate baseline adjustment is often beneficial even when the variable is not needed to remove systematic confounding.
Therefore, unnecessary adjustment should not be equated with inevitable loss of precision. Its consequences must be assessed within the actual design and statistical method.
Likewise, regression overfitting is not identical to causal overadjustment. Overfitting concerns how well a statistical model generalizes beyond the observed data, while causal overadjustment concerns inappropriate conditioning relative to the intended effect.
Can adjusting for an instrumental variable make confounding worse?
Under certain causal structures, yes.
An instrumental variable influences exposure and satisfies additional assumptions concerning its relationship with the outcome and unmeasured causes. A valid instrument is not ordinarily treated as a conventional confounder.
When unmeasured confounding remains, conditioning on a strong exposure predictor that behaves like an instrument can amplify bias in some settings.
This phenomenon illustrates why a variable's strong association with exposure does not establish that it should be included in an adjustment model.
Instrumental-variable methods are specialized identification strategies with their own assumptions. The concern here is the indiscriminate use of instrument-like variables as ordinary covariates, not the legitimate application of instrumental-variable estimation.
Does adjusting for more baseline variables always introduce bias?
No. Many baseline covariates can be adjusted for appropriately, and some improve precision without creating causal bias.
The key is their causal role. A pre-exposure variable may be a confounder, an outcome predictor, an instrument, or a collider. Its timing alone does not determine whether adjustment is appropriate.
For example, prior achievement may be an important confounder in a voluntary tutoring study. In a randomized tutoring trial, it may instead be used primarily to improve precision.
Researchers should not respond to concerns about overadjustment by avoiding all covariate adjustment. That could leave important confounding unresolved.
The objective is to identify an appropriate adjustment set, not to maximize or minimize the number of regression terms.
How can causal diagrams help prevent unnecessary adjustment?
Directed acyclic graphs, or DAGs, make assumptions about causal relationships explicit. They can help identify which variables are needed to block noncausal pathways and which variables should not be conditioned on for a particular estimand.
For example, a diagram may distinguish baseline motivation, which plausibly affects both tutoring participation and examination performance, from post-tutoring confidence, which may mediate part of tutoring's effect.
Under an adequately specified causal graph, researchers can identify sufficient adjustment sets rather than selecting covariates through automated statistical screening.
However, DAGs do not establish that their assumptions are correct. A graph omitting an important common cause may incorrectly suggest that the chosen adjustment set is sufficient.
Researchers should develop their diagrams using relevant literature, subject-matter expertise, and an explicit account of the data-generating process.
Why can outcome-driven variable selection be problematic?
Researchers sometimes fit numerous models and retain covariates according to their statistical significance or the resulting change in the exposure coefficient.
Although change-in-estimate approaches have been studied as confounder-selection procedures, they do not replace causal reasoning and can behave poorly under particular causal structures.
Choosing the final adjustment set after observing which model produces a desired result creates additional concerns about selective reporting and underestimated analytical uncertainty.
Prespecification can reduce this flexibility. Researchers may still conduct justified exploratory or sensitivity analyses, but these should be distinguished from the primary analysis.
The same principle applies to deciding which variables deserve measurement. The scientific purpose of a variable should be considered before its observed coefficient becomes available.
How does excessive measurement affect participants and data quality?
Even when additional measurements do not create analytical bias, they can have practical consequences.
Longer questionnaires may increase respondent fatigue or item nonresponse. Repeated assessments require time and administrative resources. Sensitive questions create privacy obligations and may discourage participation.
These effects depend on the population, instruments, and study procedures. A longer instrument is not necessarily inferior when each component contributes essential information.
Researchers should evaluate whether additional measurements improve the study sufficiently to justify their costs and potential consequences.
Importantly, avoiding excessive collection must not become an excuse for failing to measure essential confounders. The aim is sufficient, high-quality information rather than indiscriminate expansion or arbitrary reduction.
How should researchers distinguish primary and secondary adjustment models?
A primary adjustment model should correspond to the prespecified causal estimand and the assumptions supporting its identification.
Secondary models may investigate alternative defensible specifications, measurement uncertainty, or the sensitivity of results to additional assumptions.
For example, researchers might report a primary estimate adjusted for a causally justified baseline set and a secondary analysis examining how results change under alternative operationalizations of socioeconomic circumstances.
However, a model that adjusts for a mediator may estimate a different quantity rather than merely provide a more thoroughly adjusted version of the same effect.
Researchers should label such analyses according to their scientific purpose and avoid presenting a sequence of increasingly adjusted models as an automatic progression toward causal truth.
