Which Variables Should Be Measured to Control for Confounding?

How Do You Know Which Variables Actually Need to Be Controlled?

You are investigating whether students who use generative AI tools achieve better academic writing performance than students who do not.

The initial analysis shows an association between AI use and writing scores. However, students who choose to use AI may differ from nonusers in prior writing proficiency, motivation, technological access, and other characteristics.

Some of these differences could explain part or all of the observed association. Others may have little relevance to confounding, while adjusting for certain variables could actually introduce bias.

Should you control for age, gender, socioeconomic status, motivation, baseline writing proficiency, and every other variable available in the dataset? Would a stepwise regression procedure identify the correct controls? Could you simply include all variables significantly associated with the outcome?

The difficulty is that confounding is not defined by statistical significance alone. It concerns the causal relationships that make an observed association differ from the effect researchers intend to estimate.

Choosing variables for confounding control therefore requires an explicit understanding of the research question, temporal ordering, and plausible causal structure.

Measure Confounders According to a Defensible Causal Model

Identifying Confounders Without Introducing New Sources of Bias

What Is Confounding?

Confounding occurs when an observed association between an exposure and outcome is distorted by other causal relationships, so that the association does not represent the causal effect of interest.

Consider a study examining the effect of voluntary AI tool use on writing performance.

Students with stronger prior writing proficiency may be more likely to adopt AI tools. Prior proficiency may also influence subsequent writing scores.

Under this assumed causal structure, prior proficiency creates a pathway connecting AI use and writing performance that does not represent the effect of AI use itself.

If the pathway is not addressed, the observed association may reflect both the effect of AI use and differences in prior proficiency.

Confounding is therefore a problem of causal structure rather than merely an imbalance in observed characteristics.

Why the Traditional Confounder Checklist Is Incomplete

Confounders are often described as variables associated with both exposure and outcome that are not consequences of the exposure.

This description can be useful as an introductory heuristic, but it is not a complete causal definition.

A variable may be statistically associated with both exposure and outcome without being appropriate for adjustment.

Conversely, a variable may be important to a causal adjustment strategy even when its association is weak or imprecisely estimated in the available sample.

Modern causal inference emphasizes identifying the causal pathways that create noncausal associations and determining which variables must be conditioned on to block those pathways.

The distinction matters because statistical association alone cannot reveal the direction of causation or identify whether a variable is a mediator, collider, or confounder.

Start With the Causal Estimand

Before selecting confounders, researchers should define the effect they intend to estimate.

For example, the question may concern the total effect of AI tool use on writing performance.

Alternatively, it may concern the effect of AI use not operating through writing self-efficacy, or the effect among a particular subgroup.

These are different causal quantities and may require different analytical strategies.

Adjusting for a mediator may be inappropriate when estimating a total effect but relevant to a carefully specified direct-effect analysis under additional assumptions.

Researchers should therefore avoid treating covariate selection as a generic step independent of the estimand.

ICH E9(R1) provides a formal framework for specifying estimands in clinical trials, while broader causal inference literature extends these principles to other study designs.

Directed Acyclic Graphs Can Clarify Confounding Assumptions

A directed acyclic graph, or DAG, represents assumed causal relationships using variables and directed arrows.

For example, consider the following simplified causal structure:

Together, these relationships create the backdoor path:

AI tool use ← Prior writing proficiency → Writing performance.

Under this simplified DAG, conditioning appropriately on prior proficiency can block the backdoor path.

However, the DAG is an expression of assumptions, not evidence that the assumed relationships are true.

Researchers should develop the graph using substantive knowledge, theory, and relevant empirical evidence.

An incomplete or incorrect DAG can lead to an inappropriate adjustment set.

What Is a Backdoor Path?

A backdoor path is a path connecting exposure and outcome that begins with an arrow entering the exposure.

Some backdoor paths transmit noncausal association and must be blocked for identification of a total causal effect under the backdoor criterion.

For example, if prior proficiency causes both AI use and writing performance, the path through prior proficiency is open unless appropriately conditioned on.

However, not every backdoor path is open, and not every variable appearing along a path should be adjusted for.

Colliders can block paths unless conditioning or other selection mechanisms open them.

Researchers should therefore use the formal rules of d-separation or appropriate causal graph tools rather than merely counting arrows or adjusting for every connected variable.

What Is a Sufficient Adjustment Set?

A sufficient adjustment set is a collection of variables that, under specified causal assumptions and an applicable identification criterion, allows relevant noncausal pathways to be blocked.

More than one sufficient set may exist.

For example, a causal graph may permit adjustment for one common cause or for another combination of variables that blocks the same problematic pathways.

The smallest set by variable count is not necessarily the most practical or statistically efficient choice.

Measurement quality, missingness, sample size, and the precision of the resulting estimate may also matter.

Researchers should distinguish a theoretically sufficient adjustment set from one that can actually be measured adequately in the proposed study.

In some causal structures, no sufficient adjustment set of available observed variables exists.

Measure Important Confounders Before Exposure When Appropriate

Temporal ordering is central to causal interpretation.

For many studies, measuring potential confounders before the exposure or intervention helps establish that the measured characteristics were not consequences of the exposure.

For example, baseline writing proficiency measured before AI tool adoption may be more suitable for confounding control than writing proficiency measured after extensive AI use.

However, not every variable measured after exposure is necessarily caused by that exposure, and not every baseline variable is a confounder.

The causal role depends on the actual relationships, not merely the calendar date of measurement.

Time-varying confounding requires additional care because confounders may change over time and may themselves be affected by earlier exposure.

Conventional regression adjustment can be inappropriate for some such structures, and methods such as marginal structural models may be needed under suitable assumptions.

Distinguish Confounders From Mediators

A mediator lies on a causal pathway through which an exposure influences an outcome.

Suppose AI tool use increases students' opportunities to revise their writing, and revision activity subsequently improves writing performance.

Revision activity may then be a mediator.

If the primary objective is to estimate the total effect of AI tool use, adjusting for revision activity may remove part of the effect of interest.

This is sometimes called overadjustment.

However, mediation analysis may legitimately involve the mediator when the objective is to investigate causal mechanisms and the necessary assumptions are addressed.

The key distinction is whether the variable represents a cause of both exposure and outcome or a consequence of exposure along the pathway to the outcome.

The same observed variable may occupy different causal roles in different research settings.

Distinguish Confounders From Colliders

A collider is a variable that receives causal arrows from two other variables along a path.

Consider a hypothetical structure in which both AI tool use and writing difficulty influence whether a student seeks additional academic support.

Seeking support is then a common effect of AI use and writing difficulty under the assumed model.

If writing difficulty also influences final writing performance, conditioning on support-seeking may create an association between AI use and writing difficulty that was not present through that path before conditioning.

This is an example of collider stratification bias.

Researchers should not assume that controlling for a variable makes groups more comparable simply because it is associated with both the exposure and outcome.

Its position within the causal structure determines whether adjustment is appropriate.

Why Adjusting for Every Available Variable Can Be Harmful

Including every available covariate in a regression model may appear conservative.

However, unnecessary adjustment can introduce several problems.

• Collider bias: Conditioning on a common effect may open a noncausal pathway.
• Overadjustment: Conditioning on a mediator may remove part of the total effect.
• Reduced precision: Unnecessary variables may increase estimation uncertainty in some models.
• Measurement error: Poorly measured covariates may fail to control confounding adequately and can complicate interpretation.
• Missing-data losses: Including variables with substantial missingness may reduce the analyzed sample under complete-case procedures.

These problems are context-dependent. Some prognostic covariates can improve precision, and additional measurements can be valuable when appropriately used.

The principle is to select adjustment variables through causal and statistical reasoning rather than indiscriminate inclusion.

Statistical Significance Is Not a Reliable Confounder Selection Rule

Researchers sometimes select covariates using bivariate p-values or stepwise regression procedures.

For example, they may include only variables significantly associated with the outcome or exposure.

This approach can omit important confounders whose associations are estimated imprecisely.

It can also include variables that should not be adjusted for under the assumed causal structure.

A nonsignificant association does not establish that a variable is causally irrelevant.

Likewise, a statistically significant association does not establish that a variable is a confounder.

Statistical procedures may help evaluate model performance or estimate parameters, but they cannot replace the substantive assumptions needed to identify causal effects.

Confounding Control Depends on the Research Design

Randomized experiments and observational studies face different confounding considerations.

In an appropriately randomized trial, treatment assignment is independent of baseline potential outcomes and baseline characteristics in expectation, subject to the randomization design.

Consequently, baseline confounding of assignment is addressed by the design rather than by measuring every possible confounder.

Baseline covariates may still be collected to improve precision, support planned subgroup analyses, or address other methodological objectives.

In observational research, exposure assignment is not controlled by the investigator, and confounding may require a more substantial identification strategy.

However, observational studies do not become causally valid merely by collecting many background variables.

Researchers must consider whether important common causes are measured and whether the necessary assumptions are credible.

Selection Bias and Confounding Are Different Problems

Confounding concerns noncausal associations arising from the causal structure connecting exposure and outcome.

Selection bias can arise when inclusion in the sample or analysis depends on variables that create distorted associations.

For example, conditioning on participation may introduce collider bias when participation is influenced by exposure-related and outcome-related factors.

Adjusting for baseline confounders does not necessarily resolve selection bias introduced through recruitment or missing outcomes.

Researchers should therefore evaluate sampling bias and selection mechanisms separately from confounding control.

Some variables may be relevant to both problems, but the corresponding analytical assumptions are not identical.

Measurement Error Can Leave Residual Confounding

Even when a relevant confounder is included in the dataset, poor measurement may prevent adequate adjustment.

For example, prior writing proficiency measured through a single unvalidated self-rating may not adequately represent actual baseline writing ability.

If the measured variable captures the confounder imperfectly, residual confounding may remain.

Researchers should therefore consider validity, reliability, timing, and the level of detail required for important confounders.

In some circumstances, multiple indicators or more appropriate instruments may improve measurement.

However, no measurement strategy guarantees complete control of confounding.

Researchers should distinguish the presence of a covariate in the model from evidence that the relevant causal pathway has been adequately addressed.

Unmeasured Confounding May Remain

Researchers rarely have perfect information about every relevant causal factor.

Some confounders may be unknown, difficult to measure, or unavailable in existing datasets.

Consequently, an adjusted observational estimate may remain vulnerable to unmeasured confounding.

Sensitivity analyses can examine how strong an unmeasured confounding mechanism would need to be to alter particular conclusions under specified assumptions.

Negative controls, instrumental variables, and other identification strategies may also be useful in suitable settings, but each requires its own assumptions.

Researchers should not describe an observational association as definitively causal merely because several measured covariates were included in regression.

What About Variables Associated Only With the Outcome?

A variable that predicts the outcome but does not create a relevant confounding pathway may still be useful for statistical precision.

For example, baseline writing proficiency may strongly predict final writing performance in a randomized trial.

Adjusting for it in an appropriate prespecified analysis may improve precision even though randomization addresses confounding of treatment assignment.

This is different from measuring the variable because it is required to block a backdoor path.

Researchers should distinguish covariates selected for causal identification from those selected for precision or other analytical purposes.

The distinction helps explain why the appropriate adjustment set depends on both causal assumptions and statistical objectives.

Can a Proxy Variable Control for an Unmeasured Confounder?

Researchers sometimes use proxy variables when a confounder cannot be measured directly.

For example, prior academic grades might be used as an imperfect indicator of earlier writing proficiency.

However, a proxy does not automatically block the confounding pathway created by the underlying construct.

Its usefulness depends on the measurement relationship, causal structure, and analytical method.

Ordinary regression adjustment for a noisy proxy may leave residual confounding.

Specialized proxy-based identification approaches exist, but they require additional assumptions and suitable data.

Researchers should not claim that a confounder has been fully controlled merely because a related variable was included.

Use a Causal Variable Selection Matrix

A variable selection matrix can document why particular covariates are measured and how they will be used.

The classifications are hypothetical. Their appropriateness depends on the actual causal relationships in the population.

Researchers should not transfer the table mechanically to another study.

The matrix should be developed alongside the causal diagram, measurement plan, and primary analysis specification.

Measure Confounders Before Finalizing the Analysis Plan

Confounding control should be considered during study design rather than after the data have been collected.

Researchers should identify the causal effect of interest, develop a plausible causal model, determine which variables are needed for identification, and evaluate whether they can be measured adequately.

These decisions should be aligned with identifying variables essential to the primary research question.

If essential confounders cannot be measured or addressed through another credible strategy, the study may need to limit its claims to association rather than causation.

Transparent acknowledgment of identification limitations is preferable to presenting extensive statistical adjustment as proof that confounding has disappeared.

Selecting Confounders for an Observational Study of AI Tool Use

Evaluating Candidate Variables

The team recognizes that the correct adjustment set cannot be established from the variable names alone.

It depends on the causal assumptions represented in the diagram.

Developing the Adjustment Strategy

Suppose the adjusted association differs from the unadjusted association.

This difference may be consistent with confounding, but it does not independently establish that the adjusted estimate is unbiased or causal.

The credibility of the conclusion depends on the causal model, measurement quality, positivity, consistency, exchangeability, and other assumptions relevant to the design.

The researchers should report these assumptions and avoid claiming that statistical adjustment has definitively eliminated confounding.

Common Misconceptions About Confounding Control

Select Confounders Through Causal Reasoning Before Statistical Modeling

Begin by defining the causal effect of interest and specifying the exposure and outcome.

Then identify plausible causal relationships using substantive knowledge and an explicit conceptual or graphical model.

Document the selected adjustment strategy before examining the main outcome associations whenever possible.

Where uncertainty exists about the causal structure, consider how alternative plausible assumptions affect the adjustment set and resulting conclusions.

The objective is not to control for the greatest number of variables. It is to identify and address the sources of confounding relevant to the causal question.

Before Selecting Variables for Confounding Control

Frequently Asked Questions About Selecting Confounding Variables

Confounding Control Requires the Right Variables, Not the Most Variables

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.