How Do You Judge Whether Statistical Adjustment Was Appropriate?

Does Adjusting for More Variables Make an Estimate More Trustworthy?

A paper reports an association, then presents an "adjusted" model controlling for age, sex, socioeconomic status, baseline measurements, comorbidities, and several other variables. The adjusted estimate may differ substantially from the crude result. Which one should you trust?

The word adjusted can sound reassuring because it suggests that alternative explanations have been statistically removed. But adjustment is not a generic cleaning procedure. Its validity depends on why particular variables were controlled, how those variables relate to the exposure or intervention and outcome, when they were measured, and what effect the researchers intended to estimate.

Appropriate adjustment can reduce confounding or improve precision. Inappropriate adjustment can fail to remove confounding, obscure part of the effect of interest, or even introduce bias that was not present before adjustment.

Judge the Logic of Adjustment, Not the Number of Covariates

Adjustment Must Match the Question the Study Is Trying to Answer

First determine why the researchers adjusted at all

Statistical adjustment can serve different purposes. In observational causal research, researchers often adjust for confounding. In randomized trials, baseline covariate adjustment may instead be used primarily to improve statistical precision. Prediction models use covariates for yet another purpose: predicting outcomes rather than necessarily estimating causal effects.

Those goals should not be conflated.

A variable useful for prediction is not necessarily a variable that should be controlled when estimating a causal effect. Before judging an adjusted model, identify the estimand or scientific question it is supposed to answer.

In observational studies, confounding is the central concern

Cochrane describes a confounder, more precisely a confounding domain, as a pre-intervention prognostic factor that predicts the outcome and also predicts which intervention or exposure an individual receives. When such factors are not handled appropriately, the observed association can differ from the causal effect of interest.

Suppose an observational study finds that people receiving treatment A have worse outcomes than people receiving treatment B. If clinicians preferentially give treatment A to patients who are more severely ill, baseline disease severity may confound the treatment-outcome association. Adjustment for well-measured indicators of severity may make the comparison more informative.

But statistical adjustment can address only the confounding represented adequately by the measured and correctly modeled variables. Unmeasured or poorly measured confounding can remain.

Do not assume every variable associated with the outcome is a confounder

A common mistake is to collect many variables and include everything associated with the outcome in a regression model. Causal adjustment requires more reasoning than that.

A variable may predict the outcome without confounding the exposure-outcome relationship. Another variable may be associated with the exposure but lie downstream of it. Yet another may be a common effect of two variables, sometimes called a collider, and conditioning on it can create an association that did not previously exist.

Cochrane specifically cautions that adjusting for factors that are not confounders, particularly variables that may be affected by the intervention, can introduce bias.

Ask when each adjusted variable was measured

Timing is one of the most useful appraisal clues.

If researchers want the total causal effect of an intervention, variables caused by that intervention generally should not simply be treated as baseline confounders. Suppose an educational intervention increases student engagement, which subsequently improves achievement. If researchers adjust for engagement measured after the intervention began, they may remove part of the pathway through which the intervention works.

The resulting estimate may answer a different question from the total effect.

There are legitimate analyses involving mediators and other post-exposure variables, but these require methods and assumptions appropriate to those questions. "We adjusted for everything available" is not a sufficient rationale.

Adjustment can create bias through conditioning on a collider

Consider an exposure and another factor that both influence whether someone enters the analyzed sample. If researchers condition on that shared consequence, an association can be induced between the exposure and the other factor even when none existed beforehand.

This phenomenon is commonly described as collider bias. It is one reason covariate selection should reflect the assumed data-generating structure rather than a mechanical search for variables correlated with the outcome.

Directed acyclic graphs, or DAGs, are one tool researchers can use to make causal assumptions explicit and identify adjustment sets. A DAG does not prove that the assumed causal structure is correct, but it can reveal the consequences of those assumptions more clearly than an unexplained list of regression covariates.

Randomized trials require different reasoning

When randomization has been conducted properly, treatment assignment should not be confounded by baseline characteristics in the same way as an observational treatment comparison. Random baseline differences can still occur, but testing those differences and deciding whether to adjust according to their statistical significance is inappropriate.

CONSORT 2025 states that adjusted analyses in randomized trials can improve power and precision, particularly for prognostic baseline variables, and recommends that adjustment be outlined prospectively. It also notes that analyses should commonly account for variables used in stratified randomization or minimization.

Therefore, a randomized trial that says "we adjusted for age because age differed significantly between groups at baseline" deserves scrutiny. Baseline significance testing is not a sound method for deciding which covariates to include after randomization.

Covariate selection should not be driven mechanically by P-values

Researchers sometimes screen potential confounders using univariable significance tests and adjust only for variables with P < 0.05. Another strategy is to add or remove covariates until the exposure estimate becomes significant.

Neither approach provides a sound general basis for confounder identification.

Confounding is fundamentally about relationships among variables and the target causal effect, not whether a covariate happens to cross a significance threshold in the current sample. Subject-matter knowledge, prior evidence, study design, and an explicit causal model can provide stronger justification.

If adjustment choices appear to have been modified until a favorable result emerged, the problem also overlaps with selective reporting of significant analyses.

Compare adjusted and unadjusted estimates, but interpret the difference carefully

Seeing both estimates can be useful. A substantial change after adjustment tells you that the modeled covariates materially affect the estimated association.

It does not prove that the adjusted estimate is correct.

The change could reflect successful control of confounding, inappropriate adjustment, differences in the analytic sample caused by missing covariate data, model misspecification, or several mechanisms at once.

Cochrane generally prefers appropriately adjusted effect estimates from non-randomized intervention studies over crude summaries because adjustment may reduce confounding, while also emphasizing the need to record which variables were used. That preference assumes the adjustment itself is methodologically defensible.

Residual confounding remains possible after adjustment

A paper may say that results were "adjusted for confounders," but adjustment rarely guarantees complete removal of confounding.

Cochrane notes that residual confounding can remain when relevant confounding domains are unmeasured, measured with error, or modeled inadequately. Socioeconomic position, disease severity, prior achievement, health status, and similar constructs may be difficult to capture perfectly with one observed variable.

Consequently, an adjusted observational estimate should not automatically be interpreted as though randomization had occurred.

How continuous covariates are modeled matters

Suppose age is included as a linear term, implying that each additional year has the same relationship with the outcome across the entire age range. That assumption may or may not be reasonable.

Researchers may instead use transformations, polynomial terms, splines, categories, or other functional forms. CONSORT recommends reporting how continuous variables used in adjusted analyses were handled.

Arbitrary categorization can discard information, while an inappropriate linear assumption can leave residual structure unmodeled. The presence of a variable in the regression equation does not guarantee that it has been controlled adequately.

Missing covariate data can change the comparison

If an adjusted model requires variables that are missing for many participants, the adjusted analysis may use a smaller or systematically different sample from the crude analysis.

Suppose an unadjusted estimate uses 4,000 participants while an adjusted complete-case model uses only 2,600 because several covariates contain missing values. A difference between estimates could reflect both adjustment and the change in analyzed participants.

Check how missing covariate data were handled and whether sensitivity analyses support the result.

Adjustment cannot rescue a fundamentally biased design

Regression is not a statistical washing machine. Adding variables to a model cannot necessarily repair severe selection bias, poor measurement, unmeasured confounding, inappropriate temporal ordering, or a fundamentally incomparable treatment group.

An adjusted estimate may be preferable to a crude estimate while still carrying substantial bias. The appropriate question is how credible the assumptions required for that estimate are.

When Adjustment Changes the Question Instead of Improving the Answer

The lesson is not that fewer covariates are always better. It is that every adjustment should have a reason connected to the effect being estimated.

Common Mistakes When Judging Adjusted Analyses

Ask What Each Adjustment Is Doing to the Estimate

When reading an adjusted result, do not begin with the regression coefficient. Begin with the scientific question and work forward to the variables the authors controlled.

A good adjusted analysis should let you explain, at least conceptually, why the major covariates belong in the model. If the paper provides only a long list of controlled variables with no rationale, your confidence should not increase merely because the analysis looks more sophisticated.

Before Trusting an Adjusted Effect Estimate

Questions About Statistical Adjustment

Adjustment Is a Scientific Decision Before It Is a Statistical One

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.