03 · What You Need to Know
Recognizing When an Indirect Outcome Cannot Carry the Scientific Claim
Indirectness Is a Relationship Between Evidence and a Claim
A measure is not inherently too indirect in every context.
Its adequacy depends on what researchers intend to conclude from it.
For example, the number of learning platform logins is a direct measure of recorded login activity, assuming the system records that activity accurately.
It is a much more indirect indicator of student engagement and an even less direct measure of learning achievement.
If researchers ask whether an intervention increases platform access, login frequency may be appropriate.
If they ask whether the intervention improves conceptual understanding, the same measure is insufficient without additional evidence.
Researchers should therefore evaluate indirectness relative to the target construct and intended inference rather than judging an indicator in isolation.
Distinguish Construct Indirectness From Surrogate Endpoint Validity
Two problems are often combined under the label of surrogate measurement.
The first concerns construct validity: does the observed indicator adequately represent the phenomenon researchers claim to measure?
The second concerns surrogate validity: can the indicator legitimately substitute for another outcome, particularly when estimating an intervention's effect on that outcome?
For example, platform logins may inadequately represent cognitive engagement. That is primarily a construct measurement problem.
Even a valid measure of cognitive engagement may not reliably predict an intervention's effect on long-term academic achievement. That is a surrogate validity problem.
Researchers should diagnose both issues separately because improving the measurement of an intermediate construct does not automatically validate it as a surrogate for a final outcome.
A Conceptual Chain Can Reveal the Inferential Gap
Consider a hypothetical educational intervention.
The greater the number of unexamined assumptions between the measurement and the final claim, the more carefully researchers should evaluate the inference.
However, counting causal steps is not a formal validation method.
A relatively indirect biomarker may be useful when extensive evidence supports its relationship to the final endpoint. A seemingly close intermediate outcome may be unreliable when the relationship has not been validated.
Scientific adequacy depends on evidence, not visual proximity in a conceptual diagram.
Prognostic Validity Does Not Establish Surrogate Validity
A variable may predict who experiences a favorable outcome without predicting whether an intervention will improve that outcome.
For example, students with high immediate examination scores may generally demonstrate stronger long-term performance.
That association could reflect previous knowledge, motivation, educational opportunities, or other characteristics.
An intervention designed specifically to improve immediate examination performance might not affect those underlying factors.
Consequently, the intervention could raise test scores without producing comparable improvements in long-term competence.
Researchers should distinguish prediction of individual outcomes from prediction of intervention effects.
This distinction is central to the surrogate endpoint literature developed from Prentice's framework and subsequent validation methods.
Trial-Level Surrogacy Is Particularly Important for Treatment Claims
When a surrogate is used to infer an intervention's effect on a final outcome, evidence across interventions or trials can be highly informative.
Trial-level surrogacy examines whether treatment effects on the surrogate correspond to treatment effects on the final endpoint across relevant comparisons.
For example, researchers may investigate whether interventions producing larger changes in a biomarker also produce larger improvements in clinically important outcomes.
Buyse and colleagues describe approaches to surrogate validation using meta-analytic evidence from randomized experiments.
However, strong trial-level association does not automatically establish that the relationship will apply to every future intervention.
Differences in mechanisms, populations, and study conditions may limit transportability.
Researchers should evaluate the scope of validation rather than treating a surrogate as universally established.
The Surrogate Paradox Shows Why Favorable Intermediate Effects Can Mislead
An intervention may improve a surrogate while failing to improve, or even worsening, the final outcome.
This possibility is commonly discussed as the surrogate paradox.
For example, a hypothetical instructional intervention may improve immediate assessment performance through intensive test preparation while reducing opportunities for deeper application.
It is therefore possible for immediate scores to improve without a corresponding benefit in later independent problem-solving.
In clinical research, an intervention may influence the surrogate and final outcome through different pathways.
These possibilities demonstrate why the causal structure matters.
A favorable surrogate effect is not sufficient proof of a favorable final effect.
Unmeasured Pathways Can Undermine a Surrogate
A proposed surrogate may capture only one of several pathways through which an intervention affects the final outcome.
Suppose an educational intervention improves writing performance partly by increasing revision quality.
It might also affect motivation, instructional time, feedback interpretation, or other processes.
Revision quality may be a meaningful mediator without fully representing the intervention's total effect on writing performance.
If researchers treat it as a complete substitute for the final outcome, important pathways may be overlooked.
Similarly, a clinical treatment may affect a biomarker while having additional beneficial or harmful effects unrelated to that biomarker.
Researchers should consider whether the surrogate captures enough of the relevant causal information for the intended claim.
A Strong Correlation Can Conceal Important Limitations
Suppose a proposed surrogate and final outcome have a strong observed correlation.
This may be encouraging, but it does not establish that changing the surrogate will change the final outcome.
The correlation could arise from common causes, selection processes, or relationships that do not remain stable under intervention.
Furthermore, correlations estimated in one population may not apply to another.
Researchers should avoid relying on an arbitrary correlation coefficient as a universal criterion for surrogate validity.
Validation requires a framework appropriate to the causal and predictive claim being made.
Surrogate Validity May Depend on the Intervention Mechanism
A surrogate that performs well for one class of interventions may fail for another.
For example, an immediate knowledge assessment may predict later competence among students receiving a particular instructional approach.
However, an intervention focused narrowly on examination preparation may alter the immediate score without producing comparable long-term learning.
The same surrogate can therefore have different relationships to final outcomes across intervention mechanisms.
Researchers should examine whether validation evidence applies to the intervention being evaluated.
Transporting surrogate relationships across substantially different interventions requires additional justification.
Population Differences Can Limit Surrogate Usefulness
Surrogate relationships may also vary across populations.
For example, an early educational assessment may have different predictive properties among novice learners and experienced professionals.
Similarly, clinical surrogate relationships may depend on disease stage, comorbidities, or other relevant characteristics.
Researchers should evaluate whether the validation population resembles the intended study population in ways relevant to the surrogate-final outcome relationship.
However, demographic similarity alone does not establish transportability.
The mechanisms and measurement conditions underlying the relationship also matter.
Measurement Timing Can Make a Surrogate Misleading
Even a theoretically relevant surrogate may be assessed too early or too late to capture the intended process.
For example, an immediate post-intervention knowledge test may reflect short-term acquisition but not durable retention.
A later assessment may better represent persistence, although it still may not establish professional competence.
Researchers should consider the expected timing of intervention effects and the relationship between the surrogate and final outcome.
Repeated measurement may help characterize the trajectory of an intermediate process.
However, collecting more time points does not automatically validate the surrogate relationship.
Measurement Precision Cannot Compensate for Conceptual Misalignment
A surrogate can be measured very precisely while remaining inappropriate for the intended inference.
For example, a digital platform may record every click with negligible recording error.
Those data may be excellent measures of recorded interactions.
They are not necessarily valid measures of understanding, learning quality, or long-term competence.
Researchers should distinguish technical accuracy from construct validity and surrogate validation.
A large sample can produce narrow confidence intervals around an estimate of the wrong outcome.
Statistical precision does not repair an unsupported interpretation.
How Much Validation Evidence Is Enough?
There is no universal evidence threshold suitable for every surrogate and every intended use.
The required strength of evidence depends on the final outcome, intervention, population, available studies, and consequences of an incorrect decision.
For example, using an exploratory biomarker to guide future research may require a different evidentiary standard from using it to support a major clinical effectiveness claim.
Likewise, an educational study may legitimately examine short-term performance without claiming long-term professional competence.
Researchers should specify the intended use of the surrogate and evaluate whether the validation evidence is adequate for that use.
Where uncertainty is substantial, conclusions should be correspondingly limited.
Surrogates Can Remain Useful Even When They Are Not Validated Replacements
A proposed surrogate may still provide valuable information about an intervention's immediate effects or possible mechanisms.
For example, an early knowledge assessment may show whether students acquired particular concepts.
That is a legitimate educational outcome even if it does not establish long-term professional performance.
Similarly, a biomarker may help investigate biological processes without serving as a validated substitute for a clinical endpoint.
The appropriate response to insufficient surrogate validation is not necessarily to discard the measurement.
It may be to narrow the claim to what the measurement actually establishes.
When Should Researchers Reject a Proposed Surrogate?
A surrogate may be inappropriate for the intended purpose when it poorly represents the intermediate construct, lacks credible evidence linking intervention effects to the final outcome, or has validation evidence that does not apply to the study context.
It may also be unsuitable when important alternative pathways could reverse the relationship between surrogate improvement and final benefit.
Researchers should consider the consequences of a false inference.
For example, a policy decision affecting many students or patients may require stronger evidence of meaningful benefit than a preliminary feasibility study.
Rejecting a surrogate for one claim does not mean the variable has no scientific value.
It means that the evidence does not support the proposed substitution.
What Alternatives Are Available?
When a surrogate is insufficient, researchers can consider several alternatives.
They may measure the final outcome directly, extend follow-up, use an appropriate existing data source, or conduct a subsequent study.
They may also redefine the primary research question around the intermediate outcome that can be measured adequately.
For example, a study unable to assess professional competence may focus on immediate problem-solving performance rather than claim long-term professional effectiveness.
The choice depends on feasibility, ethical considerations, measurement quality, and the intended decision.
See whether surrogate outcomes should be used when final outcomes take too long to measure for the broader design considerations.
Use a Surrogate Indirectness Audit
| Potential weakness |
Diagnostic question |
Interpretive consequence |
| Construct mismatch |
Does the surrogate represent the intended intermediate phenomenon? |
Even the intermediate outcome may be inadequately measured |
| Weak prognostic evidence |
Does the surrogate predict the final outcome? |
Its predictive usefulness may be limited |
| Absent intervention-effect validation |
Do changes caused by interventions predict final outcome effects? |
Claims about treatment benefit may be unsupported |
| Alternative causal pathways |
Can the intervention affect the final outcome independently? |
Surrogate improvement may not reflect total benefit |
| Population mismatch |
Does validation evidence apply to the intended participants? |
Transportability is uncertain |
| Intervention mismatch |
Was validation conducted for comparable intervention mechanisms? |
Surrogate relationships may not generalize |
| Timing mismatch |
Is the surrogate measured when the relevant process occurs? |
Important changes may be missed or misinterpreted |
| Decision consequences |
What happens if the surrogate gives a misleading signal? |
Stronger final-outcome evidence may be necessary |
The audit is intended to reveal assumptions and evidence gaps, not to calculate a universal surrogate validity score.
Watch Out
Do not infer that an intervention improves a final outcome merely because it changes a related indicator in the expected direction. A surrogate can be prognostic, theoretically plausible, and precisely measured while still failing to predict intervention effects on the outcome of greatest interest.
