Which Conclusions Are Mostly Based on Indirect Evidence?

How Directly Does the Evidence Actually Address Your Conclusion?

A study can be rigorous, precisely estimated, and internally convincing while still providing only indirect evidence for the conclusion you want to make.

Perhaps researchers studied university students while your conclusion concerns secondary-school learners. Perhaps an intervention improved engagement, but the claim concerns academic achievement. Perhaps studies compared two interventions only through their separate comparisons with a third option. Or perhaps researchers measured an appealing proxy rather than the outcome that actually matters.

The central question is therefore not simply whether the evidence is strong. It is: how many inferential steps separate the evidence researchers actually collected from the conclusion you want to draw?

Indirect Evidence Requires an Inferential Bridge

Indirectness Is About the Distance Between Evidence and Question

Formal evidence-assessment frameworks recognize indirectness as a distinct reason for reducing confidence in a conclusion. In GRADE, certainty is assessed for particular outcomes by considering risk of bias, inconsistency, indirectness, imprecision, and publication bias. Indirectness concerns how closely the available evidence matches the question being answered.

Cochrane describes two broad forms. Evidence may address a restricted version of the question because the studied population, intervention, comparator, or outcome differs from the one of interest. Evidence may also rely on an indirect comparison, such as comparing intervention A with intervention B through studies in which each was separately compared with another option.

Population Indirectness Occurs When the People Studied Are Not the People in the Claim

Suppose a literature consistently finds that an intervention improves performance among university students. If you want to conclude that it improves performance among primary-school pupils, an inferential step has appeared.

The university evidence may still be relevant. But confidence in transferring the conclusion depends on whether differences between the populations could plausibly modify the effect. Age, prior knowledge, developmental stage, language, socioeconomic circumstances, institutional conditions, or other characteristics may matter depending on the phenomenon.

The appropriate response is not automatically to reject the evidence. It is to ask whether the population difference is consequential for this particular conclusion.

Outcome Indirectness Occurs When Researchers Measure Something Else

One of the most common inferential leaps occurs between what researchers measure and what authors ultimately claim.

Student satisfaction is not academic achievement. Intention to use a technology is not sustained adoption. Engagement is not necessarily learning. Immediate test performance is not long-term retention.

These outcomes may be related, sometimes strongly. Yet evidence about one does not automatically establish another.

An Intermediate Outcome Can Support a Mechanism Without Establishing the Final Outcome

Suppose an educational intervention increases the amount of time students spend practicing. If practice plausibly contributes to achievement, the result provides evidence relevant to a possible pathway.

It does not by itself establish that achievement improved. Additional links in the chain must hold: the additional practice must be productive, sufficiently substantial, and capable of affecting the eventual outcome.

The longer the chain of required assumptions, the more carefully the conclusion should be stated.

Intervention or Exposure Indirectness Matters When What Was Studied Differs From What Is Being Recommended

A literature may evaluate one version of an intervention while conclusions are drawn about a broader category. For example, studies might examine a highly structured AI tutoring system with teacher supervision, yet the synthesis concludes that “AI improves learning.”

The broader claim removes features that may have been responsible for the observed effect. If implementations differ substantially, evidence about one should not automatically be treated as direct evidence about all.

This is closely related to context-dependent conclusions. Sometimes what initially looks like indirectness reflects uncertainty about whether a result transfers to another implementation or setting.

Indirect Comparisons Introduce Another Evidential Step

Suppose researchers want to know whether intervention A is better than intervention B, but no study compares them directly. Instead, some trials compare A with C and others compare B with C.

Those studies can potentially support an indirect comparison between A and B, particularly within appropriately conducted network meta-analysis. But the inference relies on assumptions about the comparability of the evidence contributing to those different comparisons. GRADE therefore treats indirect comparisons as another source of potential indirectness.

Mechanistic Evidence Can Be Powerful but Still Indirect for an Outcome Claim

Evidence that a proposed mechanism operates can strengthen the plausibility of a conclusion. Laboratory evidence might show that a particular cognitive process occurs, for example, while observational evidence shows the expected behavioral pattern.

Yet evidence for a mechanism is not automatically direct evidence for the magnitude of a real-world outcome. The mechanism may operate alongside competing processes, be too weak to produce a practically important effect, or behave differently outside the conditions under which it was observed.

This does not make mechanistic evidence unimportant. It means its evidential role should be described accurately.

Directness Is Relative to the Exact Question

The same study can be direct evidence for one conclusion and indirect evidence for another.

A survey asking teachers whether they use generative AI is direct evidence about reported AI use. It may be less direct evidence about actual behavioral frequency if reporting is inaccurate. It is much more indirect as evidence that AI use improves student learning.

Before judging indirectness, write the conclusion precisely. Otherwise, there is no stable target against which directness can be assessed.

Indirect Does Not Mean Weak in Every Respect

A large randomized experiment can provide highly credible evidence about the population and outcome it actually studied while remaining indirect for another population or outcome. Conversely, a study can be perfectly direct but seriously biased.

These are separate dimensions. GRADE treats indirectness independently from risk of bias, inconsistency, and imprecision for exactly this reason.

Multiple Indirect Steps Can Accumulate

Indirectness becomes particularly consequential when several extrapolations occur simultaneously. Imagine evidence from university students showing that an intervention increases self-reported engagement over two weeks, followed by a conclusion that the intervention will improve long-term achievement among secondary-school students.

The claim crosses population, outcome, and time. Each transition may be plausible. Together, however, they require substantially more assumption than the original evidence.

When Engagement Evidence Becomes a Learning Claim

The positive engagement studies are not irrelevant. They simply answer a different question more directly. Once the outcome in the conclusion is specified, the evidential structure becomes much easier to see.

Common Mistakes When Using Indirect Evidence

Draw the Inferential Chain Before Writing the Conclusion

For each important conclusion, write down what the studies directly observed. Then identify every step required to move from those observations to your proposed claim.

A useful synthesis often separates levels of directness. You might conclude that evidence is direct and relatively consistent for improved engagement, limited but direct for short-term achievement, and largely indirect for long-term academic outcomes.

That is more informative than assigning one confidence label to the entire literature.

Check How Directly the Evidence Supports Your Conclusion

Questions About Indirect Evidence

Count the Inferential Steps Between Evidence and Claim

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.