Will the Proposed Study Add a Comparison That Is Genuinely Missing?

Is the literature missing the comparison that actually matters?

A literature review may show that an intervention, method, technology, policy, or strategy has been studied many times. Yet something important can still be missing: researchers may have compared it with the wrong alternative for the question you need to answer.

Perhaps a new teaching method has repeatedly been compared with no intervention when instructors actually choose between that method and an established teaching approach. Perhaps two widely used treatments have each been compared with placebo but rarely with each other. Or perhaps researchers have compared high and low exposure groups while the meaningful decision concerns two specific forms of exposure.

That can justify a new study, but not simply because a particular pair has never appeared in the literature. There are usually many possible comparisons. The important question is whether the missing comparison prevents researchers from answering a consequential question.

A missing comparison matters when it changes the question the evidence can answer

How to determine whether a missing comparison is worth studying

The comparator helps define the question

A comparison is not an incidental feature added after the research question has been written. In comparative research, it partly determines what question the study can answer.

The PICO framework used widely in intervention research makes this explicit by specifying Population, Intervention, Comparison, and Outcome. Cochrane guidance similarly emphasizes that intervention groups alone do not completely specify a synthesis question; researchers must determine which comparisons will be made. The resulting contrast is what permits an effect or difference to be interpreted relative to something else.

Consider a study finding that students using an AI tutoring system outperform students receiving no additional support. That comparison can provide evidence about AI tutoring versus no additional support. It does not establish that AI tutoring is better than human tutoring, retrieval practice, conventional digital tutoring, or another realistic alternative.

Change the comparator and you change the claim the evidence can support.

Ask what decision or inference requires the comparison

One way to evaluate a proposed comparator is to ask what someone would do differently depending on the result.

If an established intervention is already standard practice, comparing a new intervention only with no intervention may answer a question that decision-makers no longer face. Cochrane guidance specifically notes that where an established intervention is used in practice, comparing a novel alternative with that established intervention may be more informative than comparing it with no intervention.

The same logic applies outside healthcare. If universities deciding whether to adopt a new assessment system must choose between that system and their existing assessment process, the practically relevant comparison may be new versus existing practice, not new versus nothing.

This does not mean active comparators are always superior. A placebo, no-treatment, wait-list, baseline, or other control may be exactly what a particular causal or explanatory question requires. The comparator should follow from the question rather than from a generic hierarchy of control groups.

Different comparators answer different questions

The labels themselves can also hide substantial variation. “Usual care,” “standard practice,” and similar comparators may mean different things across studies or settings. Cochrane therefore recommends specifying comparators clearly, including relevant details of what participants actually receive.

A comparison can be missing even in a crowded literature

Study volume can create the impression that a question has been thoroughly investigated. Yet researchers may repeatedly make the same comparison.

Imagine 25 studies comparing a new learning strategy with no structured learning strategy. If the realistic educational choice is between the new strategy and an established evidence-based strategy, the literature can be large while the decision-relevant comparison remains poorly studied.

This illustrates why the need for new data depends on what the existing evidence permits researchers to conclude, not merely on how many studies exist.

The missing comparator should be relevant, not merely novel

With several possible interventions, methods, or categories, the number of pairwise comparisons can grow rapidly. Many will never have been tested directly.

That mathematical abundance of untested pairs should not be confused with an abundance of important research gaps.

Cochrane guidance on planning syntheses recommends selecting comparisons that address important research and clinical questions rather than treating every possible pairing as equally informative.

A proposed comparison becomes more compelling when there is a clear theoretical, practical, clinical, policy, or methodological reason to distinguish those particular alternatives.

Check whether the comparison is already available indirectly

A lack of direct head-to-head studies does not always mean that no comparative evidence exists.

Suppose studies compare A with C and other studies compare B with C, but few directly compare A with B. Under appropriate assumptions and with a suitable evidence network, methods such as network meta-analysis can combine direct and indirect evidence to estimate relative effects among multiple interventions. Cochrane describes network meta-analysis as a method for simultaneously evaluating multiple competing interventions when the evidence structure permits it.

This does not make direct comparisons unnecessary. Indirect comparisons depend on assumptions, including sufficient comparability across the relevant studies, and direct evidence can still materially improve an evidence network.

The practical lesson is narrower: before declaring a comparison completely absent, determine what comparative information already exists and how direct and credible it is.

Ask whether direct evidence would reduce an important uncertainty

The strongest justification for a new comparison identifies what remains uncertain because that comparison is missing.

Perhaps two interventions both appear beneficial relative to no treatment, but practitioners cannot determine which performs better. Perhaps a technology performs well against traditional instruction, but the real uncertainty concerns whether it adds anything beyond another digital intervention. Perhaps an observational literature compares extreme groups while leaving the practically relevant middle range poorly characterized.

The missing comparison then connects directly to an uncertainty that the proposed study could plausibly reduce.

The comparator must be implemented credibly

Choosing the right comparator conceptually is not enough. It must also be represented fairly in the study.

If a new intervention is implemented intensively while “usual practice” is poorly described or delivered below its normal standard, the resulting contrast may not answer the intended question. Likewise, comparing a polished new technology with an obsolete or artificially weakened alternative can exaggerate the practical relevance of the result.

Comparator details may include content, intensity, duration, personnel, co-interventions, and implementation conditions. Cochrane guidance stresses that these features can affect both the magnitude of observed effects and the applicability of findings.

A new comparison should improve the evidence, not merely expand the design

Adding another group increases cost, recruitment demands, analytical complexity, and often sample-size requirements. The comparison therefore needs a reason to exist.

Ask what would happen if that group were removed. Would the study lose the ability to distinguish between two plausible explanations? Would a real decision remain unanswered? Would the resulting evidence merely reproduce a comparison already well established?

If removing the comparator changes nothing important about what the study can conclude, its contribution may be weak. If removing it eliminates the study's ability to answer the central unresolved question, the comparison is doing substantive work.

When an active comparator changes the question

The contribution does not come from adding a third box to the research design. It comes from aligning the comparison with a question that previous studies could not answer directly.

Common mistakes when claiming that a comparison is missing

Choose the comparator from the question, not from convenience

A defensible comparison should connect directly to what researchers or decision-makers need to distinguish.

The proposal should ultimately be able to say not merely that “A has never been compared with B,” but why knowing the difference between A and B changes what can be concluded or decided.

Before calling a comparison a research gap, check what it would add

Questions about missing comparisons in research

The comparison is valuable when it answers a question the existing evidence cannot

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.