Which Study Should You Replicate When Several Findings Need Verification?

When Several Studies Need Verification, Which One Deserves Your Replication?

Suppose you have identified several published findings in your field that could reasonably be replicated. One is influential but already supported by several studies. Another is based on a small study and remains uncertain. A third has practical consequences for policy or professional practice. A fourth would be straightforward to reproduce with the resources available to you.

All four may be legitimate candidates, but they are not necessarily equally valuable replication targets. Research time, participants, funding, laboratory access, data collection, and researcher attention are limited. Choosing which study to replicate is therefore partly a question of where additional evidence would be most informative.

The decision should not be reduced to citation counts, convenience, or whether a finding happens to be controversial. What matters is the relationship among the importance of the claim, the uncertainty surrounding it, the consequences of being wrong, what evidence already exists, and whether your replication could meaningfully reduce that uncertainty.

Prioritize the Replication That Could Resolve the Most Important Uncertainty

Replication Priority Depends on the Value of the Evidence You Could Add

There is no universal ranking system that tells researchers which published study should be replicated next. The National Academies of Sciences, Engineering, and Medicine recommends considering factors such as whether results matter for individual or policy decisions and whether they could make a substantial contribution to basic scientific knowledge when resources are directed toward replication research.

A useful way to approach the choice is to ask a slightly different question: If I replicate this study well, how much could the result change what researchers or decision-makers reasonably believe?

This shifts attention from the characteristics of the original paper to the informational value of the proposed replication.

How Important Is the Underlying Claim?

Some findings occupy more consequential positions in a field than others. A claim may underpin a widely used theory, justify a common intervention, influence policy, support a measurement instrument, or serve as the premise for substantial subsequent research.

Importance is not identical to popularity. A highly cited paper may be influential, but citation counts alone do not establish that its central finding deserves replication. Conversely, a less famous finding may matter considerably within a specialized field or professional context. This is why choosing between a famous result and one that actually matters to your field requires more than comparing bibliometric visibility.

Ask what would change if the finding were less reliable than currently assumed. If the answer is "very little," the case for prioritizing it may be weaker. If substantial theory, practice, policy, or subsequent research depends on the claim, independent verification may have much greater value.

How Much Genuine Uncertainty Remains?

Replication is especially informative when reasonable uncertainty remains about whether a finding is reliable or about the conditions under which it occurs. Nosek and Errington characterize replication in terms of whether possible outcomes would provide diagnostic evidence about a claim from prior research. From this perspective, the purpose is not simply to repeat procedures. It is to confront an existing claim with new evidence.

Consider what is already known. Has the finding been independently tested? Do multiple studies converge on it? Are the available studies methodologically strong? Are results inconsistent? Has most of the evidence come from the same research group, population, dataset, or experimental setting?

A finding that has never received independent replication may warrant attention precisely because apparent confidence rests on limited independent evidence. At the other extreme, another replication of an extensively tested finding may add relatively little unless there is a specific unresolved question.

What Are the Consequences If the Finding Is Wrong?

Uncertainty matters more when decisions depend on the result. A finding used to justify educational practice, clinical decisions, public policy, organizational procedures, or resource allocation may merit verification even when it is not theoretically fashionable.

This creates an important distinction between uncertainty and consequential uncertainty. Many findings are uncertain. Fewer are uncertain in ways that could substantially affect what people do.

The National Academies specifically identifies importance for individual and policy decision-making as a consideration when directing resources toward replication. A claim with major consequences but weak supporting evidence can therefore be a particularly strong replication candidate.

Would Another Study Actually Reduce the Uncertainty?

A study should not receive priority merely because its evidence is weak. You also need to ask whether your proposed replication can improve the evidential situation.

Imagine an original study with serious design problems. Repeating those problems faithfully might demonstrate whether the same result appears under the same flawed conditions, but it may do little to establish whether the underlying scientific claim is credible. In such circumstances, the more useful investigation might involve a stronger design, an extension, or a different test of the claim.

This is especially relevant when considering whether a weak original study is worth replicating. Weak evidence can increase the need for verification, but methodological weakness can also reduce what an exact repetition would teach you.

How Much Independent Evidence Already Exists?

Replication priorities should be informed by the broader evidence base rather than by one paper viewed in isolation. A single dramatic study may appear to demand replication until you discover that several independent teams have already tested the same claim. Another seemingly ordinary study may represent almost the entire empirical foundation for an important proposition.

Before selecting a target, search for direct replications, conceptual replications, extensions, systematic reviews, meta-analyses, registered reports, dissertations, preprints, and relevant null findings. Publication status and terminology can make replication evidence surprisingly easy to miss.

Remember that neither one successful replication nor one unsuccessful replication should normally be treated as the final verdict. Replicability concerns a body of accumulating evidence, and disagreements among studies can sometimes reveal boundary conditions, methodological differences, or previously unrecognized sources of variation.

Could Your Replication Provide Evidence That Existing Studies Cannot?

The strongest replication candidate may not simply be the study with the least evidence. Your research may be particularly informative because you can test the claim independently, use a more appropriate sample, achieve substantially greater statistical precision, reproduce an important procedure faithfully, or examine a theoretically meaningful condition not represented in the existing evidence.

This is where the distinction between replication and extension becomes consequential. Changing a population, context, measure, or procedure can make a replication more informative, but too many changes may shift the research question away from whether the original claim holds. You should therefore decide what claim you are actually trying to test before deciding how closely the new study must resemble the original.

Can You Conduct the Replication Rigorously?

Scientific importance does not eliminate practical constraints. A replication that requires equipment, specialist expertise, sample access, intervention fidelity, proprietary materials, or a sample size beyond your resources may not be feasible.

Feasibility should function as a constraint rather than as the sole selection criterion. Choosing the easiest study simply because it can be completed quickly risks producing low-value duplication. Yet choosing an extraordinarily important study that you cannot test adequately may produce equally weak evidence.

Think in Terms of Expected Informational Value, Not a Mechanical Score

You can organize candidate studies around several questions without pretending that each criterion can be measured precisely.

These criteria should support judgment rather than replace it. The relative importance of each factor will vary by discipline and research problem. Ethical costs may also matter substantially in fields involving human participants, animals, scarce samples, expensive equipment, or potentially harmful interventions.

Choosing Among Three Plausible Replication Targets

A different conclusion could follow if one assumption changed. If the team lacked access to the population or resources required to test Finding B rigorously, another candidate might become preferable. Replication priority is therefore a comparative judgment about both what needs to be known and what your proposed study can credibly establish.

Common Mistakes When Choosing What to Replicate

Build Your Justification Around Why This Replication Deserves Priority

If several candidate findings appear worth replicating, do not begin by asking which paper you like most. Map the evidence around each claim first. Your justification should explain why additional evidence is needed, what uncertainty remains, why resolving that uncertainty matters, and why your proposed study is capable of doing so.

Finally, make the prioritization logic explicit in your proposal or manuscript. You do not need to claim that the selected study is objectively the single most important replication possible. A more defensible argument is that, relative to plausible alternatives, it addresses an important unresolved claim and can contribute evidence capable of changing the state of knowledge.

That is also a stronger basis for justifying replication without pretending that the project is completely novel. The contribution lies in reducing a meaningful uncertainty, not in disguising verification as discovery.

Before Choosing a Study to Replicate

Questions About Prioritizing Replication Studies

Replicate Where New Evidence Can Matter Most

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.