Have You Considered the Strongest Argument Against Doing the Study?

What Would a Well-Informed Skeptic Say About Your Proposed Study?

By the time researchers develop a proposal, they usually know how to defend it. They can explain the gap, describe the contribution, justify the methods, and point toward the findings they hope the study will produce.

That familiarity creates a less comfortable problem: you may become much better at defending the study than evaluating whether it should exist.

Try reversing the task. Assume that an informed researcher understands the topic, takes the underlying problem seriously, and nevertheless thinks your particular study should not be conducted. What is the strongest case that person could make?

Perhaps existing evidence already answers enough of the question. Maybe the proposed design cannot distinguish the explanations that matter. Participant burden is disproportionate. Recruitment assumptions are unrealistic. The expected contribution is too incremental. Or a different study could produce substantially better evidence for the same effort.

If you cannot articulate the strongest objection yourself, you may not yet have stress-tested the idea seriously enough.

A Strong Research Idea Should Survive Its Best Counterargument

How Do You Build a Serious Case Against Your Own Study?

This exercise resembles a pre-mortem used in project planning. Instead of asking what might go wrong in the abstract, you temporarily assume that the project has failed and work backward to identify plausible reasons. Gary Klein's description of the pre-mortem emphasizes creating space for knowledgeable people to voice reservations during planning rather than after failure has occurred.

Research evaluation already incorporates related logic. Current NIH review criteria ask whether the scientific background justifies undertaking a proposed study, whether its approach is rigorous and feasible, whether emerging challenges can be addressed, and whether the investigators and environment provide the necessary expertise and resources. Those criteria are specific to NIH-funded biomedical and behavioral research, but they illustrate a useful general principle: a study deserves scrutiny not only for what could go right, but also for the weaknesses that could prevent it from producing valuable evidence.

Attack the Need for the Study Before Attacking the Methods

The strongest objection may occur before methodology enters the discussion.

Ask whether the research problem is actually supported by evidence. Perhaps the problem is based largely on anecdote or an exaggerated reading of the literature. If the problem is genuine, ask whether existing evidence has already reduced the important uncertainty sufficiently.

A flawless method cannot rescue a study that does not need to be conducted.

Challenge the Claimed Research Gap

Imagine a skeptical reviewer saying, “You have not found a gap. You have found studies using different terminology.”

Could that criticism be correct?

Search adjacent disciplines, alternative terminology, recent reviews, citation networks, and evidence from comparable populations. Try to make the proposed gap disappear. If it survives, your justification becomes stronger. If it disappears, discovering that before data collection is useful.

The relevant test is whether the gap remains genuine after a sufficiently serious search.

Challenge the Importance of Filling the Gap

Now suppose the gap is real. A critic can still ask, “So what?”

There are countless things researchers do not know. Why does this particular uncertainty deserve participants, funding, time, and attention?

Construct the strongest argument that nothing consequential would change even if the study answered its question perfectly. If that argument is persuasive, the project may involve a gap that is genuine but not important enough to fill.

Attack the Link Between the Question and the Evidence

A critic might accept that the question matters but reject the study as incapable of answering it.

Perhaps the study asks a causal question using evidence that supports only association. Maybe the outcome is represented by a weak proxy. The proposed participants differ substantially from the population named in the question. The design cannot distinguish between competing explanations.

The strongest objection here is not “the method could be better.” It is “even if you execute this method perfectly, the evidence will not support the answer you intend to claim.”

That criticism should be resolved before proceeding.

Attack the Feasibility Assumptions

Imagine the study six months after launch. Recruitment is at one-third of the expected rate. A required dataset has not been released. The specialist collaborator is unavailable. The budget cannot support the planned transcription. Which of those events would seriously damage the project?

Current NIH review guidance explicitly defines feasibility in terms of whether an approach is sound and achievable, including plans for problems or new challenges that emerge. It also separately evaluates recruitment, retention, timelines, expertise, and resources where relevant.

Use the same discipline informally. Challenge whether the participants can really be reached, whether required data are obtainable, and whether the time, expertise, and resources actually exist.

Ask Whether the Study Creates More Burden Than Knowledge

Another strong argument may concern proportionality.

Suppose participants must complete repeated assessments, disclose sensitive information, travel, or remain in a study for months. What does the project offer in return scientifically? If the likely information gain is modest, participant burden can become a serious objection rather than a procedural detail.

This does not mean that burdensome research is inherently unjustified. It means the burden should be proportionate to the likely value of the knowledge and unnecessary demands should be removed.

Ask Whether a Different Study Would Make Yours Redundant

The strongest objection may not be “do not investigate this question.” It may be “do not investigate it this way.”

Perhaps a longitudinal design would answer the temporal question better than another cross-sectional survey. Existing administrative data could answer the question with less participant burden. A replication would address the uncertainty more directly than introducing another predictor. A synthesis of existing studies might be more informative than collecting another small dataset.

This distinction matters because a worthwhile question does not automatically justify your preferred study.

Distinguish Fatal Objections From Fixable Weaknesses

Not every criticism should stop the project.

The distinction prevents two opposite mistakes: defending a fundamentally weak study because much work has already gone into it, or abandoning a worthwhile question because the first design contains repairable problems.

Invite Someone Who Is Not Invested in the Idea

Self-critique has limits. You know what you intended, which makes it surprisingly easy to supply missing logic unconsciously.

Ask a knowledgeable colleague to argue against the study. Do not ask whether they “like the idea.” Ask them to identify the single strongest reason not to spend resources on it.

A useful critic should understand the research sufficiently to attack its strongest version rather than an obvious caricature. The goal is not negativity. It is adversarial quality control before the study becomes expensive to change.

Write the Objection in Its Strongest Form

A weak counterargument is easy to defeat: “Some people may not find the topic interesting.” That teaches you little.

A stronger version might be: “Existing longitudinal evidence already answers the central question with greater internal validity than your proposed cross-sectional study, so the new data are unlikely to reduce meaningful uncertainty.”

Now you have something worth answering.

Your Response Should Change the Study When the Criticism Is Right

The purpose of this exercise is not to produce a rebuttal paragraph for a proposal.

If the objection exposes a genuine problem, revise the project. Narrow the question. Improve measurement. Secure another site. Change the design. Remove participant burden. Search the literature again. Or stop.

A counterargument that changes nothing regardless of its strength has become theatre rather than evaluation.

What Happens When the Strongest Objection Is Better Than Your Original Rationale?

The strongest argument against the original study did not kill the research problem. It exposed that the first version of the study was not the best use of that problem.

What Makes This Stress Test Too Easy to Pass?

Run a Research Pre-Mortem Before You Commit

Imagine that the study has been completed and knowledgeable researchers conclude that it was not worth conducting. Write down the most plausible reason they give.

Then strengthen that criticism until you would regard it as fair if it appeared in a rigorous peer review.

Once the strongest objection is visible, go one level deeper. Identify the assumptions most likely to make the study fail. A broad objection often becomes actionable only when translated into specific assumptions that can be tested before the study begins.

Have You Made the Best Case Against Your Own Study?

Questions About Challenging Your Own Research Idea

Your Study Should Survive More Than Its Own Rationale

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.