Is It Acceptable to Leave a Nonsignificant Analysis Out of a Paper?

Do You Really Have to Report Every Nonsignificant Analysis You Ran?

You run an additional analysis and nothing particularly interesting appears. The result is nonsignificant, the manuscript is already long, and the analysis seems peripheral to the main argument.

Can you simply leave it out?

Sometimes, yes. Research papers are curated accounts rather than complete transcripts of every command researchers ever executed. But there is an important condition: the reason for omission should not simply be that the result failed to reach statistical significance or contradicted the preferred conclusion.

A Nonsignificant Analysis Can Sometimes Be Omitted, but Not Because It Is Nonsignificant

The Right Question Is Why the Analysis Is Being Omitted

Not Every Analysis Belongs in the Main Manuscript

Real research produces far more analytical output than a journal article can sensibly narrate.

You may inspect distributions, test assumptions, run diagnostics, explore alternative coding decisions, estimate robustness models, investigate incidental patterns, or conduct analyses that later prove irrelevant to the research question.

A manuscript that reports every one of these operations with equal prominence could become less transparent rather than more. Readers need a coherent account of the methods and evidence necessary to evaluate the study, not a chronological dump of statistical output.

ICMJE's current manuscript guidance reflects this distinction. It recommends presenting results in logical sequence, emphasizing or summarizing the most important observations, and placing extra material or technical details in appendices or supplementary materials when appropriate. At the same time, it states that data should be provided for all primary and secondary outcomes identified in the Methods.

So the choice is not between “report absolutely everything” and “report only what worked.”

Statistical Significance Is a Poor Editorial Filter

Consider two analyses that are identical in purpose and relevance. One gives p =.03 and the other gives p =.18.

If you would include the first but omit the second solely because of those p-values, statistical significance is determining what readers get to see.

That is precisely the type of result-dependent selection that can produce reporting bias.

A useful counterfactual question is therefore:

Would I still leave this analysis out if it had produced a statistically significant result supporting my hypothesis?

If the answer is no, reconsider the omission.

Prespecified Primary Analyses Should Not Disappear

If an analysis tests the study's primary hypothesis or primary outcome, omitting it because the result is nonsignificant can fundamentally misrepresent what the research was designed to evaluate.

The same concern can apply to relevant prespecified secondary outcomes and analyses, depending on the study design and reporting framework.

For medical research, ICMJE recommends reporting data on all primary and secondary outcomes identified in the Methods. It also explicitly argues that studies with nonsignificant or inconclusive findings should be submitted or otherwise made publicly available rather than disappearing because of their results.

For randomized trials, CONSORT 2025 requires prespecified primary and secondary outcomes to be identified explicitly.

These specific requirements should not be indiscriminately applied to every research design, but they illustrate a broader reporting principle: planned central evidence does not become optional when the answer is inconvenient.

An Exploratory Analysis Is Different From a Prespecified Test

Suppose you run 20 exploratory correlations while getting to know a dataset. Must all 20 appear in the manuscript?

Not necessarily.

Some may be irrelevant to the eventual research question, statistically inappropriate after further examination, redundant, or simply part of exploratory data analysis rather than inferential claims made in the paper.

However, selection becomes more concerning if you report the one statistically significant exploratory correlation while silently discarding the 19 unsuccessful searches. The reader may then interpret that isolated result as though it arose from a much narrower analytical process.

This is where omission intersects with p-hacking. The problem is not merely that unsuccessful analyses were left out. The problem is that the successful analysis was selected from a broader search and presented without the context needed to interpret it.

Robustness Analyses Become Important When They Change the Conclusion

Suppose your preferred model gives p =.03, while several equally defensible specifications produce estimates closer to zero and nonsignificant results.

Those alternatives are not disposable simply because they complicate the conclusion. Their disagreement tells readers that the result is sensitive to analytical choices.

Conversely, if numerous robustness analyses produce essentially the same substantive conclusion, you may not need to describe every model in the main text. A concise summary, table, supplementary appendix, or repository can communicate the relevant stability without exhausting the reader.

This is why analytical flexibility becomes questionable when unfavorable specifications are systematically hidden while favorable ones receive privileged treatment.

Diagnostic Analyses Often Serve a Different Purpose

Not every analysis tests a substantive hypothesis.

Researchers may run residual diagnostics, check model assumptions, inspect influential observations, compare coding approaches during data cleaning, or verify that an algorithm behaves as expected. Reporting requirements for these procedures depend on their relevance to evaluating the final analysis.

You do not necessarily need to reproduce every diagnostic statistic in the main manuscript. But if a diagnostic reveals a serious violation and your final analysis depends on how you addressed it, that information becomes methodologically relevant.

The distinction is therefore not “significant analyses must be reported, diagnostics may be omitted.” It is whether readers need the information to understand why the final method and conclusion are defensible.

Omission Can Become Selective Reporting Even Without Changing an Outcome

Imagine that a paper reports three models supporting the hypothesis but omits two equally relevant models that do not.

No outcome has necessarily been changed. Yet the reported evidence may still be selectively favorable.

This illustrates why selective outcome reporting is only one form of a broader selective-reporting problem. Researchers can selectively report models, subgroups, time points, specifications, outcomes, or entire analyses.

Supplementary Material Is Often Better Than Disappearance

Space constraints are real, although electronic publishing has made them less absolute than they once were. ICMJE specifically notes that electronic formats allow additional detail and that supplementary material can contain technical information without interrupting the main text.

If an analysis is relevant but too detailed for the main narrative, supplementary material may provide a sensible compromise.

Repositories can also preserve analysis code, robustness checks, and additional outputs where appropriate. The main article should tell readers enough to know that these analyses exist and why they matter.

Three Nonsignificant Analyses, Three Different Reporting Decisions

All three analyses are nonsignificant. Yet “nonsignificant” does not determine whether they belong in the paper.

Analysis A belongs because it tests the central planned question. Analysis B belongs because it reveals that the conclusion depends on analytical specification. Analysis C may reasonably be omitted because it is peripheral to the research report, not because it failed to cross a significance threshold.

Now imagine that Analysis C had produced p =.002 and the researcher suddenly decided to build a major conclusion around it. The previously irrelevant exploratory search would become relevant to interpretation. Readers would need enough context to understand how that finding was discovered.

Common Misunderstandings About Omitting Nonsignificant Analyses

A Better Rule Than “Report Every Nonsignificant Result”

Instead of asking whether a result is significant, ask what role the analysis plays in the evidential structure of the paper.

One practical test catches many problematic omissions: imagine that the result had been strongly significant in the predicted direction. Would you suddenly consider the analysis central enough to report?

If so, its current “irrelevance” may be partly a consequence of the result itself.

Before Leaving an Analysis Out of Your Paper

Frequently Asked Questions About Omitting Nonsignificant Analyses

Nonsignificance Is Not a Reason for an Analysis to Disappear

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.