01 · The Question
What Happens When a Study Measures More Outcomes Than Readers Ever See?
A study measures six outcomes. Two support the hypothesis. Four do not.
The paper discusses the two favorable outcomes in detail. One unfavorable outcome appears briefly without enough information to evaluate it. The other three never appear.
The reported results may all be technically true. Yet the paper can still give readers the wrong impression about the overall evidence. This is the central problem behind selective outcome reporting .
03 · What You Need to Know
Selective Outcome Reporting Can Occur Even When the Study Itself Is Published
Publication Bias and Outcome Reporting Bias Are Different Problems
Reporting bias can occur at more than one level.
Publication bias
Whether an entire study becomes publicly available may depend on the nature or direction of its results.
Selective outcome reporting
The study is published, but whether or how particular outcomes appear in the report depends on their results.
A published article therefore does not necessarily reveal all the relevant evidence generated within that study.
This matters to individual readers, but it also matters to evidence synthesis. If statistically favorable outcomes are more likely to be reported completely, systematic reviews and meta-analyses built from published reports can inherit the distortion.
Selective Outcome Reporting Is More Than Simply Leaving a Variable Out
The most obvious form is complete omission: an outcome was measured or prespecified but disappears from the paper.
Selection can also be subtler.
Reporting practice
Example
Potential effect
Complete omission
A prespecified nonsignificant outcome is not mentioned
Readers may assume it was never measured
Incomplete reporting
An outcome is mentioned but estimates or uncertainty are insufficient for evaluation
The result cannot be adequately assessed or incorporated into evidence synthesis
Selective time point
A favorable three-month result is reported while an unfavorable prespecified 12-month result is omitted
The intervention may appear more effective or durable than the full evidence indicates
Selective measurement
One favorable scale is reported while another prespecified measure of the same domain disappears
Readers see the operationalization producing the preferred result
Selective prominence
A favorable secondary outcome dominates the abstract while the primary result receives little attention
The apparent importance of findings may differ from their prespecified role
Changed outcome status
A favorable secondary outcome becomes primary in the paper
The finding can appear to have been the study's central planned test
Some of these practices overlap with outcome switching . The concepts are related, but selective outcome reporting is broader because an outcome can remain officially “secondary” and still be selectively omitted or incompletely reported.
Statistical Significance Can Influence Which Outcomes Reach the Paper
One influential empirical study compared protocols with publications for 102 randomized trials. Chan and colleagues identified 3,736 efficacy and harm outcomes and found that 50% of efficacy outcomes and 65% of harm outcomes per trial were incompletely reported. Statistically significant outcomes had higher odds of being fully reported than nonsignificant outcomes for both efficacy and harm data.
The researchers also found that 62% of the trials had at least one primary outcome that had been changed, introduced, or omitted between the protocol and publication.
A separate review of 519 randomized trials likewise documented incomplete outcome reporting and examined its relationship with statistical significance.
These studies concern randomized trials and should not simply be treated as prevalence estimates for every research discipline. They nevertheless illustrate the mechanism clearly: when favorable results have a better chance of complete reporting, the published literature can systematically look more positive than the underlying research.
Nonsignificant Does Not Mean Unreportable
A common pathway to selective reporting begins with a seemingly harmless judgment: “Nothing happened for this outcome, so there is nothing to report.”
A nonsignificant result may indeed require less narrative attention than a central or informative finding. But if the outcome was prespecified, central to the research question, or necessary to understand the overall pattern, statistical significance should not determine whether it disappears.
The more appropriate question is whether results that do not support the hypothesis are relevant to evaluating the study's claims.
Often they are.
Selective Outcome Reporting Can Distort an Otherwise Accurate Sentence
Suppose an intervention significantly improves one of eight measured outcomes. Writing “the intervention significantly improved outcome X” may accurately describe that particular comparison.
But if readers are not told that seven other relevant outcomes showed no clear improvement, the sentence can acquire an evidential weight it would not have in the complete context.
Selective reporting therefore does not require fabricating a number or making a literally false statement. A report can mislead through selection .
This is one reason research integrity cannot be reduced to asking whether every sentence in a manuscript is technically true.
Outcome Switching Is One Possible Form of Selective Reporting
Suppose a trial prespecifies anxiety as its primary outcome and wellbeing as secondary. Anxiety shows little difference between groups, while wellbeing produces a favorable result. The publication promotes wellbeing to primary and gives little attention to anxiety.
The primary-outcome designation has changed, which raises an outcome-switching issue. The suppression or de-emphasis of the unfavorable planned outcome also raises a selective-reporting issue.
By contrast, a study could selectively report outcomes without changing any labels at all. It might simply omit three prespecified secondary outcomes because their findings are inconvenient.
Incomplete Reporting Can Be Selective Too
An outcome does not need to vanish completely to become difficult to evaluate.
A manuscript might state that “there was no significant difference in adverse events” without providing group-specific event counts or an effect estimate. Another outcome might receive a full estimate, confidence interval, table, and discussion because it supports the intervention.
Chan and colleagues explicitly examined incomplete reporting, including whether published information was sufficient for inclusion in a meta-analysis. Their findings illustrate why the completeness of reporting matters alongside whether an outcome is mentioned at all.
Not Every Omission Is Selective Outcome Reporting
Studies can generate enormous quantities of data. A journal article is not necessarily expected to contain every exploratory variable, diagnostic check, descriptive statistic, or incidental measurement.
The relevant concern is whether the selection is outcome-dependent and whether omission materially changes the reader's understanding of the research.
An outcome omitted because it is outside the scope of a particular report is not equivalent to an outcome omitted because it failed to reach p <.05. Large projects may also legitimately publish different outcomes in separate papers, provided that this structure is transparent and does not conceal contradictory evidence.
Watch Out
Page limits are not a good reason to make unfavorable outcomes invisible. When complete reporting would overwhelm the main article, supplementary materials, repositories, protocols, registrations, and linked reports can preserve access to relevant results.
Prospective Records Help Reveal What Is Missing
Selective outcome reporting is difficult to detect when the final paper is the only record available. Readers cannot know that an outcome disappeared if they never knew it existed.
Protocols, registrations, and statistical analysis plans provide an external record against which publications can be compared. SPIRIT 2025 emphasizes protocols as transparent records of investigators' original intentions, while CONSORT 2025 requires randomized-trial reports to identify prespecified primary and secondary outcomes and important changes made after commencement.
These records are therefore useful not merely for administrative compliance. They make selective reporting more detectable.
04 · A Practical Example
How One Significant Outcome Can Make an Entire Study Look Positive
Hypothetical Example
A Program With Six Prespecified Outcomes
A researcher evaluates an educational intervention using six prespecified outcomes: examination performance, retention, attendance, academic confidence, engagement, and course satisfaction.
Five outcomes
Examination performance, retention, attendance, confidence, and engagement show little clear evidence of improvement.
One outcome
Course satisfaction produces a statistically significant difference favoring the intervention.
The manuscript
The results section reports satisfaction in detail, briefly mentions examination performance, and omits the other four outcomes.
The abstract
The paper concludes that the intervention produced positive student outcomes without explaining that the favorable result occurred for only one of six prespecified measures.
The satisfaction finding can be real. Selective outcome reporting does not require inventing it.
The distortion arises because readers are denied the broader pattern needed to judge what the study found. “One favorable result among six planned outcomes” and “the intervention improved student outcomes” are not evidentially equivalent stories.
A transparent report could give satisfaction appropriate attention while still reporting the other prespecified outcomes and interpreting the complete pattern.
06 · What This Means for You
How to Decide What Results Belong in the Research Report
You still need editorial judgment. Transparent reporting does not mean giving every output identical prominence or filling the manuscript with results unrelated to its research question.
The safer principle is to separate importance from favorability .
A simple decision framework
If an outcome was prespecified as primary
Report it regardless of whether its result supports the hypothesis or crosses a significance threshold.
If a prespecified secondary outcome is relevant to the reported study
Do not omit it merely because the result is null, negative, or inconvenient.
If an outcome cannot fit in detail in the main article
Use appropriate supplementary or linked reporting while making its existence and relevance clear.
If an exploratory outcome produces an especially interesting result
Report it with appropriate prominence, but preserve its exploratory status and the context of the other relevant outcomes.
If you are tempted to omit an outcome because it weakens the story
Treat that temptation as a warning that the result may be necessary for an accurate account of the evidence.
For randomized trials, CONSORT 2025 requires reporting of prespecified primary and secondary outcomes and provides a framework for documenting protocol changes. Other study types should follow the relevant reporting guideline, protocol commitments, journal requirements, and disciplinary standards.
The larger principle is portable: readers should not need access to your private dataset to discover that the complete results tell a materially different story from the published ones.
07 · A Quick Checklist
Check Your Manuscript for Selective Outcome Reporting
Before submitting your results, check:
Have all relevant prespecified primary outcomes been reported regardless of their statistical significance?
Have relevant prespecified secondary outcomes been reported or made appropriately accessible?
Can readers identify outcomes that were added, removed, redefined, promoted, or demoted after the original plan?
Are outcome time points and measurement definitions consistent with the protocol, or are changes explained?
Are unfavorable results reported with enough information for readers to evaluate them?
Would you have made the same reporting decisions if the statistical significance of the outcomes were reversed?
Does the abstract reflect the overall pattern of important outcomes rather than only the most favorable one?
Could a reader comparing your protocol or registration with the paper understand and explain every consequential discrepancy?
11 · Cite this Guide
How to Cite This Guide
This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.
Recommended (Field Guide)
APA
MLA
Chicago
Copy Citation
