03 · What You Need to Know
A Citation Can Be Accurate Yet Still Participate in a Misleading Argument
Citation Accuracy and Citation Completeness Are Different
There are at least two questions to ask about a citation.
First: Does the cited source actually support the statement attached to it?
Second: Is the set of sources presented an adequate representation of the relevant evidence for the claim being made?
A researcher can succeed at the first and fail at the second.
For example, all four cited studies might indeed report positive findings. But if the paragraph says those studies establish that “research consistently shows” an effect while five directly relevant studies report null findings, the problem lies in the selection and synthesis rather than in the accuracy of the individual citations.
ICMJE's current recommendations require authors to cite references accurately and to ensure that cited references support the associated statement. They also advise using direct references to original research whenever possible and caution that review articles do not always represent original work accurately.
A Literature Review Is a Selection, but the Selection Needs a Rule
No serious literature review can reproduce an entire field. There may be thousands of relevant papers, with different populations, methods, outcomes, theoretical assumptions, and levels of quality.
Selection is therefore unavoidable.
The methodological question is what determines inclusion.
Principled selection
Studies are selected using relevance, predefined criteria, methodological quality, conceptual scope, date restrictions when justified, or another explicit rationale.
Directional selection
Studies are selected or emphasized because they support the argument the author wants to make, while contradictory evidence is disproportionately omitted or minimized.
A narrative review can use scholarly judgment rather than a formal systematic-review protocol. That does not make directional selection acceptable. It simply means the author needs to be especially clear about scope and search boundaries.
Cherry-Picking Can Make Disagreement Look Like Consensus
Suppose studies are split on whether an intervention improves student retention.
A literature review citing only studies that found improvements can honestly say, “The cited studies reported improvements.”
But if it concludes, “Research consistently shows that the intervention improves retention,” the sentence makes a much broader claim than the selected evidence warrants.
The omission matters because readers generally interpret literature reviews as representations of the state of knowledge, not as curated collections of only the papers that agree with the author.
The difference between those two claims is methodological, not merely rhetorical.
Contradictory Evidence Does Not Have to Be Treated as Equally Strong
A common reaction is to swing too far in the other direction: “If there are contradictory studies, we must present both sides as equally credible.”
That is not required.
Evidence can disagree because studies differ in quality, sample, measurement, design, statistical power, context, or implementation. A strong review may conclude that the best evidence leans in one direction.
What matters is that the conflicting evidence is not ignored simply because it is inconvenient.
You can write, for example, that several high-quality studies report an effect while weaker or contextually different studies do not. That is much more informative than pretending the disagreement does not exist.
Citation Bias Can Reinforce Positive Results Over Time
Selective citation is not merely a stylistic concern. There is empirical evidence that published studies with positive results can receive more citations than studies with nonsignificant results.
A systematic review and meta-analysis by Duyx and colleagues included 52 studies examining citation bias. Across the evidence they synthesized, articles with statistically significant results were cited about 1.6 times as often as articles with nonsignificant results, while articles explicitly concluding that their hypothesis was supported were cited about 2.7 times as often. The magnitude varied across disciplines, with the strongest evidence of citation bias observed in biomedical research.
Those findings do not mean that every frequently cited positive paper was selected improperly. Citation counts are influenced by many factors. They do show, however, that the literature can develop an overrepresentation of positive findings through citation practices as well as publication practices.
Selective Citation Can Affect Future Research
Researchers do not begin every study from scratch. Literature reviews influence which questions appear unresolved, which interventions seem promising, which theories seem supported, and which assumptions appear established.
If contradictory evidence is systematically omitted, later researchers may receive an inflated impression of how strong the existing evidence is.
Empirical work examining reports of cardiovascular trials provides a striking example. Robinson and colleagues found that reports cited a median of only 25% of prior relevant trials, and supportive trials were significantly more likely to be cited than nonsupportive trials. They concluded that substantial undercitation of existing evidence could contribute to unnecessary and unscientific research.
Again, this evidence comes from specific fields and samples. It should not be treated as a universal citation rate for every discipline. The mechanism, however, is broadly understandable: omitted evidence cannot influence the narrative built from the literature.
Citation Cherry-Picking Can Happen in a Single Sentence
Consider:
“Previous research has established that social media use reduces academic performance.”
The author attaches four citations.
Those four studies do report negative associations. But suppose the literature contains substantial evidence of weak, null, mixed, or context-dependent findings that the paragraph never acknowledges.
The citations may all be legitimate. The word established may still overstate what the literature supports.
This is why citation decisions should be evaluated alongside the claim being made, not only article by article.
Systematic Reviews Have a Particular Advantage
Systematic reviews use explicit methods to identify, select, and synthesize evidence. PRISMA 2020 provides a reporting framework designed to improve the transparency of systematic reviews, including reporting how studies were identified and selected.
That transparency makes selective citation easier to scrutinize because readers can see the search and selection process rather than relying entirely on the author's narrative description.
A narrative literature review does not need to imitate a systematic review in every respect. It should nevertheless make its scope and selection logic sufficiently clear that readers understand what kind of evidence it represents.
EQUATOR Network maintains reporting resources for systematic reviews and other study types and emphasizes that reporting guidance exists to ensure that important information is available for critical appraisal and subsequent use in evidence synthesis.
Primary Studies Matter When Making Specific Claims
A review can efficiently summarize a field, but a specific empirical claim may be better supported by the original research that actually generated the result.
ICMJE explicitly recommends direct references to original research sources whenever possible and notes that review articles do not always accurately reflect original work.
This matters for citation cherry-picking because repeating the same favorable conclusion through a chain of reviews can make it appear independently supported when several references ultimately derive from the same small body of evidence.
Self-Citation Can Also Distort the Literature Map
Citation selection can become problematic when authors cite their own work disproportionately because it advances the argument.
Self-citation is sometimes entirely appropriate. Researchers should cite their own prior studies when those studies are directly relevant.
The issue arises when self-interest becomes a selection criterion or when authors exclude more relevant external evidence. ICMJE specifically states that references should not be used by authors, editors, or peer reviewers to promote self-interests.
Watch Out
A literature review can become misleading through omission without containing a single false citation. Before writing “the evidence shows,” check whether you mean “the studies I selected show.” Those are not the same claim.
04 · A Practical Example
How Selective Citations Can Turn Mixed Evidence Into Apparent Consensus
Hypothetical Example
Does a Learning Platform Improve Student Engagement?
A researcher finds 12 empirical studies examining the relationship between use of a learning platform and student engagement. Seven report positive associations, three report weak or null associations, and two report results that vary by context.
Selective review
The researcher cites the seven positive studies and states that previous literature consistently demonstrates a positive relationship.
Reader's impression
A reader unfamiliar with the field is likely to infer that disagreement in the literature is minimal.
Complete synthesis
A more balanced review explains that the evidence generally leans positive but includes null and context-dependent findings.
Interpretation
The researcher discusses why results may differ across populations, measures, or implementation settings rather than treating disagreement as an inconvenience to be removed.
The balanced version does not require counting every study as equally persuasive.
The researcher might reasonably argue that the seven positive studies are methodologically stronger or more directly relevant. That is a substantive judgment that readers can evaluate.
The important difference is that contradictory evidence has been considered rather than silently excluded because it complicates the preferred conclusion.
06 · What This Means for You
How to Build a Literature Review That Represents the Evidence Rather Than Your Argument Alone
Start with the claim you want to make, then actively look for evidence that would challenge it.
That simple step changes the literature search from confirmation seeking into synthesis.
A simple decision framework
If a study directly addresses your claim
Consider it based on relevance and methodological quality, not whether its findings support your position.
If a strong study contradicts your preferred conclusion
Include and evaluate it rather than quietly excluding it because it makes the argument more difficult.
If evidence is mixed
Describe the direction and degree of disagreement and investigate plausible reasons for the variation.
If one study is frequently cited by other papers
Check the original study before repeating a claim about its findings, particularly for consequential arguments.
If you are conducting a systematic or scoping review
Follow the applicable reporting guideline and make the search and selection process sufficiently transparent for readers to evaluate it.
ICMJE advises authors to cite direct original sources where possible, verify reference accuracy, and avoid using references to promote self-interest.
For a systematic review, PRISMA can provide a more structured framework for reporting how evidence was identified and selected. For other designs, use the relevant reporting guideline rather than assuming that one review standard fits everything. EQUATOR's reporting-guideline library is a practical starting point for identifying the appropriate framework.
A useful writing habit is to replace broad phrases such as “research shows” with more precise statements when the evidence is heterogeneous. “Several studies report…” is sometimes more accurate than “the literature establishes…” The difference may feel tiny. Methodologically, it can be enormous.
