01 · The Question
A Review Can Be Recent and Still Miss the Literature
You find a systematic review published only months ago. At first glance, that seems to settle the question of whether another review is needed.
Then you inspect the methods. Perhaps only one database was searched. Important terminology appears absent from the strategy. The authors imposed restrictions that seem difficult to justify. Known eligible studies are missing. Or the search is reported so incompletely that you cannot determine what was actually done.
Does the review's recency still make it sufficient?
Not necessarily. A review can only synthesize evidence that its methods identify and include. If the search systematically misses relevant evidence, a new publication date does not repair the problem.
03 · What You Need to Know
How to Decide Whether a Review's Search Is Good Enough
The search is part of the evidence-generating method
A systematic review does not begin with the studies that happen to be easy to find. Its conclusions depend on identifying an appropriate body of eligible evidence.
Cochrane guidance states that searches should be as extensive as possible to reduce the risk of reporting bias and identify as much relevant evidence as possible. It also emphasizes that database selection should be guided by the review topic and that inappropriate or inadequate strategies may fail to identify records that are actually present in bibliographic databases.
Search quality is therefore not a clerical detail. It can affect which evidence enters the review and, consequently, what the synthesis appears to show.
Start by asking whether the right sources were searched
There is no universal number of databases that makes a search adequate. The appropriate sources depend on the question, discipline, eligible study designs, geographic scope, and type of evidence being sought.
For Cochrane intervention reviews, for example, current guidance specifies searching CENTRAL and MEDLINE together with Embase when available, as well as relevant specialized registers and trial registers. That requirement is specific to the Cochrane context and should not be mechanically transferred to every review in every discipline.
An education review, qualitative evidence synthesis, engineering review, or interdisciplinary review may require a different combination of bibliographic databases and supplementary sources.
Watch Out
“They searched only three databases” is not, by itself, evidence of a poor search. Three well-chosen sources may be more appropriate than six poorly chosen ones. Evaluate coverage in relation to the review question and field.
Inspect whether the search represents the main concepts adequately
A database can contain a relevant article that the search strategy fails to retrieve.
Search strategies need terminology capable of capturing how relevant studies are described and indexed. Cochrane recommends combining appropriate free-text terms with controlled vocabulary where available. Within important concepts, multiple synonymous or related terms may be needed to achieve adequate sensitivity.
At the same time, adding every element of the research question to the search can make retrieval worse. Outcomes and comparators, for example, may be absent from titles, abstracts, or indexing. Cochrane cautions that it may be unnecessary or undesirable to search every component of a clinical question.
This is why a good search cannot be judged by its length. A spectacular wall of Boolean operators can still be conceptually wrong. Search strings, like reference lists, do not become rigorous through sheer acreage.
Look for restrictions that could exclude relevant evidence
Language, publication status, date, document type, geography, and other restrictions can alter what a search retrieves.
Some restrictions are defensible because they follow directly from the review's eligibility criteria. Others may be pragmatic. The important question is whether a restriction could systematically remove relevant evidence in a way that affects the review's conclusions.
Cochrane cautions that restricting searches by publication status or language can introduce risks of publication or language bias. The exact implications vary by review question and evidence base, so restrictions should be evaluated rather than automatically condemned.
Test the search against known relevant studies
One practical diagnostic is to identify several studies that unquestionably meet, or appear likely to meet, the review's eligibility criteria and ask whether the reported search could retrieve them.
If important known studies are absent from the review, investigate why. They may have been excluded legitimately. They may fall outside the search dates. They may not satisfy an eligibility criterion that was easy to overlook. Or the search may genuinely have missed them.
A missing study is therefore a signal for investigation, not immediate proof of search failure.
Separate poor searching from poor reporting
Sometimes you cannot tell whether the search itself was inadequate because the authors did not report it sufficiently.
Poor search conduct
The methods used to identify evidence were insufficient or inappropriate for the review question.
Poor search reporting
The publication provides too little information to evaluate or reproduce what was actually searched.
These are related but distinct problems. A well-designed search can be reported badly, while a beautifully documented search can still be methodologically weak.
Cochrane reporting guidance calls for authors to report all sources consulted, dates of coverage, the date of the final search, limits applied, and exact database strategies so that readers can evaluate and reproduce the search. PRISMA-S provides additional reporting guidance specifically for literature searches in systematic reviews.
A poor search matters most when it could change the evidence base
Finding an imperfection in a search strategy does not automatically justify another review.
Suppose a review omits a potentially relevant database. If a supplementary search of that database identifies no additional eligible evidence, the omission may have little practical effect on the review's answer. Conversely, if it reveals several influential studies with different findings, the weakness becomes consequential.
The useful question is therefore not “Can I criticize this search?” but “Could this search limitation have changed which evidence was synthesized?”
Search weaknesses can propagate across several reviews
If several existing reviews leave the same important search blind spot, the number of published reviews may exaggerate how thoroughly the question has been examined.
Review authors often use earlier reviews to identify terminology and studies, which can be useful. But repeated reliance on similar sources and search assumptions can also reproduce omissions.
This is one reason to inspect the methods of the strongest relevant reviews rather than assuming that multiple publications independently confirm comprehensive evidence coverage.
A recent review may already be temporally old
Recency of publication and recency of searching are also different.
A review appearing this year may have completed its search substantially earlier. Cochrane reporting guidance recommends reporting the final search date so that readers can judge currency.
If the search itself is old, you may be dealing with two questions at once: whether the original search was adequate and whether important evidence has appeared since it ended. By contrast, an older review can sometimes remain adequate when little consequential evidence has appeared.
Methodological criticism should be specific enough to test
If you believe an existing review's search is inadequate, specify the problem.
For example: a major discipline-specific database was omitted; important synonyms were absent; an unjustified date restriction excluded eligible research; only English-language publications were eligible without a substantive rationale; or the strategy used too many restrictive concepts and appears to have sacrificed sensitivity.
Then test the consequence where feasible. Search the missing source. Add the missing terminology. Examine studies excluded by the restriction. Check whether known eligible studies would have been retrieved.
This turns “the search looked weak” into a defensible assessment of whether the evidence base may be incomplete.
The real question is whether the review still answers your question
A search flaw matters because of what it does to the answer.
If consequential evidence was missed, then the existing review may not adequately answer the research question despite being recent. A new review could then be justified if a more appropriate search and synthesis would materially improve what is known.
If the search weakness is minor and unlikely to affect the evidence or interpretation, conducting another near-duplicate review may offer little value.
07 · A Quick Checklist
Evaluate Whether a Recent Review Searched Well Enough
Before treating a recent review as sufficient, check:
Identify every database, register, website, and other information source reported as searched.
Judge whether those sources appropriately cover the discipline, question, and eligible evidence.
Inspect the full search strategies where available rather than relying on the authors' statement that the search was comprehensive.
Check whether important synonyms, spelling variants, controlled vocabulary, and alternative terminology were represented appropriately.
Examine language, date, publication-status, document-type, and other restrictions for possible consequential exclusions.
Check the final search date rather than assuming it is close to the publication date.
Investigate known relevant studies that do not appear among the included studies.
Distinguish inadequate searching from inadequate reporting of the search.
Test whether suspected search weaknesses actually reveal additional eligible evidence where feasible.
Ask whether the missing evidence could materially change the synthesis or its interpretation.
