01 · The Question
Are You Selecting Evidence, or Simply Selecting What You Can Download?
You find 2,000 potentially relevant records. Some have a PDF waiting one click away. Others sit behind publisher paywalls. A few appear only as dissertations, conference papers, reports, or records in databases your institution does not subscribe to. Some older papers seem to have vanished into the academic equivalent of a locked filing cabinet.
The temptation is understandable: work with the studies you can access easily.
For informal background reading, practical access will inevitably influence what you read first. For a review intended to identify evidence systematically, however, availability should not quietly become an eligibility criterion unless the methodology explicitly justifies such a restriction. Otherwise, the evidence base may reflect your subscriptions, search habits, and patience as much as the research question.
03 · What You Need to Know
Access and Eligibility Answer Different Questions
A Study Does Not Become Irrelevant Because the PDF Is Inconvenient
Eligibility criteria answer whether a study belongs in the evidence base. Access determines whether you can obtain enough information to evaluate or use it. Those are different methodological issues.
Suppose two studies investigate exactly the population, intervention, outcome, and study design specified in your protocol. One is open access and the other is behind a paywall. The open-access status of the first study does not make it more relevant to the research question.
Ineligible study
The study does not satisfy the predefined criteria required to answer the research question.
Potentially eligible but inaccessible report
The available record suggests possible eligibility, but you cannot yet obtain enough information to make or complete the eligibility assessment.
Keeping those categories separate prevents a practical obstacle from being disguised as a scientific exclusion.
Start With Sources Appropriate to the Question, Not Merely the Ones You Already Use
A systematic search should not be defined by whichever database happens to be open in your browser. Database coverage differs by discipline, publication type, geography, language, and indexing practices.
Cochrane guidance recommends selecting bibliographic databases according to the review topic and using searches that are sufficiently extensive to reduce the risk of reporting bias and identify relevant evidence. It also recommends considering sources beyond conventional bibliographic databases, including trial registers and relevant grey literature such as reports, dissertations, theses, and conference abstracts.
The exact sources needed depend on the research question and review method. A review in education will not necessarily use the same database combination as a clinical intervention review. The broader principle is portable: choose information sources because they cover the evidence you need, not because they are the easiest ones for you to search.
Convenience Can Enter the Review Earlier Than Full-Text Retrieval
Access bias is not limited to paywalls. Researchers can privilege convenient evidence at several points.
| Convenience decision |
What may happen |
Better question |
| Searching only one familiar database |
Studies indexed elsewhere may never become candidates |
Which information sources cover this evidence base? |
| Using only easily downloadable papers |
Access status influences study selection |
Does the study satisfy the eligibility criteria? |
| Ignoring dissertations or reports |
Published journal literature may dominate the evidence considered |
Which publication types are relevant to the methodology? |
| Stopping when retrieval becomes difficult |
Hard-to-obtain studies disappear without transparent accounting |
What reasonable retrieval steps should be attempted and documented? |
The methodological concern is cumulative. If the search finds only convenient records and full-text retrieval then retains only convenient reports, the final evidence base can become progressively less representative of the studies relevant to the question.
Open Access Is Valuable, but It Is Not an Eligibility Criterion by Default
Open-access publishing makes research substantially easier to obtain and reuse within the terms of the applicable license. That practical benefit should not be confused with methodological eligibility.
If a systematic review includes only open-access studies because those are the only studies the reviewers can retrieve, the resulting evidence base is being restricted by publication access. That may exclude otherwise eligible research.
If your project intentionally studies open-access literature itself, of course, access status may genuinely define the research question. That is a different situation from using open access as an unacknowledged shortcut.
Look for Legitimate Alternative Routes to the Full Text
A publisher page is not necessarily the only lawful route to a paper. Depending on the study and your institutional resources, a report may also be available through a university library, institutional repository, subject repository, author manuscript, interlibrary loan or document-delivery service, research archive, or another legitimate source.
Reports can also exist in multiple forms. Cochrane emphasizes that systematic reviews include studies rather than individual reports and notes that a single study can generate multiple reports. Searching and linking those reports can provide additional information about the same underlying study.
This distinction matters when a conference abstract appears inaccessible or incomplete. A full journal article, trial-register record, thesis, report, or another publication may describe the same study.
Search Beyond Published Journal Articles When the Review Requires It
Restricting a review to conventional journal articles because they are easiest to locate can interact with publication and reporting processes. Some studies appear in dissertations, reports, conference proceedings, registries, or other sources rather than standard journals.
Cochrane recommends that searches for intervention reviews be as extensive as possible to reduce the risk of publication bias and specifically identifies relevant grey-literature sources as potentially important. It also recommends checking previous reviews and reference lists of included studies and relevant systematic reviews.
This does not mean every review must search every conceivable source. The sources should fit the question and methodology. “I did not know where else to look,” however, is different from a reasoned decision that another source is unlikely to contribute relevant evidence.
Reference Lists and Citation Searching Can Find Studies That Database Searching Misses
Database searching is powerful but imperfect. Relevant records may use unexpected terminology, be indexed inconsistently, or sit outside the databases you searched.
Cochrane recommends checking reference lists of included studies and relevant systematic reviews and describes citation searching of key articles as an additional method for identifying studies. It also notes that citation searches that yield many additional eligible records may signal weaknesses in the original search strategy.
This is a useful distinction for a large literature. Following citations is a method for discovering candidates. It does not mean those candidates should be included because they were cited.
Do Not Let Retrieval Difficulty Change the Inclusion Criteria Midway
Imagine that your protocol includes dissertations. During screening, you discover that retrieving them is considerably more laborious than downloading journal articles. Quietly dropping dissertations at that point changes the evidence source according to retrieval burden.
Sometimes practical constraints genuinely force methodological changes. If so, report the change, explain the reason, and consider its implications. Transparency does not eliminate the limitation, but it prevents the limitation from masquerading as the original design.
Watch Out
“Full text unavailable” should not be used as a convenient catch-all exclusion reason when the real issue is that no serious retrieval attempt was made. Define a reasonable retrieval process for your project and document what remains unavailable after that process.
Not Every Literature Review Requires Systematic-Review-Level Retrieval
The required effort depends on what you claim to have done.
A researcher preparing a brief conceptual introduction may reasonably work with a selected body of literature and need not pretend to have identified every eligible study. A systematic review, by contrast, makes stronger claims about systematic identification and selection and therefore requires methods appropriate to those claims.
The problem arises when a convenience sample of accessible papers is presented as though it represents a comprehensive or systematic evidence base.
If the real difficulty is that your search retrieves an enormous number of records rather than that reports are inaccessible, address the size and precision of the literature search directly. Restricting the evidence to whatever is easiest to download does not solve that methodological problem.
04 · A Practical Example
When the Most Convenient Papers Are Not the Whole Evidence Base
Hypothetical Example
A review of an educational intervention
A researcher screens records concerning a digital intervention for nursing education. Twenty-four reports appear potentially eligible from their titles and abstracts. Sixteen can be downloaded immediately through the researcher's institutional subscriptions or open-access links. The remaining eight are harder to obtain.
1. Keep eligibility separate from access
The researcher does not automatically discard the eight difficult reports. They remain potentially eligible until sufficient information is obtained to decide otherwise.
2. Search for alternative versions
The researcher checks library holdings, repositories, related publications, and study identifiers and discovers that two conference records correspond to later full journal articles.
3. Use library retrieval options
Several reports are obtained through document delivery or interlibrary services available to the researcher.
4. Document unresolved cases
After reasonable retrieval efforts, one potentially relevant report remains unavailable. The researcher records what is known about it and handles it according to the review methodology rather than silently deleting it from the process.
5. Report the limitation
The final review explains the unresolved record and considers whether its absence could affect interpretation.
The point is not that every inaccessible report can always be recovered. It cannot. The methodological improvement comes from preventing convenience from deciding eligibility invisibly.
07 · A Quick Checklist
Before Excluding a Study Because You Cannot Easily Access It
Before treating a difficult-to-access record as unavailable, check:
Does the record appear potentially eligible according to the predefined research question and criteria?
Have you checked the legitimate access routes available through your institution or library?
Have you searched for an author manuscript, institutional or subject repository copy, or another legitimate version where appropriate?
Could another report, thesis, conference paper, registry record, or publication describe the same underlying study?
Does your methodology require searching relevant grey-literature or non-journal sources?
Have you checked reference lists and, where appropriate, citation links for additional candidate studies?
Have you recorded unresolved potentially eligible reports separately from studies excluded for substantive reasons?
If access constraints changed your methodology, have you documented and justified the change?
