01 · The Question
What If a Study Included in Your Systematic Review Has Been Retracted?
A systematic review may include a study that was legitimate when first published but was subsequently retracted. The retraction might have occurred because of unreliable findings, serious methodological errors, ethical concerns, or problems with the publication process.
The difficulty is that retracted articles do not necessarily disappear from databases, reference managers, or downloaded PDF collections. Researchers may encounter an apparently ordinary article without realizing that its publication status has changed.
Generative AI can help researchers examine bibliographic information and potentially identify publication-status warnings. But can it reliably detect retractions, and what should researchers do when an included study is found to have been retracted?
03 · What You Need to Know
How Retraction Detection Fits Into a Systematic Review
What Does It Mean When a Research Article Is Retracted?
A retraction is a formal mechanism for correcting the scholarly record when a publication contains problems serious enough to undermine confidence in its findings or otherwise warrant withdrawal from the reliable literature.
According to the Committee on Publication Ethics (COPE), retractions may arise from major errors, unreliable data, ethical problems, publication misconduct, or other circumstances that compromise the scholarly record. The purpose is to correct the literature rather than simply punish authors.
Importantly, retraction does not always imply deliberate misconduct. Honest errors can also produce findings that are no longer reliable.
COPE's Retraction Guidelines, updated in 2025, explain the responsibilities of editors and publishers and the importance of accessible, clearly linked retraction notices.
How Is a Retraction Different From a Correction or Expression of Concern?
Not every publication-status notice means that a study has been retracted. Researchers need to distinguish the type of notice before deciding how it affects the review.
| Publication status |
What it generally indicates |
Review implication |
| Retraction |
The publication has been formally retracted because of serious concerns or other grounds identified in the notice. |
Investigate the reason and determine whether its findings can legitimately contribute to synthesis. |
| Correction |
An error or omission has been amended without necessarily invalidating the publication. |
Use the corrected information and assess whether extracted data require revision. |
| Expression of concern |
Significant concerns exist, but their investigation or resolution may be incomplete. |
Examine the notice and consider the implications for confidence in the evidence. |
| Withdrawal |
A publication or manuscript has been withdrawn under circumstances defined by the publisher. |
Verify the notice and the publisher's meaning before deciding how to handle the record. |
A correction does not automatically invalidate an entire study. Conversely, an expression of concern should not be ignored simply because a formal retraction has not occurred.
Where Can Researchers Check Whether a Study Has Been Retracted?
Researchers should prioritize authoritative publication-status information rather than relying on an AI-generated answer.
Several sources can provide useful evidence:
-
Publisher and journal websites: The original article page and linked editorial notices can establish the publisher's recorded status and reasons for retraction.
-
Crossmark: Crossref's service allows readers to examine registered updates, including corrections and retractions, when participating publishers provide the relevant metadata.
-
Crossref: Its metadata services include publication updates and retraction information supplied by members and Retraction Watch.
-
Retraction Watch Database: A curated resource containing retraction records collected from publisher notices and other sources.
-
Bibliographic databases: Services such as PubMed may identify retracted publications and link them to associated notices, depending on coverage and indexing.
No single source should be assumed to provide perfect or immediate coverage. Publication-status metadata may be incomplete, delayed, or inconsistent across services.
What Changed When Crossref Integrated Retraction Watch Data?
Crossref acquired the Retraction Watch Database in 2023 and subsequently integrated Retraction Watch retraction information into its production REST API in January 2025.
This integration made curated retraction metadata more accessible to research software and other systems using Crossref records. The information can include retraction relationships originating from publishers and Retraction Watch.
Crossref also provides the underlying Retraction Watch dataset for download, with updates made on working days. These developments make structured retraction checking more feasible than relying exclusively on manual searches.
For researchers using automated workflows, Crossref's Retraction Watch documentation explains how retraction information is represented in its metadata services.
However, a language model does not automatically have access to current Crossref records. A reliable workflow must actually retrieve and inspect the relevant metadata rather than ask AI to recall an article's status from its training data.
How Could Generative AI Assist With Retraction Detection?
Generative AI may support the workflow by preparing identifiers for lookup, organizing retrieved status information, comparing retraction notices with bibliographic records, and summarizing the implications of verified notices for further examination.
For example, researchers might supply a list of included studies containing DOIs. A structured workflow could retrieve available Crossref metadata and identify records associated with retraction notices. AI could then help organize the flagged cases into a verification table.
In this example, the authoritative status information comes from Crossref or the publisher. AI assists with interpretation and organization rather than independently establishing the retraction.
This distinction is essential because a model may otherwise produce false claims about a study's status or overlook recent retractions.
Why Are DOIs Useful but Not Sufficient?
A Digital Object Identifier provides a persistent identifier that can help distinguish publications with similar titles and support metadata retrieval.
However, not every relevant publication has a DOI, and metadata records can contain errors or incomplete relationships. A retraction notice may also have its own DOI, separate from that of the original article.
Researchers should therefore compare identifiers with titles, authors, publication details, and linked notices when necessary.
An apparent mismatch should trigger investigation rather than an automatic conclusion that the study is unaffected.
Can AI Detect a Retraction From the Article's PDF Alone?
Not reliably. A PDF downloaded before a retraction may contain no indication that the publication's status later changed.
Even a PDF downloaded after a retraction may not contain a visible notice, depending on the publisher's procedures and the version accessed.
Crossmark was developed partly to make updates visible across publication formats, including PDFs where the service is implemented. Nevertheless, researchers should check current publication-status records rather than assuming that the document itself contains complete information.
A model analyzing only the original PDF cannot establish that no subsequent retraction has occurred.
Should Every Retracted Study Be Automatically Excluded?
Not without examining the purpose of the review, the retraction notice, and the consequences for the evidence.
For reviews evaluating intervention effects, a retraction caused by unreliable outcome data may provide strong grounds for excluding the affected results from the primary synthesis. Retractions arising from other publication problems may require a more context-specific decision.
Some reviews may deliberately investigate retracted research, publication integrity, or the consequences of retractions. In those cases, retracted articles may be the very objects of study.
Researchers should avoid a mechanical rule that treats all retraction notices as identical. They should also avoid continuing to use unreliable findings merely because a study satisfied the original eligibility criteria.
When a retracted report belongs to a study with several publications, the team should investigate whether other reports provide independently usable evidence or whether the concerns affect the underlying study as a whole.
What If a Study Is Retracted After the Review Is Published?
A retraction can occur months or years after a systematic review has been completed. This means publication-status checking is not necessarily a one-time activity.
If a study contributing to an existing review is subsequently retracted, researchers should evaluate whether the retraction affects the review's findings or conclusions. Depending on the consequences, a correction, update, or other editorial action may be appropriate.
Research published in PLOS ONE has documented inconsistencies in how retractions are reflected across scholarly information systems and emphasized the importance of addressing retracted evidence in secondary research.
Review teams should therefore preserve sufficient study identifiers and extraction records to identify affected findings if publication statuses change.
When Should Retraction Checks Be Conducted?
A practical approach is to check publication status when studies are selected for inclusion and again before finalizing the manuscript or evidence synthesis.
For long-running reviews, living systematic reviews, or projects experiencing substantial delays, additional checks may be justified.
These are workflow recommendations rather than universal timing requirements. Researchers should follow applicable organizational and journal guidance and document the date and sources of the checks.
Watch Out
Never rely on a chatbot's statement that a paper is "not retracted" unless the workflow has actually checked current, authoritative publication-status information. The absence of a warning in an AI response is not evidence that the article remains valid.
04 · A Practical Example
Finding a Retracted Study Before Completing a Meta-Analysis
Hypothetical Example
A Systematic Review of AI-Based Learning Interventions
A research team has included 35 studies examining AI-supported learning interventions in higher education. Before finalizing the synthesis, the researchers check the publication status of the included reports.
A structured metadata lookup flags one publication as potentially retracted. Generative AI is used to organize the returned information and identify the relevant record for manual verification.
Step 1: Match the publication
Researchers compare the DOI, title, authors, and publication details to ensure that the flagged record corresponds to the included report.
Step 2: Verify the official notice
The team examines the publisher's retraction notice and confirms that the article was formally retracted because of unreliable outcome data.
Step 3: Determine the affected evidence
Researchers identify which extracted outcomes and effect estimates came from the retracted publication and whether related reports are also affected.
Step 4: Reassess the synthesis
Under the review's documented methodological decision, the unreliable findings are removed from the primary effect synthesis and the analysis is recalculated.
Step 5: Document the decision
The team records the retraction notice, verification date, affected results, analytical changes, and implications for the review's conclusions.
What If the Pooled Result Changes?
Suppose the original meta-analysis suggested a beneficial intervention effect, but the recalculated estimate becomes smaller and its confidence interval includes no effect after the retracted study is removed.
The researchers should report the change and explain its implications. The example does not establish that every retraction will materially alter a synthesis, but it illustrates why publication status can be consequential.
It also demonstrates the proper division of responsibility: the metadata service flags the publication, the publisher's notice establishes its status, and researchers determine how the verified information affects the review. AI may assist with organizing these steps without becoming the authority on publication integrity.
06 · What This Means for You
Build Publication-Status Verification Into Your Review Workflow
Retraction checking should be treated as a documented research-integrity procedure rather than an optional question asked casually of a chatbot.
Preserve reliable bibliographic identifiers, use authoritative status sources, investigate flagged records, and document how verified notices influence eligibility or synthesis.
A practical retraction-checking framework
If no retraction is found in the checked sources
Record the sources and date checked without claiming that the article is guaranteed free from publication-integrity concerns.
If a formal retraction is confirmed
Read the notice, identify the affected findings, and determine the implications for eligibility and synthesis.
If a correction is identified
Examine the corrected information and revise extracted data or interpretations where necessary.
If an expression of concern is identified
Investigate the stated concerns and consider their implications without treating the notice as equivalent to a completed retraction.
If a study is retracted after review publication
Assess whether the affected evidence changes the conclusions and whether an update or editorial action is warranted.
How Should AI Retraction Detection Be Evaluated?
Where researchers use an AI-enabled workflow to flag retractions, its performance should be tested against independently verified publication-status records.
Evaluation should include correctly identified retractions, missed retractions, false-positive flags, and confusion between retractions, corrections, and expressions of concern.
Because retracted publications may represent a small proportion of a review's records, overall accuracy can be misleading. A system that labels every article as unretracted could appear accurate while missing the very cases it is intended to identify.
The principles of evaluating classification errors are relevant, although the reference categories and consequences differ from ordinary eligibility screening.
What Should Be Reported?
Record the publication-status sources consulted, dates of checking, identifiers used, confirmed notices, and decisions concerning affected studies.
If AI materially contributed to identifying or organizing retraction information, describe the tool, its role, verification procedures, and limitations through the review's AI-use reporting.
Researchers should also ensure that the review's study-selection documentation accurately reflects any exclusions resulting from confirmed retractions, without inventing or retrospectively altering the underlying record counts.
