Can Generative AI Be Used in a Systematic Review?

Can You Use Generative AI Without Compromising a Systematic Review?

Systematic reviews can involve screening thousands of records, examining full-text articles, extracting study characteristics, assessing methodological quality, and synthesizing evidence. These activities take considerable time, so it is understandable that researchers are interested in whether tools such as ChatGPT can reduce the workload.

But systematic reviews are not simply literature summaries. Their credibility depends on predefined methods, comprehensive searching, reproducible decisions, and transparent reporting. If generative AI makes an incorrect exclusion decision or invents an extracted finding, that error can affect the evidence ultimately presented to readers.

The question, therefore, is not merely whether AI can perform review tasks. It is whether researchers can use it while preserving the methodological safeguards that make a review systematic.

Yes, but AI Assistance Is Not the Same as an AI-Conducted Review

Where Generative AI Fits Into Systematic Review Methodology

What Does Using Generative AI in a Systematic Review Actually Mean?

Generative AI refers to systems capable of producing text or other content in response to instructions and supplied information. Large language models (LLMs), including those underlying many conversational AI applications, can interpret research-related instructions, classify text, summarize documents, and generate structured responses.

In a systematic review, these capabilities may support particular operations. A researcher might ask a model to identify a study's sample size from an uploaded article, compare an abstract against predefined eligibility criteria, or organize extracted information into a table.

These are different from asking AI to independently identify all relevant research, decide which studies deserve inclusion, evaluate their credibility, and write the final synthesis. The latter involves a chain of methodological judgments in which an error at one stage can influence every subsequent stage.

A generated literature summary does not become a systematic review simply because its title says so. The distinction lies in the methods used to identify, select, appraise, and synthesize the evidence.

What Does the Research Say About AI Reliability?

The evidence is promising for selected applications but considerably less reassuring when generative AI operates without human checks.

A systematic review published in Research Synthesis Methods, examining 19 comparative studies identified through searches conducted in January 2025, found substantial variation in generative AI performance across evidence-synthesis tasks. Reported incorrect exclusion decisions during screening ranged from 1% to 83%, while incorrect data extractions ranged from 4% to 31%. The included evaluations also had important methodological limitations, with many assessed as having high or unclear risk of bias in relevant domains.

These ranges should not be interpreted as the expected error rate of every AI model or workflow. They describe heterogeneous studies using different tasks, datasets, and evaluation procedures. Nevertheless, the findings provide a strong reason not to assume that a plausible AI response is necessarily accurate.

Similarly, Khraisha and colleagues (2024) evaluated GPT-4 on screening and extraction tasks involving different literature types and languages. Performance varied by task and dataset characteristics, and apparently favorable accuracy scores sometimes became less convincing after accounting for chance agreement and class imbalance.

The practical implication is that reliability must be established for the particular review workflow rather than inferred from a model's general reputation.

Which Stages of a Systematic Review Might Benefit From AI?

Generative AI may support activities throughout the review, although the strength of evidence and the consequences of mistakes differ substantially.

The important question is which activities can be delegated to AI without allowing unverified outputs to determine the evidence base.

Why Are Some AI Errors More Consequential Than Others?

Not all errors have the same methodological consequences. A poorly worded summary can usually be corrected during editing. An eligible study excluded during screening may never reach data extraction, appraisal, or synthesis.

Screening therefore requires particular attention to false exclusions, sometimes called false negatives. Researchers need to understand whether AI can apply inclusion and exclusion criteria reliably, especially when eligibility depends on subtle differences in study design, participants, intervention characteristics, or outcomes.

Extraction errors create different risks. An AI system may correctly identify a numerical value but associate it with the wrong experimental group, time point, or outcome. Such mistakes can remain difficult to notice when the resulting table appears internally consistent.

Risk-of-bias and certainty-of-evidence assessments involve further interpretive judgments. AI may help identify relevant passages, but researchers must distinguish locating information from making a defensible appraisal.

Does Generative AI Replace Established Systematic Review Software?

Not necessarily. Conventional systematic review software and machine-learning systems already support record management, deduplication, screening prioritization, and other structured operations. Some use statistical classifiers rather than generative language models.

This distinction matters because evidence supporting one automation method does not automatically establish the reliability of another. A validated screening classifier, a conversational LLM, and a specialized evidence-synthesis platform may behave differently even when presented with the same records.

Researchers should evaluate the actual tool and workflow, including its version, configuration, input format, and decision rules, rather than treating all AI-assisted methods as interchangeable.

What Must Remain Under Human Responsibility?

Researchers remain responsible for defining the review question, establishing eligibility criteria, approving the search strategy, interpreting the evidence, and ensuring that the reported methods accurately describe what occurred.

Human oversight should also address disagreements, ambiguous cases, methodological appraisal, and the consequences of AI errors. The extent of checking may differ between low-risk administrative activities and decisions capable of changing the review's conclusions.

Using AI does not automatically justify removing an independent human reviewer. Any proposed departure from established review procedures requires methodological justification, appropriate validation, and consideration of the standards applicable to the review.

Does PRISMA Permit or Prohibit Generative AI?

PRISMA 2020 is a reporting guideline, not a certification system for research methods or an approval mechanism for AI tools. It explains what authors should report about how a systematic review was conducted and what it found.

Its requirements concerning information sources, search strategies, selection processes, data collection, risk-of-bias assessment, synthesis methods, and reporting of limitations remain relevant when AI is involved.

Consequently, researchers should explain how AI was used, what decisions it influenced, how outputs were checked, and whether any limitations may have affected the results. The specific expectations may also depend on the journal, publisher, discipline, or review organization.

Transparent reporting of AI involvement allows readers to evaluate the review process rather than guessing how much of it was automated.

Using AI in a Review of Educational Technology Interventions

What Would Successful AI Assistance Look Like?

Imagine that the team discovers AI can reliably organize candidate extraction fields but performs inconsistently when abstracts describe interventions ambiguously. The researchers could retain AI assistance for organizing extracted information while continuing independent human screening.

That would be a reasonable task-specific decision. The researchers would not need to accept or reject generative AI as a whole. Instead, they would determine where it provides measurable assistance without introducing unacceptable methodological risk.

Even a seemingly minor classification error can matter. For example, an abstract describing automated feedback may refer to instructor-generated comments delivered through software rather than feedback generated by AI. The distinction could determine eligibility, and a language model might overlook it unless the criteria and source information are sufficiently explicit.

The central lesson is that the usefulness of AI depends on how well it performs the specific task, not simply on how quickly it produces an answer.

Common Misunderstandings About AI-Assisted Systematic Reviews

How to Decide Whether AI Belongs in Your Review Workflow

Start by identifying the exact task you want AI to perform. A request such as "use ChatGPT for my systematic review" is too broad to evaluate meaningfully. A more defensible proposal would specify that AI will generate provisional extraction entries from full-text reports, with every entry checked against its source.

Next, consider the consequences of mistakes. Administrative support, such as organizing verified information into a consistent format, generally presents different risks from automatically excluding records or making final judgments about evidence certainty.

What Should You Document Before Starting?

Where AI use is planned, document its intended role in the review protocol or associated methodological records. Specify the inputs, tasks, decision rules, validation procedures, and human responsibilities. Record any subsequent changes rather than presenting a revised workflow as though it had been planned from the beginning.

It is also useful to preserve prompts or instruction templates, model identification and version information where available, dates of use, output files, and records of corrections. Because generative systems and hosted services can change, documenting these details can improve interpretability even when exact reproduction remains difficult.

Ultimately, the goal is not to maximize the amount of AI used. It is to determine whether a particular application improves the research process without undermining the credibility of the resulting evidence.

Before Using Generative AI in a Systematic Review

Frequently Asked Questions About Generative AI in Systematic Reviews

Generative AI Can Support a Systematic Review, but It Cannot Establish Its Credibility

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.