Can Generative AI Reliably Distinguish Primary Studies From Review Articles?

Can AI Tell Whether a Paper Reports New Research or Reviews Existing Research?

When screening literature, one of the first questions is often deceptively simple: is this a primary study or a review article?

Generative AI can usually recognize an obvious randomized trial and an explicitly labeled systematic review. The difficulty appears in less tidy cases: secondary analyses of existing datasets, scoping reviews, methodological papers, pooled analyses, protocols, umbrella reviews, studies embedded within reviews, and articles whose titles reveal very little about what the authors actually did.

If publication type matters to your eligibility criteria or evidence synthesis, “it looks like a primary study” is not quite enough.

AI Can Classify Many Papers Correctly, but the Classification Should Be Verified

Publication Type Depends on the Research Activity, Not Just the Label

What Is a Primary Research Study?

A primary research article reports an empirical investigation conducted or analyzed by the authors to answer a research question. Depending on the discipline, this can include experiments, randomized trials, observational studies, surveys, qualitative studies, diagnostic studies, cohort studies, case-control studies, modeling studies, and many other designs.

Importantly, “primary” does not necessarily mean that the authors collected entirely new data themselves. A paper conducting a new empirical analysis of an existing dataset may still be primary research. The distinction depends on the nature of the investigation, not merely who first generated the underlying data.

What Is a Review Article?

A review article primarily synthesizes, evaluates, maps, or discusses existing literature rather than reporting a new study of participants, specimens, organizations, or other primary units of observation.

Reviews themselves are diverse. Systematic reviews use explicit methods to identify and synthesize eligible evidence. Scoping reviews map a body of literature. Narrative reviews may provide broader interpretive synthesis. Umbrella reviews synthesize existing reviews. Meta-analyses statistically combine results when appropriate, often within a systematic review.

The word review alone therefore tells you relatively little about methodological rigor or purpose.

Titles and Abstracts Often Provide Strong Classification Signals

Publication types frequently reveal themselves through familiar terminology. Phrases such as “randomized controlled trial,” “cross-sectional study,” “cohort study,” or “qualitative interviews” strongly suggest primary research. “Systematic review,” “scoping review,” and “umbrella review” strongly suggest evidence synthesis.

LLMs are particularly capable of recognizing these linguistic patterns. Evidence from AI-assisted review workflows suggests that modern models can identify study characteristics and perform literature-screening tasks with useful accuracy under some conditions. In a UK Health Security Agency evaluation spanning experimental, observational, qualitative, and modeling studies, extraction of study design achieved at least 90% acceptability, although performance varied substantially for other extracted fields.

That is encouraging, but classification becomes harder when the decisive information is absent from the title and abstract or when terminology is used inconsistently.

“Study” Does Not Mean Primary Study

Review articles are themselves studies. A systematic review may have a protocol, eligibility criteria, data extraction, statistical analysis, results, and conclusions. It may even contain a meta-analysis with hundreds of numerical observations extracted from primary studies.

An AI system that classifies publications using superficial signals such as the presence of a Methods section, statistical analysis, or the word study can therefore make mistakes.

The better question is: What are the units being investigated? If researchers recruited participants and analyzed their outcomes, that points toward primary research. If they searched databases, selected existing studies, and synthesized those studies, that points toward a review.

A Meta-Analysis Is Not Automatically Primary Research

Meta-analysis is a statistical method, not a synonym for a particular publication type. Most conventional meta-analyses synthesize quantitative results from existing studies and are therefore part of secondary research.

However, some projects conduct individual participant data meta-analysis, pooled analyses, or new analyses combining datasets from multiple studies. Their classification may depend on the methodological context and the eligibility rules of the research project in which you are using the classification.

This is precisely where a binary prompt such as “primary or review?” can be too crude. Sometimes the appropriate answer needs a more specific design label.

Secondary Data Analysis Can Still Be Primary Research

Suppose researchers analyze an existing national survey dataset to test a new hypothesis. They did not collect the original survey responses, yet the article reports a new empirical analysis of participant-level data.

Calling that paper a review merely because the data already existed would be incorrect. Secondary data analysis and secondary research synthesis are different activities.

This distinction matters in literature screening because eligibility criteria may include primary empirical analyses regardless of whether the authors personally collected the original data.

Some Articles Do Not Fit Either Category Neatly

The primary-study-versus-review distinction is useful, but scholarly publications include many other forms. Protocols describe planned studies without reporting their final results. Editorials and commentaries advance arguments. Guidelines may combine evidence reviews with expert recommendations. Consensus statements can involve structured expert processes. Methodological papers may introduce or evaluate analytical methods. Case reports describe individual cases, while letters can contain anything from commentary to original data.

Forcing every publication into only “primary study” or “review” can therefore create classification errors before the AI even begins.

If your review eligibility depends on publication type, include an “other” or “uncertain” category when appropriate.

Abstract-Only Classification Has an Information Ceiling

When an AI receives only a title and abstract, it can classify only from the information available there. Some abstracts state the design explicitly. Others do not.

Research on AI-assisted title and abstract screening has found useful sensitivity and specificity for LLM-based approaches, but errors still occurred because of missing domain knowledge or misinterpretation of eligibility criteria.

This is not merely an AI problem. Human reviewers also move records from title-and-abstract screening to full-text assessment precisely because abstracts do not always contain enough information for a defensible eligibility decision.

Full-Text Access Can Improve Classification but Does Not Make It Infallible

When the full article is available, the model can inspect methods, participant recruitment, database searches, eligibility criteria, data sources, and analytical procedures. Those details can make publication type much easier to determine.

Still, access and classification accuracy are different things. A model can have the full paper and misunderstand what it reads. Conversely, some classifications can be made confidently from a well-written abstract alone.

Before relying on a classification, therefore, consider whether the AI actually accessed enough of the paper to support the description.

Classification Performance Depends on the Exact Task

“Can AI identify study design?” sounds like one capability, but evaluations vary substantially. Some tasks ask whether a record meets predefined systematic-review eligibility criteria. Others extract a design label from full text. Still others distinguish randomized trials from observational studies or classify dozens of publication types.

Results from one setting should therefore not be converted into a universal accuracy rate for AI study classification. A model that performs well distinguishing randomized trials from reviews may perform differently when asked to separate a secondary database analysis from a pooled analysis.

Why the Word “Analysis” Does Not Settle the Classification

A useful AI classifier should therefore explain the evidence for its classification rather than return only a label. “Primary study because the authors performed a new analysis of participant-level survey data” is considerably easier to audit than simply “primary.”

Common Mistakes When AI Classifies Research Articles

Ask AI to Justify the Classification From the Paper's Methods

Generative AI can be useful as a first-pass classifier, particularly when screening large numbers of records. The most defensible workflow, however, does not ask merely for “primary” or “review.” Ask the model to identify the design and cite the textual evidence supporting that classification.

This makes errors easier to detect and reduces reliance on superficial terminology.

Recent evaluations suggest that LLMs can support screening and data extraction in evidence reviews, but performance varies across tasks and study designs, and human quality assurance remains important. The practical goal is not to prove that AI can classify papers perfectly. It is to use automation where it reduces workload without making eligibility decisions opaque.

Before Accepting an AI Classification of a Research Paper

Frequently Asked Questions About AI and Publication Type

AI Can Recognize Study Types, but Classification Should Follow the Methods

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.