03 · What You Need to Know
How to Identify and Assess Primary Study Overlap Across Reviews
What Does Primary Study Overlap Mean?
Primary study overlap occurs when two or more reviews include the same underlying research investigation. For example, three systematic reviews evaluating AI-supported writing interventions may all include the same classroom experiment.
The reviews are separate scholarly publications, but their empirical evidence is not completely independent.
Overlap is common when systematic reviews address similar questions, especially when the number of eligible primary studies is relatively limited. It does not automatically indicate an error in the individual reviews.
The methodological concern arises when an overview treats repeated evidence as independent confirmation or combines outcome data without accounting for their dependence.
Overlapping reviews
Separate reviews include one or more of the same underlying primary studies.
Duplicate counting
The same underlying evidence is counted or weighted more than once in a synthesis when that repetition is not methodologically justified.
Overlap creates the possibility of double-counting, but identifying overlap and demonstrating inappropriate double-counting are not identical tasks.
Why Does Overlap Matter for Evidence Synthesis?
Suppose four reviews report that an intervention improves student learning. If all four rely heavily on the same two experiments, the agreement among reviews should not be interpreted as four independent sets of experimental evidence.
In a narrative overview, repeated findings may receive disproportionate emphasis. In quantitative synthesis, reusing the same primary study's outcome data without accounting for dependence can give those data excessive statistical weight and produce misleadingly precise estimates.
The Cochrane Handbook explicitly identifies these concerns in its guidance for overviews of reviews. It recommends examining overlap and selecting an appropriate strategy for managing it according to the overview's purpose and methods.
Overlap is also central to distinguishing independent scientific corroboration from repeated evidence. Several agreeing reviews may provide useful interpretations without representing equally numerous independent investigations.
Where Should AI Look for the Included Primary Studies?
Researchers should begin with the lists of included studies rather than the complete bibliographies of the reviews.
A systematic review's reference list may contain background literature, methodological guidance, excluded studies, and publications cited for contextual reasons. These are not necessarily primary studies included in its synthesis.
Relevant information may appear in the review's characteristics-of-included-studies table, evidence tables, supplementary files, forest plots, or study selection documentation.
AI can help extract and standardize these records, but researchers must verify whether each identified publication was actually included in the relevant review and whether it contributed evidence to the outcome or comparison being assessed.
Watch Out
Do not calculate primary study overlap from complete reference lists without distinguishing included studies from background citations. Doing so may substantially misrepresent the evidence shared across reviews.
Why Exact Citation Matching Is Not Enough
Two reviews may cite the same study using slightly different bibliographic details. Author names, publication years, journal abbreviations, and titles may be formatted inconsistently.
AI can suggest matches using titles, DOIs, authors, and other bibliographic identifiers. These suggestions can be checked against authoritative records and the original publications.
A more difficult situation occurs when different articles report results from the same investigation. One review may cite the original trial publication, while another cites a follow-up report or secondary analysis.
These publications may have different DOIs and titles while sharing participants or data. Conversely, papers with similar authors, institutions, and topics may describe independent studies.
Researchers should therefore distinguish publication identity from underlying study identity.
What Information Helps Establish Whether Studies Are the Same?
Potentially useful identifiers include trial registration numbers, project identifiers, study names, recruitment locations, data collection periods, participant characteristics, sample descriptions, and explicit cross-references between publications.
| Evidence of identity |
How it helps |
Important limitation |
| Same DOI |
Strongly indicates the same publication record. |
Different reports of one study usually have different DOIs. |
| Same trial or project identifier |
Can link reports to a common investigation. |
One project may contain distinct study components. |
| Same participants and recruitment period |
May indicate shared underlying data. |
Details may be incompletely reported. |
| Same authors and institution |
Provides a clue for further checking. |
Research teams frequently conduct separate studies. |
| Explicit statement of secondary analysis |
Can establish a relationship with an earlier dataset. |
The extent of data and outcome overlap still matters. |
AI should flag uncertain matches rather than merging records automatically. A mistaken merge may conceal genuinely independent studies, while a missed match may lead to double-counting.
What Is a Citation Matrix?
A citation matrix, sometimes called a review-by-study matrix, records which primary studies appear in which systematic reviews.
Typically, rows represent unique primary studies and columns represent reviews. Each cell indicates whether a particular study was included in the corresponding review.
This structure allows researchers to inspect overlap directly, identify studies appearing in several reviews, and calculate measures of overlap where appropriate.
It is related to a general literature review matrix, but its primary purpose is to map the inclusion of underlying studies across reviews rather than summarize each study's methods and findings.
Cochrane recommends a citation matrix as a useful minimum approach for displaying primary study overlap in overviews of reviews.
What Is the Corrected Covered Area?
The corrected covered area (CCA) is a measure used to quantify the extent of overlap among primary studies included in multiple reviews.
It uses information from a citation matrix and adjusts the observed repeated occurrences relative to the number of unique primary studies and reviews.
The measure was described by Pieper and colleagues in methodological research on overlapping systematic reviews. Subsequent guidance by Hennessy and Johnson discusses its calculation and interpretation.
The formula requires at least two reviews and a meaningful set of primary study records. Researchers should ensure that the matrix has been constructed correctly before calculating the measure.
CCA describes the extent of overlap, not the methodological quality or certainty of the evidence. It also does not show which particular study contributes the greatest statistical weight to a pooled analysis.
Where reviews address different populations, interventions, or outcomes, an overall CCA may obscure more consequential overlap within specific comparisons. Subgroup or pairwise overlap assessment may therefore be useful.
Can AI Calculate CCA Reliably?
AI can help construct the citation matrix and explain the calculation, but numerical accuracy should be established through a spreadsheet, statistical software, or another reproducible calculation.
Errors in identifying unique studies can affect both the numerator and denominator. A model that incorrectly merges two independent studies or fails to recognize multiple reports of one study may produce an inaccurate CCA even when it applies the formula correctly.
A defensible workflow verifies study identities, constructs the matrix, checks the counts, and then calculates overlap using a reproducible method.
The researcher should retain the matrix and calculation records so that another person can inspect how the result was obtained.
Does a High CCA Mean the Reviews Are Unreliable?
No. High overlap indicates that reviews share substantial underlying evidence. It does not independently establish poor review quality, biased results, or incorrect conclusions.
Two high-quality systematic reviews may include many of the same primary studies because they investigate nearly identical questions. Their methods or interpretations may nevertheless differ in meaningful ways.
The concern is how overlapping evidence is handled when drawing conclusions across the reviews.
Likewise, low overlap does not guarantee high-quality evidence. Reviews may include different studies that share methodological weaknesses or examine non-equivalent outcomes.
Overlap assessment should therefore be considered alongside review quality, primary study risk of bias, and the relevance of the evidence to the question being investigated.
How Should Researchers Manage Overlapping Reviews?
There is no single strategy appropriate to every overview. The choice depends on the research question, available data, review quality, and intended synthesis.
Cochrane guidance describes approaches that include retaining relevant reviews while avoiding double-counting primary study outcome data, selecting reviews according to predefined criteria when overlap is substantial, or retaining overlapping reviews with transparent justification and acknowledgment of the limitations.
For an overview intended to describe the existing review literature, retaining several overlapping reviews may be appropriate. For an analysis that combines quantitative results, dependence requires particular methodological attention.
AI may help organize these options, but it should not independently decide which reviews to exclude or how statistical dependence should be handled.
Why Overlap Assessment Must Be Specific to the Claim
Two reviews may include the same primary study but extract different outcomes or investigate different comparisons.
For example, one review may examine writing performance, while another examines student satisfaction. Both may include the same classroom experiment, but the shared study does not mean their outcome-specific evidence is identical.
Researchers should distinguish overlap in study inclusion from overlap in the particular outcome data supporting a conclusion.
This distinction matters when synthesizing findings across multiple sources. The relevant concern is whether the same evidence receives inappropriate repeated influence in the specific synthesis being conducted.
04 · A Practical Example
Detecting Shared Primary Studies Across Three Systematic Reviews
Hypothetical Example
Three Reviews, Five Unique Primary Studies
A researcher examines three hypothetical systematic reviews investigating AI-assisted feedback and academic writing. After verifying the included-study lists, the researcher constructs the following citation matrix.
| Primary study |
Review A |
Review B |
Review C |
| Study 1 |
Yes |
Yes |
Yes |
| Study 2 |
Yes |
Yes |
Yes |
| Study 3 |
Yes |
Yes |
No |
| Study 4 |
No |
Yes |
No |
| Study 5 |
No |
No |
Yes |
All reviews and primary studies in this example are hypothetical.
Step 1: Identify unique studies
The researcher verifies that the matrix contains five distinct primary investigations rather than assuming that every bibliographic record represents a unique study.
Step 2: Count study occurrences
Review A includes three studies, Review B includes four, and Review C includes three. There are 10 total study occurrences.
Step 3: Calculate overlap
With N = 10, r = 5, and c = 3, the corrected covered area is 50%.
Step 4: Interpret the matrix
Studies 1 and 2 appear in all three reviews, while Studies 4 and 5 appear in only one review each. The researcher examines whether the shared studies contribute to the same outcome-specific conclusions.
An Inaccurate AI Interpretation
"The three systematic reviews collectively provide evidence from 10 independent studies, confirming that AI-assisted feedback has been extensively validated."
This interpretation confuses study occurrences with unique investigations. The matrix contains only five distinct primary studies. Moreover, the matrix alone provides no information about their methodological quality or whether their results support the stated effectiveness claim.
A More Defensible Interpretation
"The three hypothetical systematic reviews collectively contain 10 primary study occurrences representing five unique investigations. The corrected covered area is 50%, indicating substantial overlap within this citation matrix. Two primary studies appear in all three reviews. Consequently, agreement among the reviews should not be interpreted as independent confirmation from 10 studies. Further assessment is required to determine the methodological quality of the underlying evidence and whether overlapping outcome data have been counted more than once."
The revised interpretation describes what the overlap calculation establishes without treating it as a measure of evidence quality or scientific consensus.
