03 · What You Need to Know
What Makes the Use of AI Detection Fair or Unfair in an Investigation?
Research-Integrity Investigations Must Begin With an Allegation, Not a Percentage
An investigation should begin by identifying the specific conduct alleged to have violated an applicable rule.
For example, an allegation might concern prohibited AI-generated drafting, failure to disclose substantial AI assistance, fabrication of references, or misrepresentation of how research data were analyzed.
These allegations are not interchangeable. Each requires different evidence and may fall under different institutional procedures.
A detector score cannot establish which allegation applies. It may classify textual characteristics, but it does not independently identify the tools used, the researcher's intentions, or the requirements that governed the work.
Investigators should therefore establish the relevant policy, its effective date, and the conduct it regulated before considering whether a detection report has any evidentiary value.
Formal research misconduct also has jurisdiction-specific definitions. For example, US Public Health Service regulations address fabrication, falsification, and plagiarism. An alleged breach of an institutional AI-disclosure rule does not automatically meet that regulatory definition.
What Is the Difference Between Screening, Inquiry, and Investigation?
Institutions may use different terminology and procedures, but an important distinction exists between preliminary assessment and formal fact-finding.
| Stage |
Primary Purpose |
Appropriate Treatment of an AI-Detection Result |
| Initial assessment |
Determine whether an allegation is sufficiently specific and falls within institutional jurisdiction. |
Examine whether the report provides credible information rather than assuming the score establishes wrongdoing. |
| Preliminary inquiry |
Determine whether sufficient grounds exist for further examination under applicable procedures. |
Consider limitations, relevant records, and alternative explanations without presuming guilt. |
| Formal investigation |
Evaluate evidence concerning the alleged conduct under the governing standards. |
Assign evidentiary weight only where the detector's relevance and reliability can be justified. |
| Determination and review |
Reach and, where applicable, review a reasoned finding. |
Do not substitute an automated classification for proof of a policy violation. |
This table describes a general distinction rather than a universal procedural sequence. Some institutions use different stages or terminology.
The US Office of Research Integrity provides a more specific regulatory framework for covered Public Health Service research misconduct proceedings. Institutions should follow the rules applicable to their own jurisdiction and case.
Why Is the Scientific Reliability of the Detector Essential?
Evidence should be capable of supporting the proposition for which it is introduced.
If an investigator uses a detector report to suggest that a manuscript was generated by AI, the relevant question is whether the tool can distinguish AI-generated and human-written material with sufficient reliability under comparable conditions.
Published evaluations have identified substantial limitations in AI-text detection. Weber-Wulff and colleagues reported accuracy and reliability problems across 14 evaluated tools. Liang and colleagues demonstrated substantial false-positive disparities in a sample of non-native English writing.
These studies do not establish the current performance of every detector. They demonstrate why investigators should demand tool-specific validation rather than relying on commercial accuracy claims.
Relevant validation should address the type of academic writing examined, language background, text length, model version, and relevant false-positive and false-negative rates.
Without such information, the detector's evidentiary value may be too uncertain to justify its use.
Why Is a Detection Score Not the Same as Proof?
Most general-purpose detectors analyze the final text rather than observing the actual writing process.
A numerical result may represent a classification score, a tool-specific estimate of AI-associated text, or another proprietary measure. It should not automatically be interpreted as the probability that the researcher used AI.
Even a high score can be wrong. Conversely, a low score cannot establish that AI assistance was absent.
This distinction is examined in why AI-detection scores should not be treated as proof.
For investigative purposes, the report's most defensible interpretation may be limited to the fact that a particular system classified a particular version of the manuscript in a particular way.
Whether that classification provides meaningful evidence of prohibited conduct is a separate question.
How Can Algorithmic Bias Affect Procedural Fairness?
Fair procedures require attention to whether evidence is systematically less reliable for certain populations.
Liang and colleagues found that seven evaluated GPT detectors disproportionately misclassified essays written by non-native English speakers. Their study reported an average false-positive rate of 61.3% across those detectors for 91 TOEFL essays.
That figure cannot be generalized to every research manuscript or contemporary detection system. Nevertheless, the study provides evidence that detector performance may vary substantially across linguistic populations.
If a tool produces more false positives for one group, using it without appropriate safeguards may expose members of that group to disproportionate scrutiny.
Equal application of the same software is not necessarily equivalent to equitable treatment.
Institutions should examine language-related AI-detection bias and other relevant subgroup performance differences before allowing detector results to influence consequential decisions.
Why Does the Burden of Proof Matter?
A research-integrity investigation should follow the burden and standard of proof established by the applicable rules.
These requirements vary. In US Public Health Service research misconduct proceedings, for example, the governing regulations establish a preponderance-of-the-evidence standard and address the evidence necessary for a misconduct finding.
Other jurisdictions and institutional procedures may use different standards.
A detector score should not silently reverse the burden by requiring the respondent to prove that AI was never used.
Researchers may be able to provide drafts, research notes, and revision histories. Such materials can support an explanation, but their absence does not automatically establish misconduct.
The institution remains responsible for reaching a conclusion that satisfies the applicable evidentiary requirements.
What Does Procedural Fairness Require?
Procedural fairness generally involves giving the respondent sufficient information about the allegation and a meaningful opportunity to address the evidence.
The exact requirements depend on the governing procedure, but several safeguards are particularly relevant to AI-detection disputes.
- The allegation should identify the specific conduct and policy provision at issue.
- The respondent should receive sufficient information about the detector report to understand and challenge its interpretation.
- Investigators should consider the detector's documented limitations and relevant error rates.
- Relevant explanations and supporting records should be evaluated without presuming that the score is correct.
- Decisions should distinguish established facts from uncertain inferences.
- Applicable confidentiality, conflict-of-interest, and review requirements should be respected.
Human review is necessary for consequential institutional decisions, but it is not automatically sufficient. A reviewer who accepts a score uncritically may reproduce the detector's errors rather than correct them.
Can Investigators Be Influenced by the Detector's Initial Score?
Yes. Automated recommendations can influence human judgment, a phenomenon often discussed in the literature on automation bias.
A numerical score may appear precise and objective even when its statistical meaning is poorly understood.
For example, an investigator who first sees a high AI-detection score may interpret ordinary features of academic writing as suspicious. The same passages might not have attracted concern without the initial classification.
This does not mean every investigator will be biased. It means that procedures should reduce the risk of treating automated outputs as presumptively correct.
One possible safeguard is to evaluate independently identified concerns and documentary evidence before assigning weight to the detector report.
Investigators should also document alternative explanations rather than considering only evidence consistent with the initial allegation.
What Evidence Is More Directly Relevant Than a Detection Score?
Evidence connected to the actual writing process may be more informative than a statistical classification.
Relevant materials can include contemporaneous manuscript drafts, document histories, research notes, correspondence, AI-use records, and explanations from the researcher.
Such evidence should be evaluated for authenticity, completeness, and relevance. A document history may support a chronology of revisions without proving that every passage was composed without AI assistance.
Similarly, an AI conversation record may establish that a tool was used but not necessarily that its output was incorporated into the manuscript.
The evidentiary question is whether the records support or contradict the specific alleged conduct.
A detailed examination of stronger evidence for investigating undisclosed AI use can help investigators distinguish direct documentation from circumstantial indicators.
When Should an AI-Detection Result Be Excluded or Given Little Weight?
Exclusion or minimal evidentiary weight may be appropriate when the detector's performance cannot be established for the relevant writing, its scoring method is insufficiently documented, or the report cannot be meaningfully challenged.
Other concerns include unsupported claims of accuracy, inappropriate processing of confidential material, and substantial risks of bias.
A report may also be irrelevant when the alleged misconduct concerns something the detector does not measure, such as fabricated data or undisclosed AI-assisted analysis.
Investigators should not assume that admitting more technical information necessarily improves the quality of the investigation.
Sometimes the more rigorous decision is to recognize that a particular tool cannot answer the question being asked.
Why Must Confidentiality Be Protected During Investigations?
Research-integrity investigations may involve unpublished manuscripts, confidential research data, correspondence, and personal information.
Submitting these materials to external AI-detection services can create additional risks if the provider retains content or uses it for purposes beyond the investigation.
Institutions should verify applicable data-protection obligations, contractual restrictions, and research confidentiality requirements before using external tools.
The International Committee of Medical Journal Editors also emphasizes protecting manuscript confidentiality when AI systems are used in editorial processes.
Although journal peer review and institutional investigations are distinct activities, both illustrate why unpublished research material requires careful handling.
