03 · What You Need to Know
What Research Reveals About Language Background and AI-Detection Bias
What Does English as an Additional Language Mean in Academic Writing?
English as an additional language describes situations in which a person uses English alongside one or more other languages. It is often preferred in educational contexts because it does not define a person's linguistic identity solely by whether English was their first language.
Related research frequently uses terms such as non-native English speaker, second-language writer, or L2 writer. These categories overlap but are not necessarily interchangeable.
For example, a researcher may have learned English early in life, completed university education in English, and published extensively in international journals while still identifying another language as their first language.
Language background is therefore not a reliable substitute for academic writing proficiency. Nor should researchers be assumed to share a particular writing style simply because they belong to the same broad linguistic category.
These distinctions matter when interpreting studies that compare writing from groups classified as native and non-native English speakers.
What Did Liang and Colleagues Find About AI-Detection Bias?
One influential investigation was published by Liang and colleagues in Patterns in 2023 under the title GPT detectors are biased against non-native English writers.
The researchers evaluated seven widely used GPT detectors using 91 TOEFL essays written by non-native English speakers and 88 essays written by US eighth-grade students.
The results revealed a substantial difference in classification performance between the two collections of human-written essays.
| Reported Finding |
Result |
Interpretation |
| Average false-positive rate for TOEFL essays |
61.3% |
Across the seven detectors, a substantial proportion of the evaluated non-native English writing was incorrectly classified as AI-generated. |
| TOEFL essays unanimously flagged by all seven detectors |
19.8% |
Some human-written essays were consistently misclassified across every evaluated system. |
| TOEFL essays flagged by at least one detector |
97.8% |
Almost every essay in the sample received an AI classification from at least one detector. |
| Classification of US eighth-grade essays |
Near-perfect accuracy |
The comparison sample experienced substantially fewer misclassifications under the tested conditions. |
These findings provide evidence of unequal error rates in the evaluated systems. They do not establish that 61.3% of all academic manuscripts written by EAL researchers will be falsely flagged, nor do they describe the current performance of every commercial detector.
The comparison also involved different essay collections, not two otherwise identical groups of professional researchers. Differences in writing context, genre, age, and educational background limit how directly the results can be generalized.
Nevertheless, the findings demonstrate that a detector can appear highly accurate for one population while performing substantially worse for another. The original study provides the detailed experimental results.
Why Might Additional-Language Writing Be Mistaken for AI-Generated Text?
One explanation concerns linguistic predictability.
Some AI-detection approaches use perplexity, a measure of how predictable a text is to a particular language model. Text that contains highly probable word sequences tends to have lower perplexity under that model.
Liang and colleagues found that the TOEFL essays unanimously misclassified by their evaluated detectors had significantly lower perplexity.
The researchers suggested that restricted variation in word choice may contribute to this pattern. Writers working in an additional language may sometimes prefer familiar vocabulary or established expressions, particularly when prioritizing grammatical accuracy.
However, linguistic variation is not determined solely by language background. It can also reflect genre, educational experience, disciplinary conventions, and individual writing preferences.
A detector that interprets predictable language as evidence of machine generation may therefore misclassify writing produced through entirely legitimate human processes.
The underlying statistical mechanisms used by AI-text detectors help explain why this overlap can occur.
What Happened When Researchers Changed the Vocabulary of the Essays?
Liang and colleagues conducted an additional experiment to investigate whether changes in vocabulary would affect classification.
They used ChatGPT to enhance the word choices in the TOEFL essays, aiming to produce language more closely resembling the lexical patterns of native English writing.
After this intervention, the average false-positive rate across the evaluated detectors decreased from 61.3% to 11.6%, a reduction of 49.7 percentage points.
The researchers also conducted a contrasting experiment in which they simplified vocabulary in the US student essays. This increased the likelihood that those texts would be misclassified as AI-generated.
These findings suggest that lexical characteristics influenced the detectors' decisions. Importantly, they do not establish that sophisticated vocabulary is a reliable indicator of human authorship or that simpler vocabulary necessarily causes false positives in every system.
The experiments also should not be interpreted as a recommendation that researchers use AI to rewrite their work to obtain lower detection scores. Their scientific contribution was to demonstrate the sensitivity of the evaluated classifiers to changes in linguistic expression.
Does the Evidence Extend Beyond Student Essays to Research Manuscripts?
Liang and colleagues also examined research papers from the 2023 International Conference on Learning Representations (ICLR).
They reported that papers whose first authors were from countries categorized as non-native English-speaking had lower text perplexity than papers associated with native English-speaking countries. The reported trend remained after accounting for review ratings.
This finding is relevant to academic publishing because it suggests that differences in linguistic predictability may also occur in scholarly manuscripts.
However, the analysis does not directly establish how frequently those research papers would be falsely flagged by particular detectors. It also uses country-based categories as an approximation of language background, which cannot establish an individual researcher's linguistic identity or proficiency.
The distinction matters: evidence of different perplexity distributions is not identical to evidence of different detector error rates.
Further validation using verified human-written research manuscripts across disciplines and linguistic populations is necessary before making broad claims about contemporary academic publishing.
Is AI-Detection Bias the Same as Deliberate Discrimination?
Not necessarily. Algorithmic bias can arise even when developers do not intentionally design a system to disadvantage a particular population.
For example, a detector trained on a dataset that insufficiently represents additional-language academic writing may learn patterns that generalize poorly to that population.
Unequal performance can also arise from the selection of linguistic features, classification thresholds, or evaluation datasets.
Intentional discrimination
Deliberate differential treatment based on a protected or otherwise relevant personal characteristic.
Algorithmic performance disparity
A measurable difference in system outcomes or error rates across populations, which may arise without discriminatory intent.
Both can have serious consequences, but they require different forms of evidence to establish their causes.
In AI detection, the immediate empirical question is whether comparable human-written texts receive systematically different error rates across relevant groups. Establishing the mechanisms behind those differences requires additional investigation.
Why Is Overall Detector Accuracy Not Enough to Establish Fairness?
A detector may perform well across an entire evaluation dataset while producing substantially higher false-positive rates for a smaller group of writers.
Suppose an institution evaluates a detector using 1,000 human-written manuscripts. If most samples come from one linguistic population, the overall false-positive rate may primarily reflect performance for that population.
Poorer performance for an underrepresented group could remain obscured by the aggregate result.
Responsible evaluation therefore requires examining subgroup error rates alongside overall performance.
Relevant questions include whether the test samples are comparable in genre and discipline, whether authorship is independently verified, and whether sample sizes are sufficient to support meaningful conclusions.
Small subgroup samples can produce unstable estimates. Differences should therefore be reported with appropriate uncertainty rather than interpreted as precise or universal characteristics of a linguistic population.
Why Could This Matter More in International Academic Publishing?
International journals receive submissions from researchers with diverse linguistic backgrounds. A manuscript's English-language style may reflect individual proficiency, disciplinary expectations, coauthor contributions, or professional editing.
If an AI detector disproportionately flags certain forms of legitimate English writing, its use could create unequal scrutiny during editorial assessment.
The consequences may include requests for explanations, additional documentation, delayed editorial decisions, or reputational concerns. These are potential consequences, not evidence that they occur uniformly across journals.
Such risks become especially serious when journals use AI detectors to screen research manuscripts without sufficiently validating their performance for the populations they serve.
Scholarly writing should be evaluated for the quality of its argument, methodological transparency, accuracy, and compliance with editorial policies. Linguistic differences alone should not be treated as evidence of undisclosed AI use.
Can Improved Detectors Eliminate Language-Related Bias?
Better training data, more representative evaluation samples, and improved classification methods may reduce disparities. However, no particular intervention should be assumed effective without empirical testing.
Expanding a training dataset, for example, does not automatically guarantee fair performance if the labels are unreliable or if important differences between writing populations remain unrepresented.
Similarly, adjusting a classification threshold to reduce false positives for one group may affect sensitivity or error rates elsewhere.
Fairness evaluation should therefore be ongoing, especially when detection models, writing practices, or generative AI systems change.
Independent validation remains more informative than general claims that a detector has become fairer or more accurate.
04 · A Practical Example
How Language-Related Detection Bias Could Affect a Research Manuscript
Hypothetical Example
Two Researchers, Two Human-Written Manuscripts
Two educational technology researchers independently prepare manuscripts examining student engagement in online learning.
Researcher A writes English as a first language. Researcher B uses English as an additional language and tends to prefer familiar academic expressions when describing research procedures.
Both manuscripts are entirely human-written. Neither researcher uses generative AI to produce the submitted text.
During screening, a hypothetical detector assigns a low AI-likelihood classification to Researcher A's manuscript and a high classification to Researcher B's manuscript.
The different outcomes could reflect the detector's sensitivity to linguistic characteristics, although the scores alone would not establish the cause of the difference.
Initial result. The detector flags Researcher B's manuscript. The classification is a statistical prediction, not an observation of the writing process.
Assessment of the concern. The editor considers whether any evidence beyond the detector output suggests prohibited or undisclosed AI assistance.
Researcher response. Researcher B provides relevant drafts, literature notes, and revision records showing how the manuscript developed.
Editorial decision. The editor evaluates the evidence and applicable policy without assuming that linguistic predictability establishes AI authorship.
The scenario illustrates how an incorrect classification could create unequal scrutiny even when both researchers followed the same writing requirements.
It does not establish that every EAL researcher will receive a higher AI score. Rather, it shows why subgroup performance matters when detection tools are deployed in multilingual academic environments.
