03 · What You Need to Know
How to Judge the Scientific Acceptability of AI Performance Disparities
There Is No Universal Acceptable Performance Gap
Researchers sometimes look for a numerical threshold that determines whether an AI model is sufficiently fair across populations.
For example, they may ask whether a five-percentage-point difference in accuracy is acceptable or whether a ten-percentage-point difference should automatically disqualify a model.
Such thresholds cannot be justified independently of the research task.
A small difference in sensitivity may be consequential when the model identifies participants requiring urgent intervention. A larger difference in a low-stakes exploratory classification may have more limited consequences.
Researchers should define acceptable performance in relation to the intended application rather than adopt an arbitrary universal cutoff.
Where disciplinary standards or regulatory requirements apply, those requirements should also inform the evaluation.
Scientific Acceptability and Ethical Acceptability Are Related but Distinct
A performance disparity may be scientifically tolerable for a particular analytical objective while raising ethical concerns about its consequences.
For example, a model may provide sufficiently accurate aggregate estimates for an exploratory study but perform poorly for a subgroup whose results are used to allocate institutional support.
The scientific concern involves whether the model supports valid conclusions.
The ethical concern involves whether the system creates unjustified disadvantages or unequal opportunities.
Researchers should examine both dimensions rather than assuming that satisfactory predictive performance establishes fairness.
The broader meaning of AI fairness in research requires explicit consideration of these competing objectives.
Researchers Must Specify Which Performance Metric Matters
A difference in overall accuracy does not have the same meaning as a difference in sensitivity, specificity, precision, or calibration.
For example, sensitivity measures the proportion of actual positive cases correctly identified.
If an academic support model has lower sensitivity for working students, it may overlook more students who genuinely need assistance.
By contrast, a difference in specificity concerns the proportion of actual negative cases correctly identified.
Researchers should select metrics according to the task and consequences of errors.
Comparing groups using a metric unrelated to the intended decision may produce a statistically accurate but practically uninformative assessment.
Absolute and Relative Differences Communicate Different Information
Performance disparities can be expressed as absolute differences or relative comparisons.
Suppose the false-negative rate is 5% for one group and 10% for another.
The absolute difference is five percentage points, while the second group's false-negative rate is twice that of the first.
Both descriptions are mathematically correct but emphasize different aspects of the disparity.
Researchers should report sufficient information for readers to understand the magnitude of the difference.
When relevant, presenting underlying counts alongside percentages can prevent misleading interpretations, particularly for small subgroups.
Statistical Significance Does Not Establish Practical Importance
A statistically significant difference indicates evidence against a specified null hypothesis under the assumptions of the statistical procedure.
It does not independently establish that the difference is large, harmful, or scientifically unacceptable.
With sufficiently large samples, relatively small differences may become statistically detectable.
Conversely, a substantial disparity may fail to reach conventional significance thresholds when subgroup samples are small.
Researchers should examine effect magnitude, uncertainty, and consequences rather than relying solely on p-values.
A nonsignificant result should not be interpreted as proof that subgroup performance is equivalent.
Confidence Intervals Help Communicate Uncertainty
Subgroup performance estimates are subject to sampling uncertainty.
Researchers should report suitable confidence intervals or other uncertainty measures for the relevant metrics and differences.
For example, an estimated sensitivity difference of eight percentage points may have a wide confidence interval when the evaluation contains relatively few positive cases.
The uncertainty may be large enough to include both negligible and consequential disparities.
Researchers should therefore avoid making definitive acceptability claims from imprecise estimates.
Confidence intervals do not correct invalid reference outcomes, unrepresentative samples, or other systematic limitations.
Equivalence Testing Can Address Whether Differences Are Sufficiently Small
Researchers sometimes want to establish that performance differences are small enough to be considered practically unimportant.
Equivalence testing may be appropriate when a scientifically justified equivalence margin can be defined.
For example, researchers might specify in advance the largest subgroup sensitivity difference they consider tolerable for a particular low-risk classification task.
They can then evaluate whether the estimated difference is sufficiently precise to support equivalence within that margin.
However, the margin must be justified through scientific and contextual reasoning.
Choosing a wide margin merely to demonstrate equivalence would undermine the credibility of the assessment.
Equivalence on one performance measure also does not establish fairness in every other respect.
Noninferiority Testing May Be Appropriate for Specific Comparisons
Noninferiority testing examines whether performance is not worse than a comparator by more than a predefined acceptable margin.
For example, researchers may investigate whether an AI-assisted classification procedure performs no worse than an established reference procedure beyond a justified threshold.
Such testing can be useful when the objective is to establish adequate performance rather than superiority.
However, noninferiority should not be confused with equivalence.
Noninferiority generally addresses a directional performance requirement, while equivalence evaluates whether differences fall within specified bounds.
The choice depends on the research question and the consequences of underperformance.
Performance Differences Must Be Interpreted Against Valid Reference Outcomes
A model can appear to perform differently across groups because the reference outcome does not measure the same construct appropriately.
For example, student engagement may be defined through frequent learning platform access.
That indicator may be less suitable for asynchronous learners who download materials and study offline.
If the reference labels are conceptually inappropriate, comparing model accuracy against them may produce misleading conclusions.
Researchers should examine construct validity and measurement comparability before attributing subgroup disparities to the model.
Statistical adjustment cannot compensate for an invalid definition of the outcome.
Different Error Types May Require Different Acceptability Standards
False positives and false negatives can produce different consequences.
For an academic support model, a false negative may mean that a student requiring assistance is overlooked.
A false positive may result in unnecessary outreach or resource allocation.
However, if a positive classification triggers restrictions or stigmatizing labels, false positives may become more consequential.
Researchers should examine the decision process associated with the model.
The acceptable difference in one error type may not be the same as the acceptable difference in another.
Group Prevalence Can Affect Performance Comparisons
The prevalence of the target outcome may differ across populations.
For example, the proportion of students requiring academic support may vary across educational programs.
Such differences can influence precision and other performance measures.
Researchers should distinguish disparities caused by model errors from differences associated with outcome prevalence.
They should also examine whether the reference outcomes themselves reflect unequal opportunities or measurement problems.
A numerical disparity should be investigated before it is classified as unfair or acceptable.
Fairness Metrics Can Produce Conflicting Judgments
Different fairness criteria evaluate different aspects of model behavior.
Equal opportunity focuses on true-positive-rate parity, while equalized odds considers both true-positive and false-positive rates.
Calibration examines the correspondence between predicted probabilities and observed outcomes.
Under some conditions, these criteria cannot all be satisfied simultaneously.
Researchers should therefore justify which fairness properties are most relevant to the intended use.
The issue of conflicting AI fairness metrics illustrates why acceptability cannot be determined by selecting whichever metric produces the most favorable result.
Subgroup Sample Size Affects the Strength of the Evidence
Small subgroups may produce unstable performance estimates.
For example, a sensitivity estimate based on ten actual positive cases changes by ten percentage points whenever one additional case is correctly or incorrectly identified.
Researchers should report the number of participants and relevant outcome events within each subgroup.
They should consider whether the available sample supports a sufficiently precise estimate of the performance difference.
Where evaluation data are inadequate, researchers should acknowledge uncertainty rather than declare the disparity acceptable.
Intersectional Differences May Require Additional Examination
Performance may vary across combinations of characteristics.
For example, an AI model may perform adequately for working students overall and for Filipino-language respondents overall while performing poorly for working students who respond in Filipino.
Such patterns may be relevant when they affect the study's conclusions or consequential decisions.
However, intersectional analyses can involve small samples and heightened confidentiality concerns.
Researchers should select meaningful comparisons and avoid unsupported generalizations from sparse data.
The Research Context Changes What Is Acceptable
The same numerical disparity may have different implications across applications.
For exploratory research, imperfect classifications may be acceptable when limitations are understood and conclusions are appropriately qualified.
For participant eligibility, clinical decisions, or allocation of important opportunities, stronger performance requirements may be necessary.
Researchers should examine the likelihood and severity of harm, the reversibility of decisions, and the availability of independent review.
Acceptability should be justified in relation to these conditions rather than inferred from model accuracy alone.
Researchers Should Consider the Availability of Alternatives
An AI system should not be evaluated only against an ideal of perfect performance.
Researchers may need to compare it with feasible alternatives, including established statistical models, human procedures, or combined workflows.
For example, a model exhibiting subgroup disparities may still outperform an existing process on relevant measures.
However, better performance than an inadequate comparator does not automatically establish that the AI system is sufficiently fair.
Researchers should consider whether alternative procedures could reduce consequential errors without creating unacceptable new risks.
Stakeholder Perspectives Can Inform Acceptability Judgments
Technical experts can estimate performance disparities and uncertainty.
However, deciding which consequences are tolerable may require input from affected populations, disciplinary specialists, research ethics bodies, and institutional decision-makers.
For example, students may have concerns about being incorrectly classified as academically at risk even when the system is intended to provide assistance.
Stakeholder consultation can help identify consequences that are not captured by numerical metrics.
It does not automatically resolve competing interests, but it can improve the justification for the selected performance requirements.
Performance Thresholds Should Be Established Before Final Evaluation
Researchers should define acceptable performance criteria before examining final validation results whenever feasible.
This reduces the risk of selecting thresholds that conveniently accommodate the observed disparities.
Predefined criteria may include minimum subgroup sensitivity, acceptable calibration error, or a justified maximum difference in a consequential error rate.
These requirements should reflect the intended application and available scientific evidence.
If thresholds are revised after evaluation, researchers should explain the rationale and distinguish exploratory decisions from originally specified criteria.
Bias Mitigation May Introduce Performance Trade-Offs
Researchers may attempt to reduce subgroup disparities through revised training data, adjusted thresholds, recalibration, or alternative models.
These interventions can improve one measure while worsening another.
For example, lowering a classification threshold may increase sensitivity but also increase false positives.
Researchers should evaluate whether the trade-offs are acceptable for the intended use.
A reduction in one subgroup disparity does not establish that every fairness concern has been resolved.
Acceptability Should Be Reassessed When Conditions Change
A model evaluated in one institution or population may perform differently elsewhere.
Changes in educational delivery, participant characteristics, or data collection procedures can affect predictive relationships.
Researchers should consider whether previous acceptability judgments remain applicable after substantial changes.
External validation and appropriate monitoring may be necessary.
Fairness and performance assessments should be understood as context-dependent rather than permanent certifications.
Researchers Should Report the Basis for Acceptability Judgments
Reports should identify the performance measures, relevant groups, observed differences, uncertainty, and predefined criteria.
Researchers should explain why the selected thresholds are appropriate and how error consequences were considered.
They should distinguish statistically supported findings from normative judgments about acceptable risk.
Where the evidence remains inconclusive, the report should acknowledge that uncertainty.
A statement that subgroup differences were acceptable is not methodologically sufficient without an explanation of the criteria and supporting evidence.
