03 · What You Need to Know
Understanding Conflicts Among AI Fairness Metrics
Fairness Metrics Measure Different Properties
Algorithmic fairness is not represented by one universal mathematical definition.
Some metrics compare how frequently groups receive positive classifications. Others examine whether error rates are similar or whether predicted probabilities accurately represent observed outcomes.
These measures answer different questions.
For example, equal positive classification rates do not necessarily mean that eligible individuals are identified with equal accuracy.
Similarly, calibrated probabilities do not guarantee equal false-positive rates after classification.
Researchers should understand the property each metric evaluates before interpreting its results.
Classification-rate fairness
Examines how frequently different groups receive particular decisions or classifications.
Error-rate fairness
Examines whether incorrect and correct classifications occur at comparable rates across groups.
Probability-based fairness introduces additional requirements concerning the interpretation of predicted risk.
Demographic Parity Concerns Positive Classification Rates
Demographic parity, also called statistical parity, requires the probability of a positive classification to be equal across specified groups.
For example, an academic support system might satisfy demographic parity if 25% of working students and 25% of nonworking students receive positive support recommendations.
However, the groups may differ in the actual prevalence of support needs.
If one group has substantially more students requiring assistance, equal positive classification rates may produce different rates of missed cases.
Demographic parity can be appropriate for some fairness objectives, but it is not universally suitable for predictive tasks.
Researchers should explain why equal decision rates are relevant to the intended application.
Equal Opportunity Focuses on True-Positive Rates
Equal opportunity is commonly defined as equality of true-positive rates across relevant groups.
For an academic support model, it asks whether comparable proportions of students who genuinely need assistance are correctly identified.
Suppose 90% of actual support-need cases are identified in both groups.
The model satisfies true-positive-rate parity for those groups under the evaluation conditions.
However, false-positive rates may still differ.
Equal opportunity therefore addresses one important fairness property without necessarily satisfying others.
Its relevance depends on whether missed positive cases are a central concern.
Equalized Odds Requires Two Error-Rate Conditions
Equalized odds requires equality of both true-positive rates and false-positive rates across groups in binary classification.
In statistical terms, the predicted classification should be independent of group membership conditional on the actual outcome.
For example, an academic support model satisfying equalized odds would identify comparable proportions of actual support-need cases and incorrectly flag comparable proportions of non-support-need cases across groups.
This criterion is stronger than equal opportunity in the sense that it imposes an additional error-rate requirement.
However, satisfying equalized odds does not automatically establish calibration or valid measurement.
Researchers should evaluate whether the criterion addresses the relevant consequences of classification errors.
Calibration Concerns the Meaning of Predicted Probabilities
Calibration evaluates whether predicted probabilities correspond to observed outcome frequencies.
For example, among students assigned an estimated failure probability of approximately 20%, a calibrated model would be expected to observe failure in roughly 20% of comparable cases.
Group calibration requires this relationship to hold within relevant populations.
Calibration matters when researchers interpret predicted probabilities as estimates of risk.
However, a calibrated model may produce different classification error rates across groups when a threshold is applied.
Researchers should distinguish the calibration of risk scores from the fairness of decisions based on those scores.
Predictive Parity Is Not the Same as Calibration
Predictive parity commonly refers to equality of positive predictive value across groups.
Positive predictive value is the proportion of positive classifications that correspond to actual positive outcomes.
For example, among students classified as needing support, predictive parity would require comparable proportions to genuinely satisfy the reference criterion across groups.
This differs from calibration across the full range of predicted probabilities.
A model may satisfy predictive parity at a selected threshold without being well calibrated throughout its probability distribution.
Researchers should avoid using these terms interchangeably.
Outcome Prevalence Helps Explain Why Metrics Conflict
Outcome prevalence is the proportion of individuals in a group who experience the target outcome.
Suppose course failure occurs in 10% of students in one group and 30% in another.
Even if the model has identical sensitivity and specificity across the groups, the proportion of positive classifications that are correct will generally differ.
This follows from the relationship among prevalence, sensitivity, specificity, and positive predictive value.
Researchers should therefore recognize that unequal predictive parity does not necessarily indicate unequal error rates.
Conversely, equal predictive parity may require accepting different error-rate patterns.
Bayes' Theorem Helps Explain the Trade-Off
For a binary classification task, positive predictive value can be expressed as:
PPV = (Sensitivity × Prevalence) / [(Sensitivity × Prevalence) + (False-positive rate × (1 − Prevalence))]
The formula shows that positive predictive value depends partly on outcome prevalence.
Consequently, when prevalence differs across groups, equal sensitivity and false-positive rates will generally produce different positive predictive values, except under special conditions.
This relationship is mathematical rather than a consequence of an implementation mistake.
Researchers should distinguish unavoidable trade-offs under specified assumptions from disparities that arise because of poor model design or inadequate data.
Fairness Impossibility Results Have Specific Assumptions
Research on algorithmic fairness has established that certain desirable fairness properties cannot always be satisfied simultaneously.
For example, work by Kleinberg, Mullainathan, and Raghavan examines incompatibilities among particular risk-score fairness conditions when group base rates differ, apart from special cases.
Related research by Chouldechova examines conflicts involving predictive values and classification error rates.
These results do not establish that every fairness metric conflicts with every other metric.
They depend on specific definitions, assumptions, and model properties.
Researchers should avoid describing fairness impossibility results as proof that fair AI systems cannot exist.
The correct interpretation is that some mathematical objectives require explicit trade-offs under particular conditions.
Perfect Prediction Can Remove Some Conflicts
Some incompatibility results have exceptions when predictions are perfect.
A classifier with no false positives or false negatives satisfies several error-related fairness properties simultaneously.
However, perfect prediction is rarely a realistic assumption in complex research applications.
Researchers should also remember that perfect prediction of an invalid target does not establish fairness.
For example, a model might perfectly predict an operational definition of student engagement that fails to represent meaningful engagement.
Measurement validity remains necessary even when predictive performance is exceptionally strong.
Equalized Odds and Calibration Can Require Different Interventions
Researchers may attempt to improve fairness through threshold adjustments, recalibration, or changes to model development.
These interventions address different properties.
Recalibration aims to improve the correspondence between predicted probabilities and observed outcomes.
Threshold adjustment changes which cases receive positive classifications and therefore affects classification error rates.
Some post-processing methods may introduce randomized decisions to satisfy particular error-rate constraints.
Such procedures require careful justification when outputs influence consequential decisions.
Improving one fairness property may leave another unchanged or make it worse.
Group-Specific Thresholds Require Explicit Justification
Researchers may consider applying different decision thresholds to different groups to achieve a selected fairness criterion.
For example, a lower threshold might increase sensitivity for a group whose support needs are frequently missed.
However, the adjustment may also increase false positives.
Using group-specific thresholds can raise legal, ethical, and methodological questions depending on the application and jurisdiction.
Researchers should evaluate whether the proposed approach is permissible, justified, and proportionate to the intended use.
They should not adopt different thresholds merely to make one fairness metric appear favorable.
Fairness Metrics Depend on Reference-Outcome Validity
Every classification fairness metric relies on a definition of the actual outcome.
If that outcome is poorly measured, numerical fairness results may be misleading.
For example, a student-risk model may define disengagement through infrequent platform access.
This measure may inadequately represent asynchronous learners who study offline.
A model could satisfy equalized odds relative to those labels while still misrepresenting educational engagement.
Researchers should evaluate the validity and comparability of reference outcomes before treating fairness metrics as meaningful evidence.
Fairness Metrics May Conflict With Research Utility
Researchers may need to consider trade-offs between fairness constraints and other performance objectives.
For example, imposing a particular parity requirement may alter overall classification accuracy or the number of students identified for support.
However, maximizing predictive accuracy is not automatically the appropriate objective.
A model with slightly lower overall accuracy may be preferable if it substantially reduces consequential errors for a previously disadvantaged population.
Researchers should evaluate the scientific purpose, potential harms, and available alternatives.
The relevant trade-off should be explained rather than treated as an exclusively technical optimization problem.
Fairness Evaluation Should Consider Multiple Metrics
Researchers should generally examine more than one relevant performance measure when fairness-related decisions are consequential.
For example, a student-risk model may warrant evaluation of sensitivity, specificity, calibration, and subgroup outcome prevalence.
However, reporting many metrics does not mean that every metric must be optimized equally.
The purpose is to understand the model's behavior and identify the trade-offs associated with possible decisions.
Researchers should justify which criteria receive priority.
The broader question of what fairness means in AI-assisted research should guide metric selection.
Small Samples Can Make Fairness Comparisons Uncertain
Subgroup fairness estimates may be imprecise when evaluation samples are small.
For example, a false-positive rate based on only a few actual negative cases can change substantially when one classification changes.
Researchers should report subgroup sample sizes, outcome counts, and suitable uncertainty measures.
They should avoid claiming that fairness criteria are satisfied merely because observed differences are not statistically significant.
Likewise, apparent conflicts between metrics may require additional evaluation before firm conclusions are drawn.
Intersectional Fairness Adds Further Complexity
Fairness metrics evaluated across broad demographic groups may conceal disparities at the intersection of characteristics.
For example, a model may achieve similar sensitivity across language groups overall while performing poorly for working students using a particular language variety.
Researchers should consider relevant intersectional comparisons when the task and potential consequences justify them.
However, increasingly fine subgroup divisions can produce sparse data and confidentiality risks.
Evaluation should balance scientific relevance with statistical precision and ethical responsibilities.
Fairness Metrics Do Not Capture Every Form of Unfairness
Statistical parity criteria evaluate defined properties of predictions or decisions.
They do not independently establish that a model's target is appropriate, that affected individuals can challenge decisions, or that the resulting interventions are accessible.
For example, an academic support system may satisfy equal opportunity while offering services at times that working students cannot attend.
The classification may meet a selected metric while the broader process produces unequal benefits.
Researchers should therefore evaluate fairness throughout the relevant workflow rather than only at the model-output stage.
Researchers Should Select Fairness Criteria Before Final Evaluation
Fairness metrics should be selected according to the intended application and consequences of errors.
Whenever feasible, researchers should specify their primary criteria before examining final validation results.
This reduces the risk of selecting whichever metric makes the model appear most favorable.
Exploratory analyses may identify additional concerns, but they should be reported as such.
The criteria should be justified through scientific reasoning and relevant ethical considerations.
Conflicting Metrics Require Transparent Decisions
When fairness criteria conflict, researchers should explain which properties are prioritized and why.
For example, an academic support system may prioritize identifying students who genuinely need assistance while also monitoring false-positive rates and calibration.
Another application may require different priorities because incorrect classifications have different consequences.
Researchers should report the observed trade-offs, uncertainty, and limitations.
The objective is not to claim universal fairness but to establish that the selected evaluation and decision procedures are defensible for the intended use.
