03 · What You Need to Know
Who Should Participate in Defining Fairness and How Should Decisions Be Made?
Choosing a Fairness Definition Is Not Merely a Mathematical Problem
Statistical fairness metrics evaluate defined properties of predictions and decisions.
For example, demographic parity concerns positive classification rates, while equal opportunity concerns true-positive-rate parity.
Equalized odds considers both true-positive and false-positive rates.
Mathematics can establish whether a model satisfies these criteria under specified conditions.
However, mathematical evaluation cannot independently determine which criterion should govern a particular research application.
That decision requires examining the research objective, consequences of errors, affected populations, and relevant ethical obligations.
The broader meaning of AI fairness in research therefore includes normative reasoning as well as technical evaluation.
The Principal Investigator Retains Research Accountability
The principal investigator or designated research leadership ordinarily carries responsibility for ensuring that the study follows its approved methodology and applicable research requirements.
This includes explaining how AI systems are selected, evaluated, and used.
However, accountability does not mean that the principal investigator must make every technical or ethical judgment independently.
Appropriate expertise and consultation may be necessary.
For example, a principal investigator studying educational technology may require assistance from statisticians when evaluating competing fairness metrics.
The research leadership should ensure that the final criteria are justified and documented rather than treating a software developer's preferred metric as sufficient authority.
Technical Experts Should Explain What Each Metric Establishes
Machine-learning specialists and statisticians can explain how fairness metrics are defined, calculated, and interpreted.
They can identify mathematical incompatibilities, evaluate subgroup uncertainty, and assess the effects of threshold adjustments.
For example, technical experts may demonstrate that equalized odds and certain calibration-related requirements conflict under particular conditions.
However, technical expertise alone does not determine whether missed positive cases are ethically more important than false alarms.
Technical specialists should make the trade-offs understandable to the wider research team.
Their role is essential, but it should not substitute for broader scientific and ethical deliberation.
Disciplinary Experts Should Evaluate the Meaning of the Outcome
Fairness metrics depend on how the research defines and measures the target outcome.
For example, an AI system may classify students as disengaged using learning management system activity.
Educational researchers should examine whether those indicators adequately represent engagement across different learning environments.
If the construct is invalid, statistical fairness evaluation may produce misleading reassurance.
Disciplinary expertise helps determine whether the categories, predictors, and reference outcomes are appropriate.
Fairness should not be evaluated independently of construct validity.
Affected Populations Can Identify Consequences Researchers Overlook
People affected by AI-assisted decisions may understand practical consequences that are not visible in technical performance metrics.
For example, students may explain that an academic-risk label could influence how instructors perceive them.
Working students may also identify barriers to accessing support services offered during conventional class hours.
These concerns can change which fairness criteria are relevant.
Researchers should consider meaningful participation from affected populations when the AI system influences opportunities, treatment, or access to resources.
However, stakeholder participation should not be treated as a ceremonial consultation after all consequential decisions have already been made.
Stakeholder Participation Does Not Require Unanimity
Different stakeholders may reasonably prioritize different outcomes.
Students may emphasize avoiding unjustified labeling, while administrators may emphasize identifying support needs within limited resources.
Researchers may prioritize valid measurement and reliable comparisons.
These objectives can conflict.
A participatory process should make disagreements visible and provide a defensible method for considering them.
Unanimous agreement is not always possible or necessary.
The research team should document how competing considerations informed the final decision and which concerns remain unresolved.
Institutional Decision-Makers Have Responsibilities but Also Interests
University administrators, healthcare organizations, government agencies, and other institutional users may have legitimate responsibilities concerning the deployment of AI systems.
They may need to consider available resources, operational feasibility, legal requirements, and institutional accountability.
However, organizational convenience should not automatically determine the fairness definition.
For example, a university may prefer a threshold that limits the number of students referred for academic support.
That choice may be operationally convenient while leaving substantial unmet needs among particular populations.
Researchers should distinguish institutional constraints from scientific and ethical justification.
Research Ethics Committees May Need to Review Consequential Decisions
When AI systems affect participant recruitment, eligibility, access to interventions, or other consequential aspects of a study, relevant research ethics oversight may be necessary.
Ethics committees can evaluate whether risks are justified, whether participants are treated appropriately, and whether safeguards are adequate.
However, ethics approval does not independently establish that a particular fairness metric is mathematically or scientifically appropriate.
Researchers remain responsible for methodological validation.
The role of ethics oversight depends on the research design, institutional requirements, and applicable jurisdiction.
Legal and Regulatory Requirements May Constrain the Choice
Some applications are subject to legal obligations concerning discrimination, privacy, accessibility, or automated decision-making.
These requirements may affect which variables can be used and how decisions must be reviewed.
Researchers should not assume that a statistically favorable fairness metric establishes legal compliance.
Likewise, a metric selected for scientific purposes may not satisfy every applicable legal obligation.
Where consequential decisions are involved, appropriate legal or compliance expertise may be required.
Researchers should identify applicable requirements rather than relying on generalized claims about AI regulation.
Fairness Definitions Should Follow the Research Purpose
A fairness criterion should address the actual function of the AI system.
For example, a model intended to identify students needing academic support may prioritize sensitivity because missed positive cases could prevent assistance.
A system used to allocate scarce scholarships may raise different concerns about eligibility, false positives, and equal access.
The same fairness metric is not necessarily appropriate for both applications.
Researchers should define the intended use before selecting a metric.
They should also distinguish exploratory research outputs from decisions that directly affect individuals.
The Consequences of Errors Should Inform the Decision
False positives and false negatives may have different consequences depending on the application.
In a voluntary tutoring program, incorrectly identifying a student as needing support may have relatively limited consequences.
However, the same classification could be harmful if it leads to restrictions or stigmatizing treatment.
Similarly, failing to identify a student who needs assistance may result in a missed opportunity for intervention.
Researchers should examine both the likelihood and severity of relevant harms.
The fairness criterion should reflect those consequences rather than merely optimize a convenient numerical property.
Fairness Metrics May Be Mathematically Incompatible
Some fairness criteria cannot all be satisfied simultaneously under specified conditions.
For example, differences in outcome prevalence can create conflicts between predictive parity and equalized error-rate requirements.
Researchers should understand the assumptions underlying these incompatibilities.
They should not claim that mathematical trade-offs make fairness irrelevant.
Instead, the conflict makes transparent prioritization more important.
The relationship among conflicting AI fairness metrics should be explained to stakeholders involved in consequential decisions.
Demographic Parity May Be Appropriate for Some Objectives
Demographic parity requires equal positive classification or decision rates across groups.
It may be relevant when the principal concern involves the distribution of opportunities or favorable decisions.
However, equal classification rates may not reflect equal need or eligibility.
For example, forcing equal proportions of academic support recommendations could overlook genuine differences in student circumstances.
Researchers should therefore justify demographic parity according to the intended decision rather than treat it as a universal standard.
Equal Opportunity May Be Appropriate When Missed Cases Matter
Equal opportunity commonly requires equal true-positive rates across relevant groups.
For a student-support model, this means identifying comparable proportions of students who genuinely require assistance.
The criterion may be appropriate when the central concern is avoiding unequal exclusion from beneficial interventions.
However, it does not require equal false-positive rates.
Researchers should examine whether that limitation is acceptable for the intended application.
Calibration May Matter When AI Produces Risk Scores
Some AI systems produce predicted probabilities rather than only binary classifications.
Calibration examines whether predicted probabilities correspond to observed outcome frequencies.
For example, among students assigned a predicted failure probability of approximately 30%, a calibrated model would be expected to observe failure in roughly 30% of comparable cases.
Group calibration may be important when decision-makers interpret risk scores directly.
However, calibration does not guarantee equal error rates after applying a classification threshold.
Researchers should distinguish the reliability of predicted probabilities from the fairness of decisions based on them.
Fairness Criteria Should Be Established Before Final Model Evaluation
Researchers should define the primary fairness objectives before examining final validation results whenever feasible.
This reduces the risk of selecting whichever metric makes the chosen model appear most favorable.
Predefined criteria may include minimum subgroup sensitivity, acceptable calibration error, or justified limits on consequential performance differences.
The thresholds should reflect the research objective and relevant risks.
Exploratory analyses may identify additional concerns, but these should be distinguished from originally specified criteria.
Stakeholder Consultation Should Be Structured
Meaningful participation requires more than asking stakeholders whether they believe a system is fair.
Researchers should explain the intended AI task, the available fairness definitions, and the practical consequences of competing choices.
Participants should have opportunities to identify concerns and question assumptions.
Where possible, consultation should occur before the fairness criteria are finalized.
Researchers should document who was consulted, which perspectives were represented, and how feedback influenced the decision.
Consultation methods should be appropriate to the population and should avoid creating unnecessary burdens or risks.
Representation Within Stakeholder Groups Matters
A small number of consulted individuals may not adequately represent the experiences of an entire population.
For example, student representatives from conventional programs may not fully understand the circumstances of working students enrolled in asynchronous courses.
Researchers should consider whether relevant subgroups have meaningful opportunities to participate.
However, representation should be guided by the research context rather than arbitrary demographic quotas.
The objective is to identify perspectives relevant to the consequences of the AI system.
Conflicts of Interest Should Be Considered
Organizations developing or deploying AI systems may have financial, operational, or reputational interests in particular evaluation outcomes.
Researchers may also have incentives to demonstrate that a selected model performs well.
These interests do not automatically invalidate fairness decisions, but they can influence which metrics receive attention.
Where appropriate, independent review and transparent documentation may reduce the risk of selective evaluation.
Researchers should disclose relevant conflicts of interest and avoid allowing performance targets to be determined solely by those benefiting from deployment.
Fairness Decisions Should Be Revisable
A fairness definition selected for one population or deployment context may become less appropriate when conditions change.
For example, a student-risk model may initially support optional academic advising but later be proposed for scholarship eligibility decisions.
The consequences of errors would change substantially.
Researchers should reassess the fairness criteria when the intended use changes.
New evidence about subgroup performance or stakeholder concerns may also justify reconsideration.
Fairness governance should allow revision rather than treating the initial decision as permanent.
Researchers Should Document Who Made the Decision
Research reports should explain how fairness criteria were selected and who participated in consequential decisions.
Relevant documentation may include the research objectives, technical analyses, stakeholder consultation, ethics considerations, and institutional requirements.
Researchers should distinguish advisory contributions from formal decision authority.
They should also describe disagreements or limitations that materially affect interpretation.
Transparency does not guarantee that the selected definition is correct, but it allows readers to evaluate the reasoning behind it.
Fairness Governance Should Extend Beyond Model Metrics
A model may satisfy a selected statistical criterion while producing unequal consequences through the broader institutional process.
For example, an academic support classifier may identify students comparably across groups while the resulting services remain inaccessible to working learners.
Researchers should examine whether fairness concerns arise during implementation, communication, or intervention delivery.
The choice of a fairness metric should therefore be situated within the complete research and decision workflow.
A technically defensible model does not automatically make every institutional use of its outputs fair.
