03 · What You Need to Know
Understanding Error Types, Severity, and Complementarity in Human-AI Evaluation
Why is average accuracy insufficient for some human-AI comparisons?
Accuracy measures the proportion of evaluated decisions that match a defined reference standard.
It does not identify which decisions were incorrect or how consequential those errors might be.
For example, an AI model and a human researcher may both classify 90% of research abstracts correctly. However, the AI may disproportionately misclassify mixed-methods studies, while the human researcher may struggle with unfamiliar quantitative designs.
The same average accuracy can therefore conceal different methodological limitations.
Researchers should examine whether these differences matter for the intended application rather than assuming that equivalent aggregate scores imply equivalent performance.
What is the difference between error frequency and error severity?
Error frequency
How often a defined mistake occurs within the evaluated tasks or outputs.
Error severity
The importance or consequences of an error relative to the intended application and its decision context.
A minor bibliographic formatting mistake may have limited consequences. A fabricated research finding or an incorrect recommendation about statistical assumptions may substantially undermine a scientific conclusion.
Two conditions can have identical error frequencies but different distributions of severity.
Severity should therefore be evaluated through explicit, task-appropriate criteria rather than assumed from the source of the response.
What kinds of errors should researchers distinguish?
The categories should reflect the task and the available reference evidence.
| Error category |
Illustrative example |
Potential consequence |
| Factual error |
Reporting an incorrect sample size |
Misrepresentation of research evidence |
| Unsupported claim |
Introducing a finding absent from the source |
Misleading interpretation |
| Methodological error |
Recommending an analysis inconsistent with the study design |
Invalid analytical decisions |
| Omission |
Failing to identify an essential limitation |
Incomplete assessment |
| Classification error |
Assigning a mixed-methods study to the quantitative category |
Incorrect research categorization |
| Numerical error |
Calculating an effect estimate incorrectly |
Inaccurate quantitative conclusions |
| Unjustified certainty |
Presenting an ambiguous conclusion as definitive |
Overconfidence in weak evidence |
Categories may overlap. Researchers should define whether errors can receive multiple labels and how overlapping classifications are handled.
They should not assume that particular errors are exclusive to humans or AI. Both may produce factual inaccuracies, omissions, or inappropriate methodological judgments.
How should researchers establish an error taxonomy?
An error taxonomy is a defined classification system for identifying and organizing mistakes.
Researchers should develop it from the intended task, relevant methodological literature, and the consequences of incorrect outputs.
For example, a study evaluating AI-generated statistical recommendations might distinguish incorrect test selection, failure to recognize violated assumptions, computational mistakes, and unsupported causal interpretations.
Categories should have clear definitions, inclusion criteria, and examples.
Researchers may pilot the taxonomy on a separate sample and refine ambiguous definitions before formal scoring.
When exploratory categories emerge during analysis, they should be distinguished from prespecified categories.
Should AI and human errors use the same classification framework?
Generally, yes, when the study compares performance on the same task.
A common framework helps prevent researchers from classifying AI mistakes more harshly or treating human errors as inherently less consequential.
However, some failure modes may require source-specific subcategories.
For example, an AI system may experience a retrieval-tool failure, while a human participant may misread a source document.
Both may lead to an incorrect final answer, but the underlying processes differ.
Researchers should distinguish comparable outcome errors from mechanisms specific to each condition.
What is paired error analysis?
Paired error analysis examines AI and human performance on the same evaluation items.
Rather than comparing only separate accuracy percentages, researchers identify cases where both are correct, both are incorrect, or only one produces the correct answer.
This approach is particularly useful because it reveals the overlap between human and AI failures.
|
|
Human correct |
Human incorrect |
|
AI correct
|
Both correct |
AI-only correct |
|
AI incorrect
|
Human-only correct |
Both incorrect |
These four categories provide information that separate accuracy scores cannot reveal.
For example, two conditions may achieve the same accuracy while having substantial disagreement about which items are difficult.
How can error overlap be measured?
One descriptive measure is the proportion of evaluation items on which both AI and the human comparator produce incorrect responses.
In this example, 18 errors are unique to AI and 18 are unique to the human comparator.
The result reveals substantial differences in the cases each condition answers incorrectly, despite equal overall accuracy.
What is potential human-AI complementarity?
Complementarity concerns whether the strengths of humans and AI can be combined to improve performance.
If AI correctly answers questions that humans miss, and humans correctly answer questions that AI misses, their outputs contain potentially complementary information.
However, this is only an opportunity for improvement.
Realizing the benefit requires a mechanism for selecting the correct answer or appropriately integrating the two judgments.
Research on human-AI complementarity distinguishes the availability of complementary information from the improvement actually achieved through collaboration.
A system may contain considerable theoretical complementarity while users remain unable to recognize when the AI is wrong.
How can researchers calculate an oracle upper bound?
For paired binary correctness outcomes, researchers can calculate the proportion of items on which at least one condition is correct.
The result indicates that a perfect selection mechanism could theoretically achieve 94% accuracy using the two sets of responses.
It does not demonstrate that an actual human-AI team would achieve that result.
Researchers should distinguish potential complementarity from empirically observed collaborative performance.
Can identical accuracy conceal different error overlap?
Yes. Consider two pairs of performers, each achieving 85% accuracy individually.
In one pair, both make errors on nearly the same items. In another, their errors overlap only occasionally.
The second pair offers greater potential for complementary correction because one performer is more often correct when the other is wrong.
However, the value of this complementarity depends on whether the correct response can be identified reliably.
Researchers should therefore report error overlap alongside individual accuracy when human-AI collaboration is a relevant application.
Should researchers compare false positives and false negatives separately?
Yes, when the task involves classification and different error types have different consequences.
For example, in research article screening, a false negative may exclude a relevant study from further consideration. A false positive may require additional manual review.
These errors have different implications for workload and evidence completeness.
Two classifiers may achieve similar accuracy but differ substantially in sensitivity and specificity.
Researchers should report appropriate confusion matrices, precision, recall, and other task-relevant measures.
The choice of metric should follow the consequences of errors rather than an assumption that all incorrect classifications are equally important.
How should error severity be scored?
Researchers can establish a rubric distinguishing errors according to their consequences.
For example, a research-methodology evaluation might distinguish minor inaccuracies, errors that materially change an interpretation, and errors that invalidate a central recommendation.
Severity criteria should be developed before final scoring where feasible.
Evaluators should assess errors using the same standards for AI and human responses.
Where judgments are subjective, independent scoring and appropriate agreement analysis may be useful.
Researchers should avoid constructing arbitrary numerical severity weights solely to produce a preferred ranking.
Should confidence and uncertainty be examined?
Yes, when confidence is relevant to the intended application.
A response that explicitly acknowledges uncertainty may be easier to handle responsibly than a confident but incorrect recommendation.
However, verbal confidence does not necessarily correspond to calibrated probability.
Researchers should distinguish expressed certainty from measured calibration.
If reliable probability estimates are available, calibration and selective prediction measures may help determine whether a system can identify cases requiring additional review.
Human confidence can also be measured, although self-reported confidence may be influenced by expertise, task difficulty, and other factors.
Can error analysis reveal systematic weaknesses?
Yes. Errors may cluster within particular task categories, languages, document types, or levels of difficulty.
For example, AI may perform well on straightforward statistical questions but struggle when a scenario contains ambiguous assumptions.
Human researchers may perform differently according to their disciplinary specialization.
Researchers should examine whether error patterns persist across relevant subgroups and repeated testing.
However, exploratory patterns should not automatically be interpreted as stable characteristics of the system.
The principles of subgroup performance evaluation remain relevant.
How should human and AI disagreements be adjudicated?
Disagreement should be assessed against an independently defensible reference standard.
For factual tasks, this may involve verifying source documents or calculations.
For complex methodological judgments, expert adjudication using explicit criteria may be necessary.
Researchers should avoid assuming that the human response is correct merely because it came from a professional, or that the AI response is correct because it appears more detailed.
Where several answers are defensible, the scoring procedure should accommodate legitimate alternatives.
Ambiguous cases should be documented rather than silently excluded.
Should error evaluators be blinded to response origin?
Blinding may reduce source-related expectations when evaluators classify error severity or methodological quality.
For example, evaluators may be more inclined to describe an AI error as a hallucination and a similar human error as an oversight.
Using a common taxonomy and concealing source labels can reduce some of these risks.
However, source-specific failure mechanisms may need to be examined after the initial blinded outcome assessment.
The procedures for blinding human-AI evaluators should preserve substantive evidence while concealing irrelevant identifiers.
How should repeated AI outputs be handled?
Generative AI may produce different responses to the same question across repeated executions.
Researchers should specify whether error analysis concerns one response per item, repeated generations, or an aggregated prediction procedure.
Repeated outputs from the same item should not be treated as independent observations without justification.
Error patterns may vary across generations, especially for ambiguous tasks.
Researchers may therefore report both item-level error incidence and generation-level variability.
These considerations connect error analysis with response stability evaluation.
How should error differences be analyzed statistically?
Statistical analysis should reflect the paired or clustered structure of the evaluation.
For paired binary correctness outcomes, McNemar's test may be appropriate for examining differences in marginal error rates.
However, equal marginal accuracy does not imply identical error patterns.
Researchers may separately examine agreement, joint errors, and category-specific differences.
When multiple human participants evaluate overlapping items, hierarchical models or other cluster-aware procedures may be needed.
Confidence intervals should accompany relevant estimates, particularly when serious errors are rare.
Does error complementarity guarantee better human-AI teamwork?
No. Complementary errors create the possibility of improvement but do not guarantee that the correct answer will be selected.
A human reviewer may defer to an incorrect AI recommendation even when their original judgment was correct.
Conversely, a reviewer may reject a correct AI answer because it conflicts with an initial assumption.
Researchers should therefore evaluate the actual human-AI collaboration process.
This may involve measuring whether users recognize disagreements, verify relevant evidence, and correct errors.
Research on human-AI complementarity emphasizes that the distribution of errors and the effectiveness of integration both influence team outcomes.
How should error analysis be reported?
A transparent report should describe the error taxonomy, reference standard, evaluation items, scoring procedures, and treatment of ambiguous responses.
Researchers should report aggregate accuracy alongside relevant error categories, severity distributions, and overlap measures.
Where collaboration is examined, potential complementarity should be distinguished from observed team performance.
The report should also identify whether analyses were prespecified or exploratory.
These practices support fair human-AI performance comparisons by making the nature of observed differences more transparent.
