03 · What You Need to Know
How AI-Assisted Participant Eligibility Screening Should Be Evaluated
What Is Research Participant Eligibility Assessment?
Participant eligibility assessment determines whether an individual satisfies the inclusion and exclusion criteria established for a research study.
These criteria define the population the study intends to investigate and may also serve important safety, scientific, and ethical purposes.
Inclusion criteria identify characteristics required for participation, such as a defined age range, diagnosis, educational status, or relevant experience.
Exclusion criteria identify circumstances that make participation inappropriate under the approved study design.
For example, a clinical study may exclude individuals with particular medical conditions or medication exposures because of potential safety concerns or methodological requirements.
Eligibility assessment must follow the actual protocol rather than a generalized understanding of who appears suitable for the study.
How Can AI Assist With Screening?
AI systems may assist researchers by extracting information from records, identifying relevant terminology, organizing screening data, and comparing documented characteristics with predefined criteria.
For example, an AI system might identify whether a medical record contains evidence of a required diagnosis or flag an exclusion condition for closer examination.
In nonclinical research, AI might help organize responses to preliminary screening questionnaires.
These applications can potentially reduce administrative effort.
However, the accuracy and appropriateness of the assistance depend on the system, data quality, screening criteria, and workflow.
A model that performs well at extracting information does not necessarily perform equally well at determining whether that information satisfies the complete eligibility requirements.
Screening Assistance and Final Eligibility Determination Are Different
Preliminary AI-assisted screening
AI identifies potentially relevant records, extracts information, or flags possible eligibility conditions for authorized review.
Final eligibility determination
Eligibility is established under the approved protocol by the authorized process, including qualified human judgment where required.
The distinction matters because preliminary screening may operate under different requirements from final decisions affecting participation.
For example, an AI-generated flag may indicate that a record requires additional examination. It should not automatically be interpreted as confirmation that the person is eligible or ineligible.
Some criteria may be objectively verifiable, while others require contextual or clinical interpretation.
Researchers should determine which components of screening can be appropriately automated and which require authorized judgment.
Why Inclusion and Exclusion Criteria Must Be Applied Precisely
Research protocols may define eligibility using precise terminology, thresholds, time periods, or combinations of conditions.
An AI system may produce a plausible classification while overlooking one of these requirements.
For example, a study might require a laboratory result within a specified period before enrollment.
A system that identifies a qualifying result from an earlier date may incorrectly classify the individual as eligible.
Similarly, a medication-related exclusion criterion may depend on dosage, timing, or a particular drug category.
Researchers should not assume that semantic similarity between a record and the protocol establishes eligibility.
Screening procedures must preserve the meaning and conditions of the approved criteria.
Why False Inclusions and False Exclusions Have Different Consequences
Eligibility screening can produce at least two important types of classification error.
A false inclusion occurs when someone who does not satisfy the relevant criteria is classified as eligible.
A false exclusion occurs when someone who satisfies the criteria is classified as ineligible.
The consequences may differ.
| Error type |
Illustrative consequence |
Relevant concern |
| False inclusion |
An ineligible participant proceeds toward enrollment |
Participant safety, protocol compliance, and study validity |
| False exclusion |
An eligible individual is not considered for participation |
Fair access, recruitment completeness, and selection bias |
| Uncertain classification treated as definite |
Missing or ambiguous information is interpreted as conclusive |
Inadequate verification and unsupported decisions |
| Incorrect extraction |
A relevant condition or date is missed or misread |
Errors propagated into eligibility assessment |
| Inconsistent classification |
Similar records receive different decisions |
Reliability and procedural fairness |
Researchers should evaluate both the frequency and consequences of these errors.
Overall classification accuracy may be insufficient when particular mistakes have serious implications.
Why Sensitivity and Specificity Matter
For binary screening tasks, sensitivity and specificity may help characterize performance.
Sensitivity describes the proportion of truly eligible individuals correctly identified as eligible, when eligibility is treated as the positive class.
Specificity describes the proportion of truly ineligible individuals correctly identified as ineligible.
These definitions depend on how the positive class is specified.
Researchers may also examine positive predictive value, negative predictive value, and other measures relevant to the workflow.
Predictive values depend partly on the prevalence of eligibility within the screened population.
A screening system can therefore perform differently when applied to populations with different eligibility rates.
Performance measures should be interpreted in relation to the intended use rather than treated as abstract indicators of quality.
Why High Accuracy Can Be Misleading
Suppose a hypothetical study screens 1,000 individuals, of whom only 50 satisfy the eligibility criteria.
A system that classifies everyone as ineligible would be correct for 950 individuals and achieve 95% overall accuracy.
However, it would fail to identify every eligible participant.
This example demonstrates why overall accuracy alone may provide a misleading picture of screening usefulness.
Researchers should examine the confusion matrix and task-relevant performance measures.
They should also determine whether particular errors have consequences that require stricter performance thresholds.
No universal accuracy threshold establishes that an AI eligibility system is suitable for every research study.
Incomplete Records Require Explicit Handling
Participant records may contain missing laboratory results, incomplete medical histories, ambiguous diagnoses, or outdated information.
An AI system may attempt to infer missing details from surrounding text.
Such inference may produce a plausible answer without sufficient evidential support.
Researchers should distinguish between a criterion that is satisfied, a criterion that is not satisfied, and a criterion that cannot yet be evaluated.
For example, an absent laboratory result should not automatically be interpreted as a normal result.
A defensible screening workflow should include a mechanism for identifying uncertain cases and obtaining additional information where authorized and appropriate.
Uncertainty should remain visible rather than being concealed within a confident binary classification.
How AI Screening Could Introduce Selection Bias
Eligibility screening influences who enters the study and therefore which population is represented in the resulting data.
If an AI system systematically misses eligible individuals with particular characteristics, the recruited sample may differ from the intended population.
For example, performance may vary across institutions that use different medical terminology or record structures.
Language differences, incomplete documentation, and unequal access to healthcare may also affect the information available for screening.
These possibilities require empirical evaluation rather than an assumption that AI screening is inherently biased.
Researchers should examine whether screening performance differs across relevant subgroups and whether such differences affect recruitment or study validity.
Participant Eligibility Is Also an Ethical Question
Eligibility criteria are not merely technical rules for organizing a dataset.
They may determine who can participate in research, who is exposed to study procedures, and whose experiences contribute to scientific evidence.
The World Medical Association's Declaration of Helsinki emphasizes the rights, safety, and well-being of research participants.
AI-assisted screening should therefore be evaluated within the study's ethical framework.
For example, an automated system that incorrectly excludes eligible individuals could affect fair access to participation.
Conversely, including an ineligible participant could expose the person to risks the protocol was designed to avoid.
The relevant ethical implications depend on the study and the consequences of the decision.
Why the Approved Protocol Must Remain Authoritative
AI systems may suggest alternative interpretations of inclusion and exclusion criteria.
However, an AI recommendation does not authorize researchers to modify the approved eligibility rules.
For example, a system might suggest that a participant is sufficiently similar to the intended population despite failing a specified criterion.
Whether the protocol permits an exception cannot be determined simply by asking AI for a justification.
Researchers should follow the approved procedures and obtain any necessary amendments or institutional authorization.
Eligibility criteria should not be relaxed or tightened informally because an AI system produces a persuasive explanation.
How Should AI Eligibility Systems Be Validated?
Validation should reflect the actual screening workflow and the population in which the system will operate.
Researchers may need to compare AI classifications with eligibility determinations made using an appropriately established reference process.
Important considerations include the accuracy of information extraction, interpretation of protocol criteria, handling of incomplete records, and consequences of classification errors.
Performance should also be evaluated across relevant subgroups and data sources where appropriate.
Independent testing may be necessary to determine whether results generalize beyond the data used during system development.
Validation should not be limited to examples selected because the AI system handled them successfully.
Where an application is consequential, prospective evaluation may be warranted before routine deployment.
What Is the Role of Human Review?
Human oversight can help identify errors that automated screening fails to recognize.
However, the effectiveness of review depends on whether authorized personnel have access to the underlying records, understand the protocol, and can challenge the AI recommendation.
A reviewer who merely confirms the system's classification without examining relevant evidence may provide limited protection.
For example, a participant may be classified as eligible despite an exclusion condition documented elsewhere in the record.
Meaningful review should allow the authorized researcher to identify and correct that error before enrollment.
The necessary oversight arrangement depends on the task and applicable requirements.
This connects to designing human oversight that is substantive rather than ceremonial.
Can AI Be Used to Prioritize Records for Review?
Potentially. AI may help identify records that appear more likely to satisfy eligibility criteria or that require additional examination.
However, prioritization can also affect which individuals receive attention.
If the system consistently ranks certain eligible individuals lower, they may be overlooked when recruitment resources are limited.
Researchers should therefore distinguish between using AI to organize review and using AI to exclude individuals from consideration.
A prioritization workflow should be evaluated for its effects on recruitment completeness and fairness.
The fact that a system does not formally issue a final eligibility decision does not mean that its influence is inconsequential.
Why Privacy and Confidentiality Matter During Screening
Eligibility assessment may involve identifiable health records, demographic information, educational records, or other sensitive data.
Submitting these records to an external AI service may introduce additional privacy and contractual obligations.
Researchers should determine whether the processing is authorized under the approved protocol, participant consent, institutional agreements, and applicable data-protection law.
Relevant considerations include access controls, retention, secondary use, subprocessors, and cross-border data transfers.
Removing direct identifiers may not eliminate reidentification risk.
Researchers should not assume that a tool is suitable for participant screening merely because it offers accurate classifications.
Can AI-Assisted Screening Affect Research Reproducibility?
Eligibility decisions influence the composition of the final research sample.
When AI contributes to these decisions, researchers should preserve sufficient information to explain how participants were assessed.
Relevant records may include the protocol criteria, system version, screening procedures, verification arrangements, and final eligibility decisions.
Documentation should comply with confidentiality and data-retention requirements.
Changes in AI systems or screening procedures may affect which records are flagged, making version control and change documentation relevant.
Reproducibility does not require public disclosure of confidential participant records. It requires appropriate transparency about the methods used and the ability to examine them through authorized processes.
When Should AI Screening Be Avoided?
AI-assisted screening may be inappropriate when the system cannot reliably interpret the eligibility criteria, relevant errors cannot be detected, or adequate human oversight is unavailable.
It may also be unsuitable when the proposed data processing is unauthorized or when the system's performance has not been adequately established for the intended population.
For example, a general-purpose language model should not be treated as an independent eligibility authority merely because it can read clinical records and produce classifications.
The broader question of AI use in high-stakes research requires assessing whether the benefits justify the risks under the available safeguards.
Where necessary conditions cannot be met, researchers should use an appropriately authorized alternative screening process.
