01 · The Question
Is It Appropriate to Use AI When Research Participants May Be Especially Vulnerable?
A researcher studying children's learning experiences may want to use artificial intelligence to analyze interview transcripts. Another researcher may consider AI-assisted coding of conversations with older adults experiencing cognitive impairment, people living with disabilities, or individuals in economically precarious circumstances.
These applications may appear similar to ordinary AI-assisted qualitative analysis. Yet the consequences of a confidentiality breach, inaccurate interpretation, or inappropriate automated decision can be considerably more serious when participants have limited power, heightened exposure to harm, or difficulty exercising their rights.
The central question is not simply whether AI can process the information. It is whether using AI is ethically justified for the particular participants, data, and research decisions involved.
03 · What You Need to Know
How Vulnerability Changes the Ethical Evaluation of AI-Assisted Research
Vulnerability Is Contextual, Not Simply a Participant Label
Research ethics frequently identifies groups that may require additional protection, including children, people with impaired decision-making capacity, and individuals whose circumstances increase their susceptibility to coercion or exploitation. Nevertheless, belonging to a demographic group does not automatically mean that every member lacks autonomy or faces identical risks.
The Council for International Organizations of Medical Sciences (CIOMS) emphasizes that vulnerability should be understood in relation to the particular circumstances that may make individuals less able to protect their interests.
A child participating in a low-risk classroom observation, for example, faces different concerns from a child disclosing experiences of abuse. Similarly, a competent older adult should not be presumed incapable of informed consent merely because of age.
Researchers should therefore examine the interaction between participant characteristics, the research environment, the information collected, and the proposed AI application.
AI May Introduce Risks Beyond Those of the Original Research Procedure
Using AI is not necessarily equivalent to conducting the same analysis manually. The technology may introduce additional data recipients, processing locations, retention practices, security dependencies, or possibilities for unintended information disclosure.
Consider a researcher who has permission to record interviews with adolescents about school experiences. Uploading those recordings to an external AI service may constitute a materially different processing activity from having authorized researchers transcribe them within an institutionally controlled environment.
The researcher should determine whether the original consent, ethics approval, data-management arrangements, and applicable legal requirements cover the proposed processing.
The relevant issue is not whether AI was mentioned somewhere in the research protocol. It is whether participants' information will be handled in a manner consistent with the permissions, safeguards, and purposes governing its collection.
Children Require Age-Appropriate Consent and Assent Procedures
Research involving children commonly requires permission from a parent or legally authorized representative, together with the child's affirmative agreement or assent when appropriate. Exact requirements depend on jurisdiction, institutional review, the child's circumstances, and applicable exemptions or waivers.
Under the United States Common Rule, 45 CFR 46 Subpart D provides additional protections for children involved in covered research. These requirements should not be assumed to apply identically in every country or research setting.
Researchers should explain AI-related processing in language that participants and their representatives can reasonably understand. Relevant information may include whether interviews will be analyzed using AI, whether external providers will process the data, and whether AI-generated classifications will influence subsequent research activities.
Consent is not a substitute for risk minimization. Even where permission has been obtained, researchers remain responsible for assessing whether the proposed AI processing is appropriate.
Removing Names Does Not Necessarily Make Participant Data Anonymous
Children and other potentially vulnerable participants may be identifiable through combinations of seemingly ordinary details. A transcript without names could still reveal a participant through references to a small school, an unusual medical condition, family circumstances, or a distinctive event.
Direct identifiers and indirect identifiers therefore require separate consideration.
Removing names and student numbers may reduce identification risk, but it does not establish that information is anonymous. Free-text narratives, audio recordings, photographs, and small datasets can be especially challenging to de-identify effectively.
Watch Out
Do not upload identifiable or potentially re-identifiable participant information to an external AI system merely because direct identifiers have been removed. Verify the service's data-handling arrangements, institutional authorization, applicable privacy requirements, and whether a less intrusive processing method is available.
Different AI Tasks Carry Different Levels of Ethical Risk
The acceptability of AI depends partly on what the system is being asked to do.
| AI application |
Principal concern |
Appropriate safeguard |
| Formatting a blank interview guide |
Limited direct participant exposure |
Review questions for suitability and age appropriateness |
| Transcribing recorded interviews |
Disclosure of voices and identifiable information |
Use authorized processing and verify transcripts |
| Coding sensitive participant narratives |
Misinterpretation and confidentiality |
Protect data and independently review consequential coding |
| Classifying participants by vulnerability or risk |
Stigmatization, misclassification, and discriminatory consequences |
Validate the approach and retain qualified human judgment |
| Determining participation or protective intervention |
Potential direct effects on rights, welfare, or safety |
Apply heightened oversight and avoid unsupported autonomous decisions |
These are illustrative distinctions rather than universal classifications. An apparently administrative task can become high stakes if it involves sensitive information or influences consequential decisions.
The more directly an AI output affects participants, the stronger the justification, validation, and oversight generally need to be. This is particularly relevant when an AI error could materially harm research participants.
AI May Misinterpret Developmental, Cultural, or Disability-Related Differences
AI systems may process language or behavior differently across populations. Children's vocabulary, communication patterns, developmental stages, and expressions of emotion may differ from those represented in an AI system's evaluation data.
Comparable concerns arise when analyzing communication involving people with disabilities, speakers of underrepresented languages, or participants whose experiences are poorly represented in training or validation datasets.
An AI system might interpret an adolescent's sarcasm as hostility, treat a communication difference as evidence of impairment, or assign an inappropriate psychological label to an ambiguous statement.
Such possibilities should not be treated as proof that every AI model is biased in the same way. They demonstrate why evidence of performance in one population cannot automatically establish suitability for another.
Researchers should evaluate whether the tool has been tested on data sufficiently comparable to their intended population and whether observed errors have unequal consequences for particular groups.
Human Oversight Must Be Substantive
Human involvement is meaningful only when reviewers have the information, competence, authority, and time necessary to challenge AI outputs.
A researcher who simply accepts AI-generated codes after a superficial inspection has not necessarily established an effective safeguard. Where an interpretation may affect participant welfare, qualified reviewers should examine the underlying evidence and understand the limitations of the AI method.
Researchers should also distinguish ordinary analytical assistance from activities that involve identifying participant safety risks or assessing eligibility for participation. These tasks may require separate validation and procedural protections.
Ethics Approval and Data Governance Must Address the Actual AI Workflow
An institutional review board, research ethics committee, or equivalent authority may need to examine AI use when it changes the research procedures, information flows, participant risks, or approved protocol.
Researchers should document the AI system, its intended function, the categories of data processed, the processing environment, retention arrangements, access controls, human review procedures, and foreseeable failure scenarios.
Where an AI workflow is introduced after approval, investigators should determine whether an amendment or additional review is required before implementation.
Approval by an ethics committee does not transfer responsibility away from the research team. Researchers remain accountable for implementing the approved safeguards and responding to newly identified risks.
06 · What This Means for You
How to Decide Whether AI Is Appropriate for Your Participants
Begin by identifying the specific purpose of AI assistance. Avoid treating the availability of an AI tool as sufficient justification for introducing it into research involving people who may face heightened risks.
Evaluate the scientific benefit against the additional exposure created by AI processing. Consider what information the system receives, what it produces, who may rely on its outputs, and what could happen if it fails.
A simple decision framework
If AI is used only for non-sensitive preparatory work
Apply ordinary research quality controls and verify that no protected participant information is inadvertently included.
If AI processes identifiable or sensitive participant information
Establish an authorized processing environment, assess confidentiality risks, and confirm applicable consent and ethics requirements before proceeding.
If AI outputs may influence participant classification or welfare
Require task-specific validation, qualified oversight, and procedures for challenging or correcting errors.
If adequate safeguards cannot be demonstrated
Do not proceed with the proposed AI workflow. Redesign the activity or use a more appropriate alternative.
Where the research involves particularly sensitive disclosures, researchers may also need to evaluate whether AI is suitable for interpreting legally or ethically sensitive information.
The decision should remain proportionate to the actual risk. Responsible research neither excludes vulnerable populations from technological benefits without justification nor exposes them to unnecessary experimentation with insufficiently evaluated systems.
07 · A Quick Checklist
Before Using AI in Research With Vulnerable Participants
Before introducing AI into the research workflow, check:
Identify the participants' specific vulnerabilities and how these relate to the proposed AI task.
Document why AI is scientifically useful and whether a less intrusive alternative exists.
Confirm relevant ethics approval, consent, parental permission, and assent requirements.
Determine whether the AI provider or processing environment is authorized to handle the data.
Assess direct and indirect identification risks before transferring participant information.
Evaluate whether AI performance is appropriate for the participants' age, language, and circumstances.
Define which decisions require qualified human review and how disagreements will be resolved.
Establish procedures for responding to confidentiality breaches, harmful classifications, or other consequential errors.
Document AI use and review the workflow if the system, research procedures, or participant risks change.
