Is De-Identification Enough to Make AI Use With Participant Data Ethical?

If Nobody Can Identify the Participants, Is the AI Use Ethical?

De-identification seems to offer an appealing solution to the privacy problem of AI-assisted research. Remove the identifiers, protect the participants, and then use the data with an AI system.

That reasoning addresses an important risk, but it can make ethics look narrower than it is. Research participants can have interests in how information derived from them is used even when researchers have substantially reduced the chance that anyone will identify a particular individual. AI can also generate inferences, expose groups to harms, enable uses participants did not anticipate, or move data into environments that conflict with commitments made during the research.

The ethical question is therefore not only “Can anyone identify these people?” It is also “Is this an appropriate use of what these people contributed to the research?”

De-Identification Helps, but Ethical Acceptability Requires More

Why Research Ethics Does Not End With Identifiability

Privacy Is Important, but It Is Not the Whole of Research Ethics

De-identification primarily addresses one cluster of concerns: the possibility that information will be connected to particular people and expose them to privacy or confidentiality harms. That is a meaningful safeguard.

Research ethics, however, also concerns autonomy, respect for persons, justice, proportionality of risks and benefits, responsible stewardship of research materials, and obligations created by the relationship between researchers and participants. An AI use can therefore present ethical concerns even when conventional identifiers have been removed.

The U.S. Secretary's Advisory Committee on Human Research Protections has specifically warned that AI and large-data research can expose gaps between formal regulatory compliance and adequate protection of participants. Its recommendations highlight not only identifiability but also group harms, profiling, bias, public trust, secondary uses, and the ability of AI systems to infer information that individuals did not explicitly disclose.

Legal Anonymity and Ethical Acceptability Are Different Questions

Suppose a dataset has been transformed so thoroughly that it qualifies as anonymous information under the applicable privacy law. That may have significant legal consequences. Under UK guidance, for example, genuinely anonymous information is not personal data and falls outside data-protection law.

It does not logically follow that every use of that information is ethically unobjectionable.

A researcher could still use anonymous information for a purpose that conflicts with what participants were told, disproportionately harms a community, reinforces discriminatory classifications, violates a data-use agreement, or undermines legitimate expectations surrounding the original research relationship.

The answers can overlap, but they are not interchangeable.

De-Identification Does Not Erase the History of the Data

A dataset does not appear from nowhere. Participants may have answered intimate interview questions, disclosed health information, allowed themselves to be recorded, completed assessments, or provided information because they understood that it would be used within a particular research project.

Removing identifiers later changes the data's privacy characteristics. It does not change how the information was originally obtained or what researchers represented to participants at the time.

This becomes particularly important when AI introduces a secondary use that was not contemplated when the study began. Whether existing consent covers an AI use that was not originally anticipated is not resolved merely by de-identifying the dataset first.

Participants May Care About Uses That Do Not Identify Them Individually

Imagine participants contributing data to research on educational inequality. Years later, a de-identified dataset is used to develop an AI system that predicts which demographic groups are likely to perform poorly academically.

No individual participant needs to be named for the new use to raise ethical questions. The model could reinforce stereotypes, produce classifications affecting a broader community, or contribute to decisions participants might reasonably find objectionable.

SACHRP has emphasized this problem in AI research, observing that potential harms may fall on groups and that existing human-subject protections, which often focus on identifiable individuals, may not adequately address profiling, bias, marginalization, and other collective effects.

AI Can Generate Information That Was Not Explicitly in the Dataset

Traditional confidentiality thinking often concentrates on preventing disclosure of information that researchers already possess. AI complicates that model because analysis can generate new classifications, predictions, or inferences.

A dataset may not explicitly state a participant's sensitive characteristic, yet patterns within the information may support an inference about that characteristic. At a group level, an AI system may derive relationships that create new risks even when no individual is named.

SACHRP has noted that combining large datasets can allow AI and machine-learning systems to infer information that individuals did not knowingly disclose. This challenges an ethical framework based exclusively on whether a particular record contains an obvious identifier.

Residual Re-Identification Risk Still Matters

Even when de-identification is ethically appropriate, the process may not eliminate identification risk completely. HHS guidance on HIPAA de-identification explicitly recognizes that properly de-identified health information can retain a very small but nonzero risk of identification.

With research data more generally, the relevant risk depends on what information remains, what external information is available, who receives the data, and what analytic capabilities can reasonably be applied.

This is particularly important for rich datasets containing detailed narratives, precise behavioral traces, genomic information, rare characteristics, images, or other information difficult to anonymize without destroying much of its analytical value.

The technical question of whether AI can re-identify participants from supposedly de-identified data should therefore remain part of the ethical risk assessment rather than being dismissed once obvious identifiers are removed.

Data Minimization Still Matters After De-Identification

De-identifying a dataset does not create a reason to upload all of it. Ethical data stewardship still favors using only the information necessary for the legitimate research purpose.

If an AI system needs three variables to perform a particular classification, supplying 40 additional variables “just in case” increases exposure without necessarily improving the research. Similarly, if an AI system only needs short excerpts, uploading complete interview histories may be difficult to justify.

The Philippine National Privacy Commission's AI guidance emphasizes privacy-by-design, governance mechanisms, security, fairness, and appropriate safeguards when AI systems process personal data. Its broader privacy framework also incorporates proportionality and purpose-limitation principles.

These principles offer a useful ethical discipline even where a particular dataset ultimately falls outside a privacy statute because it has been genuinely anonymized: disclose no more information than the research task reasonably requires.

Fairness and Bias Are Not Solved by Removing Identifiers

De-identification does not remove biases embedded in a dataset. A dataset can contain systematic underrepresentation, historical inequalities, measurement biases, or variables that function as proxies for sensitive characteristics.

An AI system can learn those patterns whether or not participant names are present.

The Philippine National Privacy Commission's AI guidance, for example, directs personal information controllers using AI systems to consider systemic, human, and statistical bias and to establish mechanisms to identify and limit their effects.

For researchers, the broader methodological point is equally important. Privacy protection and inferential validity solve different problems. A beautifully anonymized biased dataset remains a biased dataset. Peer reviewers may not applaud that distinction quite as enthusiastically as the privacy officer.

Ethics Review May Consider More Than Regulatory Identifiability

Researchers sometimes reason that if de-identification means a project no longer meets a regulatory definition of human-subject research, there can be no remaining participant-protection issue. That inference is too broad.

SACHRP's AI recommendations explicitly discuss the possibility that research can be compliant with existing regulatory exemptions while still failing to provide adequate protection against foreseeable harms. The committee argues that AI and large datasets expose limitations in traditional distinctions between identifiable and non-identifiable information.

Institutional ethics requirements may also extend beyond the minimum requirements of a particular statute. Researchers should therefore distinguish “this activity does not require a particular regulatory review” from “this activity presents no ethical concern.”

When Anonymous Data Can Still Raise an Ethical Problem

Where De-Identification Can Create False Ethical Confidence

Ask What Could Go Wrong Even if Identification Never Occurs

A useful ethical test is to imagine that your de-identification works perfectly. Assume, for the moment, that nobody will ever discover the identity of any participant. Would anything about the proposed AI use still concern you?

If the answer is yes, you have identified an ethical issue that de-identification cannot solve.

This approach avoids two extremes. De-identification should not be dismissed as useless merely because zero re-identification risk is difficult to guarantee. It can substantially improve participant protection. At the same time, researchers should not expect a technical privacy measure to answer questions that are fundamentally about purpose, autonomy, justice, and responsible research conduct.

There may also be situations where the remaining risk is disproportionate to what AI contributes. In those cases, the relevant question becomes whether using AI with participant data is worth the privacy risk at all.

Before Calling De-Identified AI Use Ethical

Questions About the Ethics of De-Identified Data and AI

Ethical AI Use Requires More Than Removing Identity

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.