Manuel B. Garcia

Manuel B. Garcia serves as the Senior Director for Educational Technology and Digital Learning at FEU Institute of Technology, Manila, Philippines. Read More

Contact Info

1607, FEU Tech Building,
P. Paredes St, Sampaloc,
Manila, Philippines
mbgarcia@feutech.edu.ph

Follow Me

When Is Using AI With Participant Data Not Worth the Privacy Risk?

AI should not be used with participant data merely because it makes a research task faster. When the same purpose can be achieved with less sensitive data, an approved conventional method, or a safer processing environment, the additional privacy exposure may be difficult to justify.

128
When AI Is Not Worth the Privacy Risk Guide 128 of 548
01 · The Question

When Should You Simply Decide Not to Use AI With Participant Data?

Much of the discussion about AI and research data asks how to make the use permissible: de-identify the dataset, choose an enterprise service, obtain an amendment, sign a processing agreement, restrict retention, or move to a safer hosting environment.

There is another option.

Sometimes AI does not add enough value to justify sending participant information through an additional processing system at all.

The question is not whether AI can perform the task. It is whether the expected research benefit is large enough to justify the additional privacy, confidentiality, methodological, governance, and compliance burden created by using it.

02 · The Short Answer

Do Not Introduce Participant-Data Risk for Marginal Convenience

In Brief

Using AI with participant data is usually difficult to justify when the research task can be accomplished adequately without disclosing participant information to the AI system, when the expected benefit is mainly convenience, when the data are highly sensitive or difficult to de-identify, when the provider's data practices cannot be adequately controlled, or when a safer alternative provides comparable research value.

The appropriate decision is contextual rather than categorical. AI can sometimes provide substantial scientific or operational benefit, but data minimization asks a prior question: what is the least participant information, and the least risky processing arrangement, reasonably necessary to accomplish the legitimate research purpose?

03 · What You Need to Know

Privacy Risk Should Buy You Something Methodologically Useful

“AI Would Be Faster” Is a Benefit, but Not Always a Strong One

Efficiency matters in research. Automated transcription can save days. AI-assisted coding can help researchers explore large qualitative datasets. Automated extraction can make thousands of clinical records tractable.

But convenience should be weighed against what the workflow requires from participants' data.

If AI saves ten minutes while requiring identifiable interview transcripts to leave an approved environment, the trade-off looks different from a carefully governed AI system that makes analysis of a dataset possible when manual processing would be practically infeasible.

There is no universal numerical threshold. The principle is proportionality: greater exposure should require a correspondingly meaningful research justification.

Ask Whether Participant Data Are Necessary for the AI Task at All

Many researchers disclose real data when the AI does not actually need them.

If you want help writing R or Python code, provide variable names and synthetic data. If you want feedback on a coding framework, create fictional excerpts. If you need assistance interpreting a statistical method, describe the design rather than uploading the participant dataset. If you want help drafting a table shell, provide the structure rather than the results file.

This is often the easiest privacy intervention because it eliminates the disclosure rather than trying to secure it.

AI requires the participant data The research task depends on information contained in actual participant records, and a meaningful analytical benefit cannot reasonably be achieved with a lower-risk representation.
AI requires information about the research The researcher needs methodological, coding, writing, or technical assistance but can obtain it without transmitting actual participant records.

Confusing those two situations causes a surprising amount of unnecessary exposure.

Use the Least Sensitive Representation That Can Answer the Question

Even when actual research information is needed, the full raw dataset may not be.

A task might work with:

  • aggregate statistics instead of participant-level records;
  • selected variables instead of the entire dataset;
  • de-identified excerpts instead of complete transcripts;
  • transcripts instead of original audio recordings;
  • cropped or transformed images instead of identifiable originals;
  • broad categories instead of precise locations or ages; or
  • a small authorized sample rather than every participant record.

This is the practical logic behind data minimization. Under the Philippine Data Privacy Act, for example, proportionality requires personal information to be adequate and not excessive in relation to the purposes for which it is processed. Comparable minimization principles appear in other data-protection frameworks.

Sensitive Data Raise the Price of a Small Benefit

The consequences of disclosure should influence how much benefit is required to justify AI processing.

A dataset containing broad, non-sensitive questionnaire responses may permit a relatively low-risk AI workflow after appropriate safeguards. Identifiable trauma interviews, genomic information, medical histories, illegal activity, sexual information, facial images, or recordings from vulnerable participants create a different risk profile.

The more serious the potential consequences of unauthorized disclosure or re-identification, the less persuasive “it will make analysis easier” becomes.

This is particularly important with clinical research data and other records in which both identifiability and sensitivity can be high.

Some Data Are Difficult to De-Identify Without Destroying Their Research Value

De-identification works well for some datasets and poorly for others.

Detailed qualitative narratives derive much of their value from context. Video research may depend on faces, gestures, movement, and interaction. Voice research may require acoustic characteristics. Rare-disease research may involve small populations whose characteristics are inherently distinctive.

If removing identifying information destroys the phenomenon you need to analyze, do not pretend the transformed dataset solves the problem. Either use an environment authorized for identifiable data or reconsider whether AI belongs in the workflow.

An Approved Alternative With Comparable Value Weakens the Case for Riskier AI

Suppose an institution already provides a secure transcription service approved for confidential research recordings. A researcher nevertheless prefers a public generative AI service because its interface is slightly faster.

If the scientific outcome is essentially the same, the additional privacy exposure of the less controlled service may have little justification.

Risk-benefit reasoning should compare realistic alternatives, not compare AI with doing nothing.

Local or Institution-Controlled AI Can Change the Trade-Off

Sometimes the problem is not AI itself but the architecture through which it is accessed.

A model running locally or inside an institutionally controlled environment may allow researchers to obtain AI assistance without transmitting participant information to an external provider. Depending on the implementation, this can reduce concerns about third-party processing, international transfer, retention, and provider reuse.

Local deployment still requires security, access controls, ethics compatibility, and methodological validation. It simply changes several components of the privacy equation.

De-Identification Should Reduce Risk, Not Serve as a Ritual

If participant data can be effectively de-identified while retaining the information needed for analysis, AI processing may become substantially easier to justify.

But de-identified data are not automatically anonymous, and removing names from a rich dataset may leave considerable residual identification risk.

Researchers should therefore ask how much the transformation actually reduces realistic risk rather than whether a de-identification step appears somewhere in the workflow.

Unclear Provider Practices Are Themselves a Reason to Stop

If you cannot determine whether the provider retains participant information, uses it for model training, permits deletion, employs subprocessors, or processes data in other jurisdictions, you cannot meaningfully assess the exposure.

Uncertainty should not be silently converted into an assumption that the provider probably handles data safely.

For low-risk public information, uncertainty may be tolerable. For confidential participant data, the inability to establish basic processing conditions can itself make the service unsuitable.

International Processing Can Add Burden Without Adding Scientific Value

If using an AI service introduces cross-border transfer assessments, contractual safeguards, data-residency review, and additional institutional approvals, ask whether the research actually benefits enough to justify those steps.

There may be good reasons to proceed. There may also be an institutionally approved alternative that avoids the issue entirely.

Where location matters, determine where the AI provider actually processes participant data before deciding whether the governance burden is worthwhile.

AI's Methodological Weaknesses Belong in the Risk-Benefit Calculation Too

Privacy exposure might be justified if AI produces a meaningful analytical advantage. That justification becomes weaker if the method itself is poorly validated.

A generative model that inconsistently codes interviews, invents clinical details, misclassifies images, or produces results that cannot be adequately validated may create both privacy risk and questionable evidence.

The appropriate comparison is therefore not:

AI benefit versus privacy risk.

It is closer to:

validated research benefit from this AI workflow versus the privacy, confidentiality, methodological, governance, and opportunity costs it introduces.

That formulation is less catchy. Research methods occasionally have this unfortunate relationship with accuracy.

Participant Expectations Can Matter Even When Formal Permission Exists

A technically permissible AI use may still be difficult to defend if it departs sharply from the context in which participants contributed highly sensitive information.

SACHRP's recommendations on AI research emphasize gaps that can arise between regulatory compliance and adequate protection of participants, particularly with de-identified data, secondary uses, inferred information, bias, and group harms.

That is why legal or technical de-identification does not settle the entire ethical question.

“Can We?” and “Should We?” Need Separate Answers

An institution may approve an AI service. The provider may offer strong contractual safeguards. Participants may have consented appropriately. The data may be adequately protected.

You can still decide that AI contributes too little to justify introducing it.

Good research governance is not a contest to find the maximum amount of technically permissible data processing. Sometimes the best privacy control is simply not creating the additional data flow.

Watch Out

Do not treat institutional approval of an AI platform as a recommendation to upload participant data. Approval means a service may be available under specified conditions. You still need a research reason for using participant information with it.

04 · A Practical Example

When AI Convenience Does Not Justify the Data Exposure

Hypothetical Example

Summarizing Twenty Sensitive Interviews

A researcher has 20 detailed interviews with survivors of workplace harassment. The transcripts contain sensitive narratives, distinctive incidents, job roles, locations, and information about alleged perpetrators.

Proposed AI benefit A generative AI system could summarize each transcript and save the researcher several hours of initial reading.
Privacy cost The complete transcripts would need extensive de-identification or processing in an environment authorized for highly sensitive participant data. Contextual re-identification remains a concern.
Methodological limitation The researcher must read the original transcripts closely anyway to conduct the planned interpretive analysis and verify any AI summaries.
Alternative The researcher performs the initial reading manually and uses AI later with synthetic examples to refine a coding framework without disclosing participant narratives.
Decision The modest time saving does not justify adding an external processing pathway for highly sensitive transcripts, so participant data are kept out of the AI system.

AI was technically capable of doing the task. That did not make it the best research workflow.

05 · What Researchers Often Get Wrong

Misunderstandings About AI Risk and Research Benefit

Misconception

“If AI Is Institutionally Approved, I Should Use It”

Approval establishes that a service may be permissible under specified conditions. It does not establish that your particular research task needs AI or that participant-data exposure is justified.

Misconception

“Any Time Saving Justifies Some Additional Privacy Risk”

Efficiency is a legitimate benefit, but the proportionality of the trade-off depends on how much time is saved, how sensitive the data are, what additional exposure occurs, and whether safer alternatives provide comparable value.

Misconception

“The Choice Is AI or Manual Work”

Often there are intermediate options: synthetic data, aggregate data, selected excerpts, locally hosted models, approved transcription services, traditional analytical software, or AI used only for tasks that do not require participant information.

Misconception

“De-Identification Makes the Privacy Cost Zero”

De-identification can substantially reduce risk but may leave residual identifiability, particularly in detailed or unusual datasets. Other ethical, contractual, or institutional restrictions may also remain.

Misconception

“More AI Means a More Advanced Methodology”

Methodological sophistication is not measured by the number of AI systems in the workflow. A simpler method that answers the research question validly while exposing less participant information may be the stronger design.

06 · What This Means for You

Use AI Only When Its Added Value Survives a Privacy Test

A useful decision begins by imagining the AI option is unavailable. How would you accomplish the task? Then compare that alternative with the AI workflow, including both scientific benefits and participant-data costs.

A simple decision framework

If AI can perform the task without participant data
Keep participant data out of the AI system and provide only the methodological or synthetic information required.
If aggregate or minimized data provide the same analytical value
Use the lower-risk representation rather than the raw participant dataset.
If participant data are highly sensitive and AI provides only modest convenience
Prefer a safer conventional workflow unless a stronger research justification emerges.
If AI provides substantial research value that cannot reasonably be obtained otherwise
Use the least risky authorized architecture and the minimum participant information necessary.
If provider practices, approval, consent, or data flows remain materially uncertain
Do not use participant data with the system until those uncertainties are resolved.

The best AI workflow is sometimes the one that never receives participant data. That does not make the research less technologically sophisticated. It means the technology has been placed where its benefit is greatest and its unnecessary exposure is smallest.

07 · A Quick Checklist

Before Deciding AI Is Worth the Participant-Data Risk

Ask whether the additional processing is genuinely justified:
Define the specific research benefit AI provides rather than relying on general claims of efficiency or innovation.
Determine whether the task can be completed using synthetic, aggregate, public, or otherwise non-participant data.
If participant data are necessary, identify the minimum records, variables, excerpts, or media required.
Assess both the likelihood of identification or unauthorized disclosure and the severity of harm if it occurs.
Compare the AI workflow with realistic lower-risk alternatives rather than with doing nothing.
Consider whether local, institution-controlled, or otherwise approved AI can provide similar benefits with less external exposure.
Verify that the AI method itself is sufficiently valid for the intended research task to justify its privacy and governance costs.
Do not proceed when provider practices or authorization remain too uncertain to assess the risk responsibly.
08 · Frequently Asked Questions

Questions About Whether AI Is Worth the Privacy Risk

Should I avoid AI completely when research involves participant data?

No. AI can sometimes provide substantial research value within appropriately governed environments. The decision should depend on necessity, benefit, sensitivity, safeguards, alternatives, and methodological validity rather than a blanket prohibition.

Is saving researcher time enough to justify AI processing?

Efficiency is a legitimate benefit, but its weight depends on magnitude and context. Saving substantial effort on a low-risk, well-governed task may be reasonable. Saving a few minutes by exposing highly sensitive identifiable data may not be.

What if AI is much better than the alternative?

A substantial scientific or operational advantage strengthens the justification, but it does not eliminate privacy obligations. Use the least sensitive data and most controlled authorized environment capable of delivering that benefit.

Does using de-identified data make AI automatically worth using?

No. De-identification changes the risk side of the equation, not the benefit side. If AI adds little research value, there may still be no reason to introduce the additional processing.

Is local AI always safer than cloud AI?

Not automatically. Local processing can reduce external disclosure and cross-border concerns, but security, access controls, model configuration, device protection, research approval, and methodological reliability still matter.

Can I use AI with synthetic data instead?

Often, especially for developing code, testing prompts, designing workflows, troubleshooting analyses, or creating examples. Synthetic data may not substitute for real observations when the AI itself is performing the substantive participant-level analysis, so match the data to the actual task.

What is the simplest rule for deciding?

Ask whether the AI needs the participant data, whether a lower-risk alternative can achieve comparable research value, and whether the remaining benefit is substantial enough to justify the exposure and governance burden. If the first answer is no, keep the participant data out.

09 · The Bottom Line

AI Should Earn Its Place in a Participant-Data Workflow

The Bottom Line

AI is not worth the privacy risk when participant data are unnecessary for the task, when a lower-risk method provides comparable research value, or when the expected benefit is too small to justify the sensitivity, disclosure, uncertainty, and governance burden introduced by the AI workflow.

Start with necessity rather than technological possibility. Use the minimum participant information required, compare realistic alternatives, and remember that declining to create an unnecessary data flow can be a stronger research decision than finding increasingly elaborate safeguards for one you did not need.

10 · Sources and Further Reading

Authoritative Sources on AI, Data Minimization, and Research Risk

11 · Cite this Guide

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.

Has the Field Guide helped your research?

If a guide helped clarify a question, inform a research decision, or move your work forward, I would love to hear about your experience. Your story may also help other researchers discover the Field Guide.

Share Your Experience
Takes only a few minutes