What Ethical Issues Can Generative AI Create in Research?

Can Using Generative AI Create Ethical Problems Even When the Research Itself Is Legitimate?

You are conducting a qualitative study involving university instructors. You have obtained the necessary institutional approval, recruited participants, and collected interview data in accordance with your research protocol.

To save time, you decide to upload the interview transcripts to ChatGPT and ask it to identify preliminary themes. The transcripts contain participants' professional experiences, descriptions of institutional disagreements, and occasional references to colleagues.

The AI system produces a useful thematic summary. Nothing appears obviously wrong with the analysis, and the researcher has not intentionally disclosed the information to another person.

However, an important ethical question has emerged. Were participants informed that their responses might be processed by an external AI provider? Does the service retain submitted information? Could the transcripts reveal participants' identities even after names are removed? And does the approved research protocol cover this form of data processing?

These questions illustrate why ethical concerns involving generative AI extend beyond whether the technology produces accurate answers or whether its use constitutes academic misconduct.

AI Can Introduce Ethical Risks Across the Research Process

Understanding the Ethical Issues of Generative AI in Research

Research Ethics and Research Integrity Are Related but Distinct

Discussions about ethical AI use in research often focus on plagiarism, fabricated citations, undisclosed AI assistance, or inaccurate AI-generated content.

These are important research integrity concerns. However, research ethics also considers how the research affects participants, communities, and other people whose information or interests may be involved.

For example, a researcher might use AI to analyze interview transcripts accurately and report the results transparently. Nevertheless, uploading those transcripts to an unauthorized external service could raise confidentiality and data-protection concerns.

Conversely, a researcher might use AI without processing any participant data but include fabricated references or unsupported findings in a manuscript. That would raise research integrity concerns even if no participant information was disclosed.

These responsibilities can overlap. A fabricated AI-generated interpretation may undermine research integrity while also causing harm if it inaccurately characterizes a participant or community.

What Ethical Issues Should Researchers Examine?

The ethical implications of generative AI vary according to the research activity. A useful starting point is to identify what the AI system receives, what it produces, and how its outputs will be used.

UNESCO's Guidance for Generative AI in Education and Research (2023) emphasizes privacy protection, human agency, and responsible governance. The World Health Organization's guidance on large multimodal models (2024) also identifies risks involving inaccurate outputs, bias, automation bias, and information security. Although the WHO guidance addresses health applications, several of its concerns are relevant to other research settings. UNESCO guidance and WHO guidance provide further discussion.

Why Does Uploading Research Data to AI Raise Confidentiality Concerns?

When researchers enter information into an external AI service, they may be transferring data to an additional organization or processing environment.

Whether this constitutes a disclosure to a separate recipient, processing by a service provider, or another legally defined arrangement depends on the provider's contractual role, technical architecture, and applicable law.

Researchers should not assume that a private-looking chat interface means the submitted information remains exclusively within their institution.

Relevant questions include whether the provider stores prompts, uses information for model improvement, permits human access under specified circumstances, transfers data internationally, or allows deletion under the applicable service arrangement.

These conditions vary across products, subscription types, enterprise agreements, and service configurations. They should be verified rather than inferred from the general name of an AI platform.

The issue becomes particularly important when determining whether an AI provider becomes an additional recipient of research data.

Removing Participant Names May Not Be Enough

Researchers sometimes assume that replacing names with participant codes makes interview transcripts safe to upload.

However, participants may remain identifiable through combinations of contextual details.

For example, a transcript might describe a university administrator's position, department, years of service, and involvement in a particular institutional dispute. Even without a name, those details may identify the individual to someone familiar with the institution.

This is particularly relevant in qualitative research involving small organizations, specialized professions, or distinctive experiences.

Pseudonymization reduces direct identification but does not necessarily make data anonymous.

Researchers should evaluate the risk of deductive identification and consider whether the information is necessary for the intended AI-assisted task.

Can AI Use Create Problems With Informed Consent?

Informed consent concerns more than obtaining a participant's signature. It involves providing information relevant to the person's decision to participate and explaining how their information will be used and protected.

Suppose participants consented to interviews being transcribed and analyzed by the research team. The researcher later decides to submit the transcripts to an external generative AI service.

Whether that change is permitted depends on the consent arrangements, approved protocol, institutional policies, and applicable data-protection requirements.

Researchers should not assume that general permission to analyze data automatically authorizes every subsequent processing method.

At the same time, not every use of AI necessarily requires a separate consent form. The relevant question is whether the proposed activity is appropriately covered by the existing arrangements and whether additional review or participant information is required.

This is closely connected to participant expectations about AI use, which may extend beyond the minimum conditions of technical or legal permission.

Does Every Use of AI Require Research Ethics Approval?

No. The use of generative AI does not, by itself, establish that an activity requires ethics committee approval.

For example, using AI to brainstorm a nonconfidential research title or improve the grammar of a general methodological description may not materially change the ethical characteristics of a study.

However, using AI to collect participant information, analyze identifiable records, infer sensitive characteristics, or influence decisions affecting participants may raise questions that require institutional assessment.

Requirements differ across institutions, jurisdictions, research designs, and funding arrangements.

Researchers should distinguish whether AI use requires ethics approval from whether the use must comply with existing institutional policies or data-protection obligations.

Where the requirement is uncertain, the appropriate institutional ethics office or review body should be consulted rather than relying on AI to determine its own regulatory status.

Can AI Change the Ethical Risk of an Already Approved Study?

Yes. Ethical risk is not necessarily fixed at the moment a research protocol receives approval.

Suppose a study was approved for qualitative analysis conducted by the research team using institutionally managed software. The researcher subsequently proposes using an external AI service to analyze complete interview transcripts.

The new arrangement may introduce different data recipients, retention conditions, security risks, or analytical procedures.

Whether these changes require an amendment, notification, or additional review depends on the applicable institutional requirements and the significance of the proposed changes.

Researchers should not assume that an approved study authorizes every later technological modification.

When considering AI use after ethics approval, the researcher should evaluate the proposed change before implementing it and follow the institution's amendment procedures where applicable.

AI Can Produce Inaccurate Interpretations That Affect Participants

Generative AI can produce fluent interpretations that are not adequately supported by the research data.

For example, a participant might say, "I rarely use generative AI because I have not found a suitable reason to incorporate it into my teaching."

An AI system might summarize the statement as evidence of resistance to technological change or anxiety about AI.

Those interpretations may be plausible in another context, but they are not necessarily supported by the participant's words.

If the researcher accepts the interpretation without checking the transcript, the final report may misrepresent the participant's experience.

The consequences can be especially serious when findings concern identifiable professional groups, institutional practices, or socially sensitive experiences.

Researchers should therefore verify AI-assisted interpretations against the source material and their chosen analytical methodology.

How Can AI Introduce Discrimination or Stigmatization?

AI-generated classifications may reproduce stereotypes or associate particular behaviors with unsupported characteristics.

For example, an AI system might characterize instructors who do not use generative AI as technologically resistant, professionally outdated, or lacking digital competence.

Such descriptions could stigmatize participants when their decisions may reflect institutional restrictions, pedagogical considerations, limited access, or other circumstances.

The ethical concern is not limited to whether the classification is statistically accurate. Researchers should also examine whether the categories are conceptually justified and whether their use could unfairly characterize individuals or communities.

The WHO's guidance on large multimodal models identifies biased and incomplete outputs as important risks and cautions against automation bias, where users may overlook errors because they rely on an AI system.

Researchers should evaluate the potential for discrimination or stigmatization before using AI-generated classifications in analysis or reporting.

Can AI Infer Sensitive Information That Participants Never Disclosed?

Generative AI may infer characteristics from indirect information, even when those characteristics were not explicitly collected.

For example, a researcher might ask AI to analyze interview transcripts for communication patterns. The model may also suggest interpretations concerning participants' emotional states, personal circumstances, or other sensitive attributes.

These inferences may be unreliable, particularly when the system lacks sufficient context or treats linguistic patterns as evidence of personal characteristics.

Even if an inference appears plausible, the researcher must consider whether generating it is necessary, justified, and consistent with the study's approved purpose.

The fact that AI can generate an inference does not establish that researchers are ethically entitled to collect or use it.

Researchers should distinguish AI-generated sensitive inferences from information participants knowingly provided and avoid treating speculative classifications as verified personal facts.

Data Minimization Still Applies When AI Makes Large-Scale Analysis Easy

Generative AI makes it convenient to process extensive documents and datasets. However, technical convenience does not justify submitting every available piece of information.

Suppose a researcher wants AI to identify possible categories in participants' descriptions of instructional practices. Uploading complete institutional personnel records alongside the interviews would likely require a separate justification.

A more limited dataset may be sufficient for the intended task.

Data minimization generally concerns limiting personal data processing to what is necessary for specified purposes, subject to the applicable legal and institutional framework.

Researchers should examine whether AI-assisted analysis is consistent with data minimization and whether a less intrusive approach could achieve the same research objective.

Removing unnecessary identifiers, limiting submitted excerpts, or using an appropriately controlled processing environment may reduce certain risks, although none of these measures automatically establishes ethical acceptability.

What Research Integrity Problems Can AI Create?

Generative AI can produce fabricated references, inaccurate summaries, unsupported interpretations, or content that appears to represent evidence that was never collected.

For example, a researcher might ask AI to generate illustrative interview quotations and inadvertently include them in a results section as though they came from actual participants.

Another researcher might accept AI-generated citations without checking whether the referenced publications exist or support the claims made.

These practices can compromise the integrity of the research record.

The International Committee of Medical Journal Editors (ICMJE) recommends that authors disclose AI-assisted technologies used in submitted work and emphasizes that human authors remain responsible for accuracy, originality, and integrity. Its guidance also cautions against confidentiality violations when privileged manuscripts are submitted to AI systems. ICMJE recommendations provide the relevant publication guidance.

Researchers should distinguish illustrative AI-generated material from genuine empirical evidence and verify every substantive claim before reporting it.

Who Is Accountable When AI Produces an Incorrect Research Finding?

AI assistance does not eliminate the researcher's responsibility for methodological decisions and reported conclusions.

Suppose an AI system incorrectly classifies several interview excerpts and the researcher reports those classifications without reviewing the original transcripts.

The resulting error cannot be adequately explained by stating that the AI tool generated the analysis.

Researchers remain responsible for determining whether the analytical procedure is appropriate, whether the outputs are supported by evidence, and whether the findings are reported accurately.

Accountability may also involve institutions, service providers, research teams, and oversight bodies, depending on their respective roles and obligations.

However, these shared responsibilities do not justify treating the AI system as an autonomous researcher capable of assuming responsibility for the study.

How Should Researchers Evaluate AI-Specific Ethical Risks?

A useful ethical assessment begins with the actual AI-assisted activity rather than a general judgment about whether AI is good or bad for research.

Researchers should identify what information will be processed, why AI is necessary, which system will be used, what safeguards are available, and how the resulting outputs may affect participants or the integrity of the study.

Such a prompt may help organize preliminary considerations, but it cannot replace institutional ethics review, legal assessment, or verification of the provider's actual data-handling arrangements.

Researchers should involve the appropriate ethics committee, data protection personnel, institutional information security specialists, or other qualified reviewers when the proposed activity warrants their expertise.

When Might the Ethical Decision Be Not to Use AI?

Researchers should not assume that every research activity benefits from AI assistance or that all risks can be adequately managed through disclosure.

For example, a researcher may lack authorization to transfer sensitive participant transcripts to an external provider. The available service may not offer suitable contractual protections, or the proposed AI analysis may require unnecessary processing of personal information.

In such circumstances, avoiding the proposed AI tool may be more appropriate than proceeding with inadequate safeguards.

Researchers may instead use authorized institutional systems, conduct the analysis without AI, or reconsider the proposed procedure.

The decision should reflect the research purpose, available alternatives, expected benefits, and residual risks.

Sometimes the ethically preferable decision is not to use AI, particularly when the potential benefit does not justify exposing participants or research information to additional risks.

When AI-Assisted Interview Analysis Creates an Unexpected Ethical Problem

The ethical concern is not that thematic analysis using AI is inherently unacceptable. It is that introducing a new processing method may change who handles the data, how information is protected, and whether the activity remains consistent with participants' consent and the approved research protocol.

Even if the AI-generated themes appear accurate, the researcher must separately address the ethical conditions under which the analysis was conducted.

Common Misconceptions About Generative AI Research Ethics

Evaluate the Ethical Implications of the Specific AI Activity

Before incorporating generative AI into a study, identify exactly what the tool will do and what information it will receive. The ethical implications of drafting interview questions differ from those of analyzing identifiable transcripts or inferring participant characteristics.

Researchers should evaluate whether the proposed activity is necessary, consistent with participant expectations, appropriately authorized, and supported by suitable technical and organizational safeguards.

Maintain documentation of the AI system, purpose of use, information processed, relevant approvals, safeguards, and significant methodological decisions.

Ethical evaluation should continue when the research procedure changes. A tool that was appropriate for drafting questions may not be appropriate for processing confidential participant data.

Before Using Generative AI in a Research Study

Frequently Asked Questions About the Ethical Issues of Generative AI in Research

Ethical AI Use Depends on What Researchers Do, Not Merely Which Tool They Use

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.