Who Should Evaluate AI-Specific Ethical Risks in a Research Study?

Who Decides Whether an AI Tool Is Ethically Appropriate for Your Research?

You are conducting a qualitative study involving university instructors and want to use generative AI to assist with interview analysis. Before proceeding, you ask your research ethics committee whether the proposed AI system is acceptable.

The committee asks how the provider stores and processes participant information. You contact the university's information technology office, which confirms that the service meets certain institutional security requirements.

However, the IT office cannot determine whether participants consented to this processing or whether AI-generated interpretations are methodologically appropriate.

Your data protection officer can advise on personal information processing, but may not be responsible for evaluating whether AI-generated categories accurately represent participants' experiences.

Each office appears to understand a different part of the problem. Yet none necessarily has complete responsibility or expertise for every aspect of the proposed AI use.

Who, then, should evaluate AI-specific ethical risks, and how should these responsibilities be coordinated without allowing important concerns to fall between institutional offices?

Evaluation May Require Several Specialists, but Responsibilities Must Be Clear

Understanding Responsibility for AI-Specific Research Ethics Evaluation

Why AI Research Risks May Require Different Forms of Expertise

Generative AI can affect several aspects of research simultaneously. A proposed system may process participant information, generate analytical interpretations, interact with participants, or introduce new technical infrastructure.

These activities raise different questions.

For example, determining whether an AI provider retains submitted transcripts requires knowledge of the service's contractual and technical arrangements. Determining whether the processing is consistent with participant consent requires examination of the approved research protocol and applicable ethical requirements.

Evaluating whether AI-generated qualitative themes are defensible requires methodological expertise.

These concerns overlap, but they should not be treated as though one type of expertise automatically establishes competence in every other area.

The NIST Artificial Intelligence Risk Management Framework (2023) emphasizes the importance of clearly defined roles, responsibilities, and multidisciplinary perspectives in AI risk governance.

For research institutions, this suggests that AI oversight may benefit from coordination among existing responsible offices rather than assuming that one individual can evaluate every relevant issue.

Who Has Responsibility for Which Risk?

The appropriate distribution of responsibilities depends on institutional structures, applicable laws, and the particular research activity.

Not every study requires consultation with every group. The objective is to obtain the expertise necessary for the actual risks, using the institution's established review procedures.

What Is the Principal Investigator's Responsibility?

The principal investigator generally has a central responsibility for ensuring that the study is conducted according to its approved procedures and applicable institutional requirements.

When introducing AI, this includes understanding what the proposed system will do, what information it will receive, and how its outputs will be used.

For example, a researcher should not simply state that ChatGPT will assist with thematic analysis without specifying whether complete transcripts, selected excerpts, or nonconfidential descriptions will be processed.

The research team should also identify relevant risks, verify factual claims about the AI service, and seek the necessary institutional guidance.

However, researcher responsibility should not be confused with unrestricted authority to approve one's own procedures.

Where an ethics amendment, data protection assessment, or institutional tool authorization is required, the researcher must follow that process.

Accountability involves recognizing when a decision requires expertise or authority beyond the research team.

What Should the Research Ethics Committee Evaluate?

A research ethics committee or institutional review board evaluates research according to its governing mandate and applicable ethical standards.

For AI-assisted research, relevant questions may concern participant welfare, informed consent, confidentiality, fair participant selection, and the adequacy of proposed safeguards.

For example, an AI chatbot used to interview participants may generate unexpected responses or ask questions that differ from those anticipated in the approved protocol.

The committee may need to consider whether participants understand the nature of the interaction and how foreseeable risks will be managed.

However, a committee may not independently possess the technical expertise needed to evaluate every AI system's architecture, security controls, or performance characteristics.

Obtaining specialized advice can support the committee's assessment without transferring its formal decision-making authority to the specialist.

Requirements for research ethics approval involving generative AI depend on the governing institutional framework.

Why the Data Protection Officer May Need to Be Involved

When AI processes personal information, the proposed workflow may raise questions concerning lawful processing, data minimization, retention, data sharing, and the roles of different organizations.

These matters may require advice from an institutional data protection officer or another qualified privacy specialist.

For example, a researcher may propose submitting pseudonymized interview transcripts to an external AI provider.

The data protection assessment may need to examine whether the information remains identifiable, whether the provider acts as a processor or controller under applicable law, and whether the relevant contractual safeguards are sufficient.

In the Philippines, the Data Privacy Act of 2012 and its implementing rules establish obligations concerning personal information processing and accountability.

The precise responsibilities of an institutional data protection officer depend on applicable law and organizational arrangements.

However, obtaining privacy advice does not automatically establish that the proposed analytical procedure is methodologically sound or ethically approved.

Why IT Approval Is Not the Same as Research Ethics Approval

Information technology and security personnel may evaluate whether a service meets institutional requirements for access control, encryption, authentication, logging, or other technical protections.

These evaluations are important, particularly when participant information will be processed outside the institution's existing systems.

However, a technically secure service may still be inappropriate for a particular research activity.

For example, an enterprise AI platform may meet institutional security requirements, but the proposed analysis may involve sensitive inferences that exceed the approved research purpose.

The IT office's approval of the platform does not establish that those inferences are necessary, valid, or consistent with participant consent.

Both may be necessary, but neither should be treated as a universal substitute for the other.

Who Evaluates the Validity of AI-Assisted Analysis?

Methodological suitability remains an important part of responsible AI use.

For example, a researcher may ask AI to classify instructors according to technological competence based on their interview responses.

Even if the data processing is technically secure and institutionally authorized, the classification may lack evidence of construct validity.

A methodological specialist may examine whether the intended construct has been adequately defined, whether the AI system's outputs are supported by appropriate evidence, and whether the analytical procedure is suitable for the research design.

In qualitative research, this may involve evaluating how AI-generated interpretations relate to the source material and the chosen analytical approach.

In quantitative research, it may involve examining predictive performance, measurement error, calibration, and potential differences across relevant groups.

Researchers should not assume that a favorable ethics review establishes the validity of every analytical technique used in the study.

Should AI Developers or Providers Evaluate Their Own Risks?

AI developers and providers may have important responsibilities for evaluating their systems' capabilities, limitations, security, and foreseeable harms.

They may provide documentation concerning data handling, model evaluations, known limitations, and available safeguards.

However, a provider's general safety assessment does not establish that its product is appropriate for every research context.

For example, a provider may demonstrate that its service meets certain security standards. That does not determine whether a particular university may upload identifiable student interviews under an approved protocol.

Researchers and institutions should evaluate the system in relation to the intended use and applicable requirements.

Provider documentation can inform that assessment, but it should not be treated as independent ethics approval for the study.

When Is Interdisciplinary Review Particularly Important?

Some AI-assisted activities involve overlapping risks that cannot be adequately evaluated from a single perspective.

For example, an AI chatbot used in an educational intervention may process student responses, provide individualized feedback, and generate information that affects participants' learning experiences.

The research ethics committee may evaluate consent and participant welfare. Technical specialists may examine system reliability and security. Educational researchers may evaluate whether the intervention is pedagogically appropriate and whether its outcomes are measured defensibly.

Data protection specialists may examine the processing of student information.

Coordination becomes especially important when a weakness in one area changes the implications of another.

For instance, unreliable AI feedback may be more ethically consequential when participants are encouraged to rely on it for important educational decisions.

Interdisciplinary review should therefore be proportionate to the complexity and consequences of the proposed AI use.

Do All AI-Assisted Studies Need a Special AI Ethics Committee?

No. A dedicated AI ethics committee is not universally required for every research activity involving generative AI.

Existing research ethics committees and institutional governance structures may be able to evaluate many proposed uses with appropriate specialist advice.

For example, using AI to improve the grammar of a nonconfidential research proposal may not warrant a specialized multidisciplinary review.

By contrast, an AI system that interacts with vulnerable participants or generates consequential classifications may require more extensive assessment.

The relevant issue is whether the institution has the authority and expertise necessary to evaluate the particular activity.

Creating another committee does not automatically improve oversight if responsibilities remain unclear or existing review processes are duplicated without adding meaningful expertise.

What Role Can Participants and Communities Play?

Researchers and institutional reviewers may overlook concerns that are important to the people whose information is being processed.

For example, participants may distinguish between analysis conducted by the university research team and processing performed through an external AI provider.

They may also have concerns about automated interpretation, possible misclassification, or the generation of sensitive characteristics.

Consulting participants or community representatives during research design may help identify such concerns and improve the clarity of consent materials.

However, participant consultation should not be treated as a substitute for required ethics approval or data protection assessment.

Nor should the preferences of a small consultation group automatically be attributed to every participant.

The relevance of participant expectations concerning AI use illustrates why ethical evaluation should consider perspectives beyond technical compliance.

Who Should Evaluate Potential Discrimination or Stigmatization?

When AI-generated classifications may affect particular groups, the assessment may require expertise in fairness, measurement, and the relevant social context.

For example, a system may classify instructors who do not use generative AI as resistant to innovation.

A methodological reviewer may question whether the classification is supported by the data. A research ethics reviewer may consider the potential consequences of stigmatizing participants or professional groups.

Where the classifications are used in consequential decisions, additional technical and institutional evaluation may be necessary.

The appropriate assessment depends on whether the concern involves an unsupported interpretation, systematic classification error, or an actual risk of unfair treatment.

Researchers should avoid assuming that a single fairness score establishes the absence of discrimination or stigmatization risks.

Who Should Evaluate Sensitive AI-Generated Inferences?

AI systems may generate characteristics that participants did not explicitly provide and that the original study did not intend to measure.

For example, a model may infer psychological characteristics from interview responses about teaching practices.

Evaluating such an activity may require methodological assessment of whether the inference is valid, ethical assessment of whether it is justified, and data protection assessment of whether the processing is authorized.

These questions should not be collapsed into a single judgment about whether the AI system can technically produce the classification.

Researchers should first determine whether the inferred information is necessary for the research purpose.

Where the inference is unnecessary or unsupported, avoiding its generation may be more appropriate than seeking additional procedures to justify it.

The issue is closely related to AI inference of unreported participant characteristics.

How Should Institutions Coordinate Different Reviews?

Multiple reviewers may be necessary, but unclear coordination can create gaps or duplicated work.

For example, an ethics committee may assume that the IT office has evaluated confidentiality, while the IT office assumes that the ethics committee has already authorized the proposed data processing.

Neither assumption should replace explicit communication about the scope of each determination.

Institutions may benefit from defining which office reviews particular risks, how specialist advice is communicated, and who has authority to approve or reject the proposed research activity.

A clear record of decisions can help distinguish technical recommendations, privacy assessments, methodological feedback, and formal ethics determinations.

The objective is coordinated oversight rather than transferring responsibility from one office to another without an explicit decision.

How Can Researchers Prepare an AI-Specific Risk Assessment for Review?

A useful submission should describe the proposed AI activity precisely enough for the appropriate specialists to identify their responsibilities.

Researchers should explain the system's function, information processed, participant involvement, intended outputs, and relevant safeguards.

They should also identify unresolved matters rather than presenting unverified assumptions as established facts.

The appropriate offices may differ across institutions. Researchers should follow their institution's established governance arrangements rather than assume that the same organizational structure exists everywhere.

How Should You Prompt AI to Identify the Appropriate Reviewers?

Generative AI can help organize the expertise that may be relevant to a proposed research workflow. However, it cannot determine the institution's actual delegation of authority without verified information.

This prompt may help researchers prepare a more focused consultation. The actual responsibilities and approval procedures must be verified with the institution.

Can AI Evaluate Its Own Ethical Suitability?

AI can identify possible risks associated with a proposed task and suggest questions for further review.

However, asking the same system whether it is safe or ethically appropriate to process participant data does not constitute independent evaluation.

The model may lack current information about the applicable service configuration, institutional agreements, or research protocol.

It may also provide an incomplete assessment or present general safeguards as though they apply to the specific account being used.

Researchers should rely on verified provider documentation and the appropriate institutional authorities for consequential decisions.

AI-assisted risk screening may be useful preparation, but it should not become a substitute for the review process it is supposed to support.

What If Different Reviewers Reach Different Conclusions?

Different assessments may reflect distinct mandates rather than an actual contradiction.

For example, an IT office may determine that a service meets security requirements, while a research ethics committee concludes that the proposed participant-facing activity requires additional safeguards.

Both conclusions may be valid within their respective areas of responsibility.

Researchers should identify the precise issue addressed by each determination and whether all required conditions have been satisfied.

Where recommendations conflict, the matter should be referred through the institution's established governance process rather than resolved by selecting whichever opinion is most convenient.

The researcher should not treat one favorable assessment as permission to disregard an unresolved requirement in another area.

Who Is Responsible if AI Causes an Ethical Problem?

Responsibility may be distributed among researchers, institutions, service providers, and other organizations according to their respective roles and applicable obligations.

For example, a provider may be responsible for meeting contractual security requirements, while the research institution may be responsible for authorizing the processing arrangement.

The research team remains responsible for conducting the study according to approved procedures and verifying the analytical claims it reports.

However, accountability should not be reduced to assigning blame after a problem occurs.

Clear responsibilities before implementation can help prevent gaps in oversight and establish procedures for responding when unexpected events arise.

The NIST AI Risk Management Framework emphasizes governance structures that clarify responsibilities and support ongoing risk management across the AI lifecycle.

When Should the Final Decision Be Not to Use AI?

Sometimes the necessary reviewers may identify risks that cannot be adequately managed within the proposed arrangement.

For example, the available provider may not be authorized to process sensitive participant information, or the intended AI-generated classifications may lack sufficient validity evidence for their proposed use.

In such circumstances, additional consultation should not become an exercise in finding someone willing to approve the activity.

The researcher may need to redesign the workflow, use an authorized alternative, or proceed without AI.

Appropriate oversight includes recognizing when the ethical decision is not to use AI.

Coordinating an Ethics Committee, Data Protection Officer, and Methodological Reviewer

The example illustrates why one favorable assessment does not necessarily settle every issue.

A technically secure AI system may still be unsuitable for the proposed analysis, while a methodologically promising analytical procedure may still require authorization to process participant information.

Common Misconceptions About Responsibility for AI Research Ethics

Match Each Ethical Question With the Appropriate Expertise and Authority

When planning AI-assisted research, identify the specific concerns created by the proposed activity before deciding which institutional offices need to be involved.

Do not assume that every office must review every minor AI task. Equally, do not rely on one favorable technical or administrative decision to settle matters outside that office's authority.

Maintain a record of the relevant assessments, their scope, the conditions imposed, and the authority responsible for the final determination.

The purpose of multidisciplinary review is not to distribute responsibility so widely that no one remains accountable. It is to ensure that the relevant questions are evaluated by people with the appropriate expertise and institutional authority.

Before Seeking Approval for AI-Assisted Research Procedures

Frequently Asked Questions About Evaluating AI-Specific Research Ethical Risks

AI Research Ethics Requires the Right Expertise, Not Just Another Approval

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.