03 · What You Need to Know
How Human Accountability Applies When AI Performs Research Actions
Why does accountability remain with humans?
Research is conducted within systems of professional responsibility. Researchers make claims about evidence, institutions establish governance requirements, and journals evaluate whether reported findings meet scholarly standards.
AI systems may assist with these activities, but they do not independently fulfill the responsibilities associated with research authorship, institutional compliance, or professional judgment.
The International Committee of Medical Journal Editors (ICMJE) states that AI-assisted technologies should not be listed as authors because they cannot take responsibility for the accuracy, integrity, and originality of submitted work.
Similarly, the Committee on Publication Ethics (COPE) emphasizes that human authors remain responsible for material produced using AI tools.
This does not mean researchers must manually perform every operation. It means that delegating a task does not automatically delegate accountability for the resulting research claims.
What is the difference between an AI error and research misconduct?
An AI system can produce incorrect outputs without a researcher intending to misrepresent evidence.
For example, AI-generated statistical code may incorrectly exclude observations, or an automated transcription system may introduce words that were never spoken.
These errors can compromise research quality and require correction.
However, they do not automatically establish formal research misconduct.
Under the United States Public Health Service framework, research misconduct involves fabrication, falsification, or plagiarism and requires the applicable standards concerning culpability, evidence, and departure from accepted practices to be satisfied.
The framework distinguishes misconduct from honest error and differences of opinion.
Responsibility for research accuracy
The obligation to verify, report, and correct research evidence and findings appropriately, including material produced through AI assistance.
Formal responsibility for research misconduct
A determination that an individual's conduct satisfies the applicable misconduct definition and evidentiary requirements.
A researcher may be responsible for correcting an AI-related error without necessarily having committed misconduct.
What if a researcher explicitly instructs AI to fabricate data?
When a researcher knowingly directs an AI system to invent observations and then reports those observations as genuine evidence, the involvement of AI does not change the underlying fabrication concern.
The United States Office of Research Integrity (ORI) defines fabrication as making up data or results and recording or reporting them.
For example, a researcher who asks AI to generate 50 fictional survey responses and reports them as responses from actual participants has misrepresented the empirical sample.
The researcher cannot reasonably treat the AI system as the responsible party merely because it produced the records.
The relevant conduct includes the decision to generate artificial evidence and the subsequent decision to represent it as genuine.
The distinction between AI-assisted and manual research data fabrication concerns the mechanism used, not a different standard of honesty.
What if AI performs a prohibited action without being instructed to do so?
This situation requires a more careful assessment.
An AI system may misunderstand instructions, generate incorrect code, or perform an unexpected action within an automated workflow.
For example, a researcher might ask an AI agent to clean a dataset, but the agent removes valid observations because it incorrectly interprets an exclusion rule.
The resulting analysis may be inaccurate even though the researcher did not intend the deletion.
Responsibility may involve examining the permissions granted to the system, the verification procedures used, and whether the researcher had reasonable opportunities to identify the problem.
However, the mere occurrence of an unexpected AI action does not automatically establish recklessness or misconduct.
Formal findings require consideration of the specific facts and governing standards.
Does human oversight mean checking every AI-generated output manually?
Not necessarily. Oversight should be proportionate to the risks and consequences of the task.
Routine formatting assistance may require relatively limited verification. An AI system modifying primary research data or generating statistical results may require substantially stronger controls.
The National Institute of Standards and Technology (NIST) AI Risk Management Framework emphasizes governance, measurement, and management of risks throughout AI system use.
In research, appropriate oversight may include automated tests, independent calculations, version control, access restrictions, and human approval before consequential actions.
The objective is to establish reliable control over the workflow rather than assume that manually reading every output is always sufficient.
Researchers should select verification methods appropriate to the scientific and operational risks involved.
Who is responsible when an AI agent acts autonomously?
Autonomous or agentic AI systems may execute code, access files, interact with external services, or perform sequences of actions without requiring approval at every step.
This creates additional governance questions because the person initiating the task may not directly control each intermediate operation.
Responsibility may be distributed among the researchers who designed the workflow, the individuals who authorized access, the institution responsible for system governance, and those who approve or report the resulting research.
However, distributed responsibility should not be confused with the absence of responsibility.
Researchers should establish what an AI agent is permitted to do, which actions require human approval, and how its activities will be audited.
The question of responsibility for errors made by autonomous AI research agents becomes particularly important when systems can modify evidence or perform irreversible actions.
Can institutions share responsibility for prohibited AI actions?
Potentially. Institutions may have responsibilities concerning approved tools, information security, research supervision, training, and the design of appropriate oversight procedures.
For example, an institution that deploys an AI system for research data processing may need to establish suitable access controls and procedures for validating consequential transformations.
However, institutional governance responsibilities are distinct from a formal misconduct finding against an individual researcher.
Different legal and institutional frameworks may allocate responsibility differently.
The existence of institutional shortcomings does not automatically excuse deliberate fabrication by a researcher, just as an individual researcher's error does not necessarily establish institutional wrongdoing.
A defensible assessment should examine the actual obligations and actions of each relevant party.
Can AI providers be responsible for research-related harms?
AI providers may have contractual, product-safety, privacy, or other legal responsibilities depending on the jurisdiction and circumstances.
For example, a provider may make representations about system capabilities, data handling, or security features that are relevant to institutional procurement and use.
However, potential provider responsibility does not automatically transfer responsibility for the scientific claims in a manuscript away from the researchers who submit it.
These are different forms of accountability.
A software defect may contribute to an incorrect analysis, while researchers remain responsible for accurately describing the methods and correcting findings when errors are discovered.
Legal liability and formal misconduct responsibility should be assessed separately rather than assumed to follow the same rules.
Why do intention, knowledge, and recklessness matter?
Formal misconduct frameworks commonly distinguish deliberate misrepresentation from honest error.
Under the United States Public Health Service rules, a misconduct finding requires evidence that the conduct was committed intentionally, knowingly, or recklessly, together with other applicable requirements.
Consider three situations involving AI-generated statistical results.
| Situation |
Accountability concern |
Appropriate assessment |
| A researcher knowingly reports invented AI-generated results. |
Deliberate misrepresentation of evidence. |
Potential fabrication requiring assessment under the applicable misconduct framework. |
| A researcher publishes results from AI-generated code containing an unnoticed error. |
Accuracy, verification, and possible negligence. |
Correction may be required; misconduct is not automatic. |
| A researcher ignores repeated warnings that AI has altered primary data. |
Possible reckless disregard of research accuracy. |
Requires examination of the warnings, conduct, and governing standards. |
These distinctions help prevent both excessive blame for ordinary errors and inappropriate excuses for deliberate misconduct.
Does disclosing AI use remove accountability?
No. Disclosure provides transparency about the role of AI, but it does not make prohibited conduct acceptable.
For example, a manuscript cannot legitimately report fabricated experimental observations merely by stating that AI generated them.
Artificial data may be appropriate for a clearly identified simulation study, but they cannot be presented as observations from experiments that never occurred.
Similarly, disclosing AI-assisted image manipulation does not justify altering scientific evidence in a misleading way.
The question of whether undisclosed AI use constitutes research misconduct is therefore distinct from whether the underlying AI-assisted activity was permissible.
Transparency is necessary in many contexts, but it is not a substitute for methodological integrity.
Can researchers delegate analytical decisions to AI?
AI may assist with selecting procedures, identifying coding errors, or suggesting interpretations, but consequential methodological decisions require appropriate justification.
For example, an AI system might recommend excluding outliers, transforming variables, or changing a statistical model.
These suggestions should be evaluated against the research design and relevant methodological principles.
Researchers should not accept analytical changes merely because the system presents them confidently or because they produce more favorable results.
Where AI materially influences the analysis, appropriate documentation may be necessary for transparency and reproducibility.
Human accountability requires maintaining a defensible relationship between the research question, analytical decisions, and reported findings.
What controls help preserve accountability in AI-assisted research?
Accountability becomes easier to establish when research workflows preserve evidence of what occurred.
Relevant controls may include access permissions, original-data preservation, version histories, processing logs, independent verification, and human approval checkpoints.
For AI agents, permissions should be limited to the activities necessary for the authorized task.
For example, an agent that only needs to analyze data may not require permission to overwrite the original dataset.
Similarly, an AI system preparing a manuscript may not need permission to submit it automatically to a journal.
The appropriate controls depend on the task and the consequences of an error.
The principle of human approval checkpoints for AI research agents is particularly relevant when actions could alter evidence, disclose confidential information, or become difficult to reverse.
What should happen after an AI system performs a prohibited action?
The immediate response should focus on preserving evidence, limiting further harm, and determining what occurred.
Researchers should avoid silently replacing files or deleting records in ways that obscure the incident.
Where appropriate, the system's access may need to be suspended while the affected data, outputs, and actions are examined.
If the incident compromises submitted or published findings, collaborators, institutional officials, or journal editors may need to be informed.
Where fabrication, falsification, plagiarism, or another serious violation is suspected, established research-integrity procedures should be followed.
Investigators should distinguish the need to correct an unreliable research record from the separate question of whether an individual committed misconduct.
The latter requires fair procedures and evidence satisfying the applicable standards.
