03 · What You Need to Know
Protecting Research Data Integrity When AI Agents Can Modify Files
What does research data modification mean?
Research data modification refers to changing the content, structure, organization, or representation of information collected or generated during a study.
Examples include correcting data-entry errors, standardizing variable names, converting date formats, removing confirmed duplicate records, and deriving new analytical variables.
Some modifications preserve the underlying meaning of the observations. Others may change which observations are included or how they are interpreted.
For example, converting a date from one standardized format to another may be a routine technical transformation.
Deleting a participant's response because it appears inconsistent with the expected hypothesis is a fundamentally different action.
Researchers should classify proposed modifications according to their scientific consequences rather than treating every file edit as equivalent.
What is the difference between raw data and processed data?
Raw research data
The original observations or source records preserved in the form established by the study's data collection and recording procedures, subject to documented corrections and applicable data governance requirements.
Processed research data
Data derived from original records through documented cleaning, coding, transformation, aggregation, or other analytical preparation procedures.
Processed data are not inherently less valid than raw data. Many analyses require transformations before statistical or qualitative interpretation is possible.
However, researchers should preserve sufficient provenance to establish how the processed dataset relates to the original observations.
An AI agent should not silently replace the source record with a modified version that cannot be reconstructed or independently checked.
Which data modifications may be suitable for AI agents?
Some operations are relatively structured and can be validated through deterministic rules.
| Modification |
Potentially appropriate agent action |
Required consideration |
| Formatting |
Standardize dates, capitalization, and field formats |
Preserve the original values and intended meaning |
| Variable naming |
Rename columns according to an approved schema |
Maintain a mapping between original and revised names |
| Category normalization |
Apply predefined mappings to equivalent labels |
Review ambiguous or context-dependent categories |
| Duplicate detection |
Flag records that meet documented matching criteria |
Verify uncertain matches before deletion or merging |
| Derived variables |
Calculate values using approved formulas |
Validate formulas and source fields |
| Missing-data processing |
Apply an approved missing-data procedure |
Preserve missingness indicators and analytical justification |
| Outlier handling |
Flag unusual observations for review |
Do not assume unusual values are invalid |
| Record exclusion |
Generate provisional exclusion recommendations |
Apply prespecified criteria and appropriate approval |
The appropriateness of autonomous execution depends on whether the agent has sufficient information to apply the rules correctly.
Where a transformation requires interpretation, provisional recommendations may be safer than automatic modification.
Why should original research data be preserved?
Original records provide an evidentiary reference for determining what was actually collected or observed.
If an AI agent overwrites these records, researchers may lose the ability to distinguish original observations from subsequent transformations.
This can undermine error investigation, reproducibility, and the defense of methodological decisions.
A safer workflow maintains protected source records and allows the agent to operate on separate working copies.
Version control, immutable storage, access restrictions, or equivalent safeguards may help preserve the original record.
The exact arrangement should reflect the data's sensitivity, institutional requirements, and research context.
What is an immutable research dataset?
An immutable dataset is a preserved version that cannot be altered through the ordinary processing workflow.
Immutability may be implemented through read-only permissions, versioned object storage, protected repositories, or other technical controls.
The objective is to maintain a stable reference against which later modifications can be evaluated.
However, immutable storage does not automatically guarantee that the original data were accurate or complete.
It preserves a recorded state. Corrections to genuine source errors may still be necessary, but they should be documented rather than silently replacing the historical record.
Can AI agents remove duplicate research records?
Agents can identify potential duplicates using exact identifiers, similarity measures, or predefined matching rules.
For bibliometric research, duplicate records may arise when the same publication is exported from multiple databases.
However, similar titles, author names, or publication metadata do not always establish that two records represent the same work.
Conference papers and subsequent journal articles may share substantial content while remaining distinct publications.
An agent that automatically merges them may distort publication counts or citation indicators.
Researchers should distinguish high-confidence matches from ambiguous cases and preserve a record of every merge or deletion.
Can AI agents normalize research entities?
Entity normalization involves mapping variant representations to a consistent identity or category.
Examples include author names, institutional affiliations, country names, and research keywords.
AI agents may help identify candidate mappings, but the process can introduce consequential errors.
Two researchers may share a similar name, while one institution may have several historical or organizational forms.
Incorrect mappings can alter collaboration networks, productivity indicators, and thematic analyses.
Researchers should preserve the original values, document the mapping rules, and verify ambiguous entities before applying consequential changes.
Should AI agents automatically remove outliers?
Generally, unusual observations should not be removed merely because they are statistically extreme.
An outlier may represent a data-entry error, a legitimate rare observation, or an important feature of the phenomenon being studied.
For example, an unusually high number of study hours reported by a participant may require verification, but its magnitude alone does not establish invalidity.
Researchers should use predefined exclusion or correction criteria and examine the substantive context.
An AI agent may flag unusual values and generate diagnostic reports.
Automatic exclusion should be restricted to circumstances where the rules are scientifically justified and appropriately authorized.
Can AI agents fill in missing research data?
AI systems may implement imputation procedures, including conventional statistical methods or appropriately validated model-based approaches.
However, missing-data treatment requires assumptions about why observations are missing and how the missingness mechanism affects inference.
Replacing missing values with plausible numbers does not recover the original observations.
Researchers should distinguish observed data from imputed values and preserve indicators identifying the imputation procedure.
The choice of method should follow the research design and analytical requirements.
An agent should not silently replace missing values with generated content merely to produce a complete dataset.
When does data cleaning become falsification?
Legitimate data cleaning seeks to improve the accuracy, consistency, or analytical suitability of the dataset through defensible procedures.
Falsification involves manipulating research materials, processes, or data, or changing or omitting data or results, such that the research is not accurately represented in the research record, under commonly used research-integrity definitions.
For example, correcting a verified transcription error is different from changing an observation because it contradicts a hypothesis.
Likewise, excluding records according to prespecified eligibility criteria differs from selectively deleting inconvenient observations to obtain a desired result.
Whether a particular action constitutes formal research misconduct depends on the applicable policies, evidence, and relevant findings about the conduct.
Researchers should not assume that AI-mediated manipulation is less consequential than manual manipulation.
Does the agent's intention matter?
AI systems do not necessarily possess human intentions in the sense relevant to research-misconduct determinations.
However, the system may be configured or prompted to optimize a particular outcome.
For example, instructing an agent to "clean the data until the intervention effect becomes statistically significant" creates an obvious methodological concern.
The researcher has directed the system toward an outcome rather than a defensible data-quality criterion.
Researchers should specify transformation objectives independently of desired findings.
Human accountability remains relevant even when the actual file modifications are performed automatically.
What is the principle of least privilege?
The principle of least privilege involves granting a system only the access necessary to perform its authorized functions.
An agent assigned to identify data-quality problems may require read access but not write permission.
An agent preparing a cleaned dataset may require permission to create a new file without permission to overwrite the original.
An agent applying approved transformations may operate within a restricted working directory.
Limiting permissions reduces the potential consequences of incorrect action selection, software defects, or compromised instructions.
The permissions granted should correspond to the specific task rather than the maximum capabilities of the agent.
What is a research data modification audit trail?
An audit trail is a record of changes made to research data and the circumstances under which they occurred.
For AI-mediated modifications, useful information may include the original and revised values, affected records, transformation rules, execution timestamps, software versions, and approval decisions.
Where relevant, the audit trail may also identify the prompt or instruction that initiated the operation.
Audit records should allow researchers to reconstruct the processed dataset from preserved source data.
However, logging alone does not establish that a transformation was appropriate.
Records must also be reviewed against the study's methodological requirements.
How can researchers measure modification accuracy?
Researchers may compare agent-generated changes with independently verified reference decisions.
For example, a sample of proposed duplicate merges could be assessed by trained reviewers using predefined criteria.
Relevant measures may include precision, recall, and the frequency of consequential incorrect changes.
A low error percentage does not necessarily establish acceptable performance if the errors affect consequential observations.
Researchers should examine severity and reversibility alongside frequency.
Should AI agents be allowed to overwrite research files?
Overwriting should generally be restricted when the file contains original observations, approved analytical records, or other important research evidence.
For routine derived outputs, overwriting may be acceptable under a documented version-control and recovery procedure.
However, the agent should not silently replace original data or erase previous analytical states.
Researchers should distinguish temporary files, reproducible derived files, and protected source records.
Permissions should reflect the consequences of losing or altering each type of file.
How should approval checkpoints work?
An approval checkpoint allows a researcher to examine proposed modifications before they are committed.
For example, an agent may prepare a table listing records recommended for deletion, the reasons for each recommendation, and the expected effect on the dataset.
The researcher can then approve, reject, or revise the proposed changes.
However, approval is meaningful only if the reviewer has sufficient information to evaluate the action.
A generic confirmation message such as "Apply all recommended changes?" may provide inadequate oversight when hundreds of consequential modifications are involved.
The principles of human approval checkpoints for AI agents are relevant.
Can autonomous data modification affect reproducibility?
Yes. Data modifications influence the analytical dataset and therefore the results that can be reproduced.
When an agent applies documented deterministic transformations, the workflow may be relatively straightforward to reconstruct.
When it makes context-dependent decisions using a changing generative model, exact repetition may be more difficult.
Researchers should preserve source data, transformation scripts, model identifiers, relevant prompts, and resulting outputs.
Where feasible, approved changes should be converted into reproducible scripts or explicit mapping tables rather than relying solely on repeated model judgments.
Reproducibility requires sufficient documentation of what was changed and why.
How should data modification be reported in a research paper?
The methods section should describe the data preparation procedures that materially affected the analytical dataset.
When AI agents performed transformations, researchers should identify the relevant system, its role, the rules applied, and the verification procedures.
Consequential exclusions, imputations, entity mappings, and corrections should be documented according to disciplinary reporting standards.
Researchers should distinguish automated suggestions from changes actually applied.
Where appropriate, a reproducible transformation script or decision log may be provided as supplementary material.
The objective is to establish an accurate account of how the original data became the analytical dataset.
