How Do Researchers Create an Audit Trail for AI-Assisted Data Cleaning?

How Can You Prove What AI Changed in Your Research Dataset?

You have used generative AI to help clean a dataset containing 600 student questionnaires. The AI suggested corrections to invalid responses, standardized academic program labels, identified possible duplicates, and generated a script that modified several variables.

Three months later, a coauthor asks why a particular response changed from 8 to 3. A reviewer wants to know how missing values were handled. You can locate the cleaned spreadsheet, but can you reconstruct the decisions that produced it?

An audit trail addresses this problem. It connects the original evidence to the transformations, decisions, and verification procedures that produced the analysis-ready dataset.

Record the Changes, Their Justifications, and Their Verification

What an AI-Assisted Data Cleaning Audit Trail Should Contain

What Is a Data Cleaning Audit Trail?

A data cleaning audit trail is a structured record of the decisions and operations through which source data become a cleaned or analysis-ready dataset.

Its purpose is to establish provenance: where the information originated, what happened to it, and how subsequent versions relate to the original observations.

For AI-assisted cleaning, the record should also distinguish the AI system's contribution from the researcher's decisions. An AI suggestion is not necessarily an approved transformation, and an approved transformation is not necessarily evidence that the intended operation executed correctly.

The UK Data Service emphasizes documenting data processing and changes to research data. The W3C PROV model provides a more formal framework for representing provenance through entities, activities, and agents.

Researchers do not necessarily need to implement a formal provenance database. A well-maintained change log, versioned datasets, and reproducible scripts may be sufficient for many projects.

What Is the Difference Between a Change Log and an Audit Trail?

A change log ordinarily records what changed. An audit trail should provide enough context to establish how and why the change occurred and who or what was responsible.

A change log can form part of an audit trail, but it does not necessarily capture the full decision-making process.

Which Details Should Be Recorded for Each Cleaning Decision?

The appropriate level of detail depends on the operation. Individual corrections may require record-level documentation, while an approved transformation affecting thousands of values can be documented through a rule, execution record, and complete change report.

For batch operations, the audit record should also identify the exact rule, script version, number of affected observations, and any unexpected outcomes.

Should Researchers Save Every AI Prompt and Response?

Not necessarily. Saving every exploratory conversation may create excessive documentation without improving reproducibility.

What matters is preserving the information needed to understand and reconstruct consequential decisions. This may include the final instructions supplied to the AI, the relevant model or tool configuration, the generated code, and the outputs that informed approved transformations.

If an AI suggestion was rejected because it would have altered valid observations, recording that rejection may be useful when it explains a significant methodological choice.

Researchers should also consider confidentiality. Prompts and chatbot outputs may contain participant information, so they should not automatically be placed in publicly accessible repositories.

A practical recommendation is to preserve material AI interactions relevant to substantive decisions rather than indiscriminately archiving every conversational exchange.

How Should Dataset Versions Be Managed?

Each important processing stage should be identifiable without overwriting the original source.

For example, a project might retain:

• survey_raw_v1.csv as the protected source export.
• survey_validated_v1.csv as the version with validation flags.
• survey_cleaned_v1.csv as the first approved cleaned dataset.
• survey_analysis_v1.csv as the version prepared for the specified analysis.

These filenames are illustrative, not a mandatory naming convention. A project may instead use institutional repositories, database snapshots, or a managed data pipeline.

The essential requirement is that each version can be identified and connected to its inputs and transformations.

For stronger file-level verification, researchers may record cryptographic checksums. A checksum can help establish whether file contents have changed, although it does not demonstrate that the contents are methodologically correct.

How Do Scripts Contribute to the Audit Trail?

A saved script records the computational instructions used to transform the dataset. It can support reproducibility when paired with the correct inputs, software environment, and execution settings.

However, a script does not necessarily explain why its rules were chosen. For example, code may delete every record with a particular flag without documenting the evidence that justified exclusion.

Researchers should therefore preserve both the executable procedure and the methodological rationale.

OpenRefine supports operation histories that can be reviewed, exported, and reapplied. Its documentation illustrates how recorded transformations can help reconstruct cleaning procedures, although researchers must still document decisions that the software history does not explain.

Should Rejected AI Suggestions Be Recorded?

Not every rejected suggestion requires a separate entry. However, consequential rejected proposals may be important to the methodological history.

Suppose AI recommends deleting a student who reports 40 hours of weekly AI use because the observation is statistically extreme. The researcher checks the original questionnaire and retains the response.

Recording that decision helps demonstrate that the observation was investigated rather than overlooked.

Similarly, if AI proposes merging two academic programs that the researcher determines are distinct, the rejected mapping may be worth preserving to prevent the same mistake during later processing.

The audit trail should remain proportionate to the risks and consequences of the decisions.

How Should Verification Be Documented?

Verification should establish that approved changes were applied correctly and that unintended changes did not occur.

For a single corrected value, the researcher may compare the cleaned record with the original questionnaire. For a batch recoding operation, a complete before-and-after comparison can establish whether every affected value follows the approved mapping.

The verification record should identify the procedure used, its result, and any unresolved exceptions.

For example, "All 42 category replacements matched the approved mapping; no unmatched values were altered" is more informative than simply recording "Cleaning completed."

The specific procedures for verifying AI-assisted data changes should be documented alongside the transformation itself.

Who Should Maintain the Audit Trail?

The research team should assign responsibility for recording and approving cleaning decisions. In collaborative projects, the person proposing a change may differ from the person authorizing it or verifying its execution.

Clear responsibilities help prevent uncertainty about who made a consequential decision and whether the final dataset was approved for analysis.

For AI-assisted workflows, the AI system should be identified as a tool or processing component rather than treated as the accountable investigator.

Institutional policies, funder requirements, and disciplinary conventions may establish additional expectations for data retention and documentation.

How Detailed Must the Audit Trail Be?

An audit trail should be detailed enough to support the reconstruction and evaluation of consequential transformations without collecting unnecessary information.

For example, trimming harmless whitespace from a categorical label may be documented through one approved rule and a generated change report. Deleting participant records or replacing missing values may require substantially more explanation.

The level of scrutiny should reflect the risk that an incorrect operation could affect the research findings.

The broader decision about which cleaning rules require explicit researcher specification determines much of what the audit trail must preserve.

Documenting AI-Assisted Cleaning of an Academic Self-Efficacy Dataset

After executing the approved transformations, the researcher saves a new dataset version and generates a complete comparison report. The original dataset remains unchanged.

Six months later, a coauthor asks why P017's response differs between the original and analysis-ready files. The researcher can locate CL-001, inspect the supporting source record, and reproduce the correction.

Interpretation: The audit trail does more than identify changed cells. It demonstrates which AI suggestions were accepted, which were rejected, and how the resulting dataset can be traced to the original evidence.

Common Misconceptions About AI Data Cleaning Audit Trails

How Should You Organize an AI Data Cleaning Audit Trail?

For many research projects, a practical arrangement consists of protected source data, a decision log, reviewed transformation scripts, versioned outputs, and verification reports.

These components should reference one another through stable identifiers or filenames so that a particular analytical value can be traced back to its source.

Do not allow an AI agent to overwrite the original data or alter the audit records without appropriate controls. The separate question of AI access to the master research dataset should be resolved before deployment.

A useful audit trail should allow another qualified researcher to understand the transformations without having to guess what an AI prompt meant or rely on an unavailable chatbot session.

Before Finalizing an AI-Assisted Data Cleaning Audit Trail

Frequently Asked Questions About AI Data Cleaning Audit Trails

An Audit Trail Should Explain the Data, Not Merely Record the AI

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.