03 · What You Need to Know
How to Make an AI-Assisted Research Workflow Reconstructable
What Does Research Workflow Reconstruction Mean?
Workflow reconstruction involves establishing how a research output was produced through identifiable procedures, information, and decisions.
For AI-assisted research, this may include the input dataset, instructions supplied to the model, system information, generated outputs, subsequent human corrections, and validation procedures.
For example, a classification variable used in a regression analysis should be traceable to the original responses and the procedure used to assign categories.
The objective is to make the scientific process understandable and evaluable.
Reconstruction should not depend exclusively on the original researcher's memory.
Reconstruction Is Not the Same as Exact Reproduction
Workflow reconstruction
Establishes what procedures, inputs, outputs, and decisions produced the reported research artifacts.
Exact computational reproduction
Seeks to obtain the same computational results under sufficiently specified data, code, software, and execution conditions.
A study may be reconstructable even when the original AI model is no longer available.
For example, preserved AI classifications, reference judgments, and final analytical code may allow future researchers to inspect the procedure and reproduce downstream statistical calculations.
However, they may be unable to regenerate the exact classifications using the original model.
Researchers should describe these distinctions accurately rather than claim complete reproducibility when important dependencies remain unavailable.
Why Saving the AI Conversation May Be Insufficient
A conversation transcript may preserve prompts and responses but omit important information about the processing environment.
For example, it may not identify the exact model version, input dataset, generation settings, or connected tools.
It may also fail to show which generated outputs were accepted, modified, or rejected.
Researchers should connect conversation records to the actual analytical artifacts and decisions.
An extensive chat history is not automatically a useful research audit trail.
Which Records Form the Core of a Reconstructable Workflow?
| Record |
Why it matters |
Illustrative artifact |
| Research task description |
Explains the purpose of AI assistance |
Classification or analysis protocol |
| Input dataset version |
Identifies the information processed |
Secure dataset reference and version |
| Model information |
Identifies the available AI system details |
Provider, model identifier, execution date |
| Prompts and instructions |
Documents how the task was specified |
Coding criteria and relevant examples |
| Configuration |
Records available execution settings |
Generation parameters and tool configuration |
| Original outputs |
Preserves what AI actually produced |
Generated labels, code, or extracted information |
| Human decisions |
Explains acceptance, correction, or rejection |
Analytical memos and revision records |
| Verification evidence |
Supports evaluation of correctness |
Code tests or reference judgments |
| Final analytical artifacts |
Connects procedures to reported results |
Validated dataset, scripts, and tables |
Not every task requires every record. The necessary detail depends on scientific significance and the intended reconstruction.
Why Input Dataset Versions Are Essential
AI outputs depend on the information supplied to the system.
A classification procedure applied before duplicate removal may produce different results from one applied after data cleaning.
Similarly, corrected interview transcripts may differ from earlier versions used for preliminary coding.
Researchers should identify which dataset version was used for each consequential operation.
Where appropriate, file checksums or cryptographic hashes can help establish whether archived artifacts have changed.
However, file integrity does not independently establish scientific validity.
How Should Researchers Preserve Prompts?
Prompts may function as procedural instructions when AI performs research tasks.
For example, a classification prompt may define categories, inclusion criteria, and how ambiguous cases should be handled.
Researchers should preserve consequential prompts and material revisions.
Where system instructions or provider-managed configurations are inaccessible, that limitation should be documented.
Exact prompt preservation may support reconstruction but does not guarantee identical future outputs.
Why Original AI Outputs Should Be Preserved
Original outputs provide evidence of what the system actually generated.
For example, a model may classify 1,000 student reflections, after which researchers correct selected labels.
Preserving only the final corrected dataset would make it difficult to determine which decisions came from AI and which were made by humans.
Where scientifically relevant and permitted, original outputs should be retained separately from final validated artifacts.
This supports examination of corrections and the effectiveness of human oversight.
How Should Human Revisions Be Documented?
Researchers may modify AI-generated code, classifications, or interpretations before using them.
Consequential revisions should be traceable.
For example, a researcher may reject an AI recommendation to exclude unusual observations after examining the original data.
The decision record should explain the relevant evidence and rationale.
Documentation need not preserve every incidental editing action, but it should capture changes that materially affect the research procedure or findings.
How Should Researchers Preserve Validation Evidence?
Validation records help establish why AI outputs were considered suitable for their intended use.
For classification tasks, relevant evidence may include reference judgments, error analyses, and performance measures.
For statistical code, relevant records may include independent calculations, code tests, and methodological review.
For qualitative interpretation, records may include analytical memos and examination of original material.
Validation evidence should be linked to the outputs and procedures it supports.
A generic statement that all outputs were checked may provide insufficient information for later evaluation.
Why Downstream Analytical Artifacts Matter
AI-generated outputs often become inputs to later research procedures.
For example, generated classifications may enter a statistical model, produce summary tables, and support manuscript conclusions.
Researchers should preserve the relationships between these artifacts.
A final table should be traceable to the analytical code and dataset from which it was produced.
Where AI influenced the creation of those inputs, the relevant contribution should also be identifiable.
Traceability helps researchers investigate errors discovered after publication.
What Is Research Provenance?
Provenance concerns the origin and history of research information and artifacts.
In an AI-assisted workflow, provenance may identify which data, model, instructions, and human decisions contributed to a particular output.
For example, a classification file may be associated with the original dataset version, model identifier, prompt, and validation record.
Provenance can support accountability and reconstruction by making these relationships explicit.
However, provenance records do not automatically establish that the underlying procedure was scientifically appropriate.
How Can Researchers Organize a Workflow Archive?
A practical archive may use structured folders, version control, metadata records, and a manifest describing the relationships between artifacts.
For example, original AI outputs can be stored separately from corrected analytical datasets.
Prompts and configuration records can be linked to the relevant processing batches.
Validation results can be associated with the outputs they evaluate.
The organization should allow qualified reviewers to identify which artifacts were preliminary, superseded, or used in the final analysis.
Clear structure is more valuable than an unorganized collection of files.
What if the AI Model Changes During the Project?
Model updates may affect output behavior and the comparability of research measurements.
Researchers should record which available model identifiers were associated with each consequential processing period.
Where changes may affect the results, additional validation may be necessary.
For longitudinal studies, the team should determine whether outputs generated under different versions remain comparable.
Reprocessing earlier data using a common validated model may sometimes be appropriate.
However, the decision depends on the research design and available evidence.
What if the Original Model Becomes Unavailable?
The disappearance of an AI model can limit exact reproduction of the original generation process.
Researchers should preserve original outputs, available technical information, and downstream analytical artifacts while the project is active.
These records may allow future researchers to evaluate the procedure even when the model cannot be rerun.
For example, preserved classifications and validated statistical code may support reconstruction of the analysis.
The limitation should be acknowledged rather than represented as complete computational reproducibility.
How Do External Tools Affect Reconstruction?
AI systems may rely on retrieval services, databases, software tools, or connected applications.
These components can materially influence the generated outputs.
For example, an AI-assisted literature summary may depend on a set of retrieved documents that changes over time.
Researchers should preserve relevant source references, retrieval conditions, and available tool records where necessary.
Documenting only the language model may be insufficient when external components determine the information it receives.
Can Researchers Reconstruct Qualitative AI Assistance?
Qualitative research may involve interpretive decisions that cannot be reduced to exact computational repetition.
Researchers should preserve relevant AI suggestions, original source material, analytical memos, and consequential decisions.
For reflexive thematic analysis, reconstruction may focus on understanding how interpretations developed rather than reproducing identical codes.
For structured classification, reference judgments and validation procedures may be more central.
The appropriate records depend on the methodology and claims.
What About Confidential Research Data?
Reconstruction requirements do not override confidentiality, consent, or data protection obligations.
Researchers may need to retain identifiable records securely without making them publicly accessible.
Where appropriate, public documentation can describe the workflow while protected source information remains under authorized access controls.
Metadata and secure references may help establish relationships between artifacts without exposing participant information.
The archive should support legitimate scrutiny while avoiding unnecessary disclosure.
Does FAIR Require All AI Research Records to Be Public?
No. The FAIR Guiding Principles concern findability, accessibility, interoperability, and reusability.
They do not require every research dataset to be openly available without restriction.
Researchers may provide descriptive metadata and establish controlled-access procedures where confidentiality obligations apply.
FAIR principles can help organize AI-assisted research artifacts, but they should be applied consistently with relevant legal and ethical requirements.
How Long Should Reconstruction Records Be Retained?
Retention requirements vary across institutions, funders, journals, and jurisdictions.
There is no universal retention period for all AI prompts, outputs, or research artifacts.
Researchers should identify which materials form part of the official research record and apply the relevant retention schedules.
Where records contain personal information, indefinite retention may be inappropriate.
The documentation plan should balance scientific accountability with information protection and deletion obligations.
How Can Researchers Test Whether Their Archive Is Sufficient?
A practical test is to ask a qualified colleague who did not perform the original AI task to examine the preserved records.
The colleague should attempt to identify the inputs, procedures, outputs, verification evidence, and decisions that produced a selected research result.
If important steps cannot be established, the archive may require additional documentation.
For computational analyses, the colleague may attempt to reproduce downstream calculations using the preserved code and authorized data.
For interpretive work, the assessment should follow the relevant methodological criteria.
Such a reconstruction exercise provides practical evidence of documentation adequacy, although it cannot guarantee that every future researcher will be able to reproduce the study.
When Is Reconstruction Insufficient?
A workflow may be insufficiently reconstructable when essential inputs, outputs, or decisions are missing.
For example, if an AI classifier generated the primary research variables but neither the original classifications nor the model information were preserved, future evaluation may be substantially limited.
The team should determine whether remaining records and validation evidence adequately support the conclusions.
Where the missing information undermines consequential findings, additional analysis or an alternative procedure may be necessary.
Researchers should avoid overstating reproducibility when essential components cannot be established.
