Should Researchers Record the AI Model Version Used?

If You Use ChatGPT Today, Will You Know Which Model You Used a Year From Now?

A researcher uses ChatGPT to classify 500 open-ended survey responses about students' experiences with generative AI. The researcher saves the prompts, generated classifications, and final coding matrix.

However, the researcher does not record which model was used.

One year later, another researcher attempts to reproduce the classification procedure using the same application and prompts. Several responses receive different classifications.

Did the model change? Were the settings different? Did the original interaction include contextual information that was not preserved?

The researcher cannot answer confidently because the original model information was never documented.

This situation illustrates why the identity of an AI application is not always sufficient to establish the technical context of a research interaction. A service may provide access to different models or change its underlying implementation over time.

Record the Model Version When It Is Available and Relevant to the Research

Recording the model is useful, but it does not guarantee reproducibility. Prompts, inputs, settings, conversation history, software environments, and provider-side changes may also influence the outputs.

Why AI Model Version Records Matter for Research Documentation

What Is an AI Model Version?

An AI model version identifies a particular model release, checkpoint, or documented implementation used to generate or process outputs.

Depending on the provider, this information may appear as a model name, technical identifier, dated release, or other version designation.

However, these identifiers do not always describe the same level of technical specificity.

A commercial service may display a general model family while its internal implementation changes. An application programming interface may expose a more specific identifier, while a locally hosted model may be associated with a particular checkpoint.

Researchers should record the information actually provided by the system rather than assume that every model name identifies an immutable implementation.

How Is a Model Version Different From an AI Application?

An AI application is the interface or service through which researchers interact with artificial intelligence.

The model is the computational system that performs the relevant processing.

For example, ChatGPT is an application provided by OpenAI. The application may offer access to different models or configurations over time.

Likewise, an AI-assisted research platform may integrate models supplied by another organization.

Recording only the application name may be sufficient for some routine activities but less informative when the model materially affects research findings.

Why Can Model Versions Affect Research Results?

Different models may respond differently to the same instructions.

They can vary in their ability to follow classification rules, interpret text, generate analytical code, or extract information from documents.

For example, one model may classify a student's statement about unreliable internet access as a technical barrier, while another may incorrectly classify it as low motivation.

If such classifications are used to produce frequency distributions or qualitative findings, differences in model behavior may affect the results.

Model information can therefore help explain the technical conditions under which the original analysis occurred.

However, the presence of different results does not by itself prove that a model update caused the discrepancy.

Is Recording the Model Version Mandatory?

Not universally. Documentation requirements depend on the research design and applicable policies.

The International Committee of Medical Journal Editors (ICMJE) recommends disclosure of AI-assisted technologies and an explanation of how they were used.

The Committee on Publication Ethics (COPE) similarly emphasizes transparency about AI involvement in research and manuscript preparation.

These general positions should not be interpreted as requiring an exact model version for every AI interaction.

Some journals or methodological reporting frameworks may require more detailed technical identification, particularly when the model itself is evaluated or used to generate research evidence.

Researchers should distinguish internal recordkeeping from the information that must appear in a published disclosure.

Which AI Activities Make Model Version Records Particularly Important?

These categories are practical recommendations rather than a universal institutional standard.

What Information Should Researchers Record Alongside the Model Version?

A model identifier becomes more useful when connected to the actual research activity.

Relevant information may include the application, provider, access date, prompts, inputs, outputs, available settings, and verification procedures.

For example, recording that a particular model was used is less informative if the researcher cannot identify which dataset it processed or which outputs entered the final analysis.

A structured research AI-use log can connect these details to the broader research record.

The record should contain actual information rather than illustrative values presented as historical facts.

What If the Model Version Is Not Displayed?

Some AI applications do not expose the exact model identifier or technical release information.

Researchers should record the available application and provider details and note that the underlying version was not disclosed.

Where possible, they may consult contemporaneous service records or documentation.

However, a general announcement that a model was available at a particular time does not establish that the researcher used it.

The absence of a precise identifier may limit later reconstruction, but inventing one would create a more serious documentation problem.

The record should distinguish known facts from unavailable information.

Can Researchers Infer a Model Version From the Date?

Not reliably in every case.

Providers may offer multiple models simultaneously, route requests through different configurations, or update implementations without providing complete technical details.

A public model release date therefore does not establish which model processed an individual interaction.

For example, knowing that a model was publicly available in October does not prove that every conversation during that month used it.

Researchers should avoid converting an inference into a precise historical record.

Where the exact version cannot be verified, that uncertainty should remain explicit.

What If a Provider Uses a Model Alias?

Some services provide aliases that may refer to changing model implementations.

An alias can be useful for identifying the interface or model family but may not uniquely identify the underlying release used at a particular time.

Where a more specific identifier is available, researchers should consider recording it.

Where only an alias is exposed, the log should preserve the alias and the access date without implying that the identifier is immutable.

Researchers should also retain relevant outputs because the same alias may not reproduce the original response later.

Does a Model Version Guarantee Reproducibility?

No. Identical model identifiers do not necessarily guarantee identical outputs.

Generative AI systems may produce variable responses, and commercial providers may alter supporting infrastructure or hidden instructions.

Outputs may also depend on prompt wording, conversation context, retrieval sources, model settings, and input materials.

For example, two researchers may use the same named model and prompt but obtain different classifications because their surrounding conversation contexts differ.

Model identification supports reproducibility but cannot replace preservation of the actual procedure and outputs.

Should Researchers Record Model Settings as Well?

Settings may affect output behavior and should be documented when they are available and methodologically relevant.

For example, an API-based research procedure may expose parameters controlling sampling or output length.

Consumer applications may not provide the same level of technical control.

Researchers should record the settings they actually selected and avoid assigning values to hidden parameters.

The relationship between recording AI tool settings and model identification is important because the same model may behave differently under different configurations.

Should Researchers Record Model Versions Used for Statistical Code?

Where AI materially contributes to code generation, recording the model can help document the development process.

However, the final executed code is generally more directly relevant to reproducing the computational analysis.

For example, a researcher may generate a Python script using an AI system and subsequently revise it extensively.

The final tested script, dependencies, data transformations, and execution environment determine the analysis actually performed.

The model identifier provides additional provenance but does not replace those records.

Researchers should distinguish reproducibility of the executed analysis from reproducibility of the original code-generation interaction.

Should Model Versions Be Recorded for AI-Assisted Qualitative Analysis?

Yes, where available and consequential to the methodology.

Different models may suggest different codes, themes, or interpretations from the same qualitative materials.

For example, a researcher using AI to classify interview excerpts should consider recording the model identifier, coding prompt, date, and actual classifications.

These records can help explain the analytical workflow and support later methodological evaluation.

However, model identification does not establish the validity of the generated categories.

Human researchers remain responsible for evaluating the outputs in relation to the chosen qualitative approach.

What About Research Comparing Different AI Models?

Model identification becomes central when the study explicitly compares AI systems.

For example, a researcher may evaluate how different models generate feedback on student essays or classify educational assessment items.

Without sufficiently precise model information, readers may be unable to determine which systems were evaluated.

The methods should identify the models, relevant configurations, evaluation dates, prompts, and procedures used to collect outputs.

Where model versions are unavailable, the limitation should be reported.

The study should not imply that results necessarily generalize to later model releases.

How Should Locally Hosted Models Be Documented?

Locally hosted or open-weight models may allow researchers to preserve more technical information than commercial interfaces expose.

Relevant records may include the model checkpoint, source repository, version or commit identifier, inference software, quantization format, and execution settings.

Where possible, checksums may help identify the exact files used.

However, the appropriate level of detail depends on the research design and available infrastructure.

Researchers should not assume that recording a general model family is sufficient when several checkpoints or configurations exist.

Licensing and access restrictions may also affect whether model files can be redistributed.

Should Researchers Save Screenshots of the Model Selector?

A screenshot may provide supplementary evidence of the model label displayed by an application at the time of use.

However, it may not establish the exact backend implementation or technical checkpoint.

Screenshots can also omit relevant context or become separated from the interaction they document.

Where available, structured metadata or service-generated identifiers may provide more precise records.

A screenshot should therefore be treated as supporting documentation rather than definitive proof of every underlying technical detail.

Can Model Information Change During a Single Research Project?

Yes. A project may involve multiple models, updates, or different tools across its stages.

For example, one model may be used for preliminary coding and another for subsequent verification.

Researchers should record which system contributed to each consequential activity rather than list all model names without connecting them to specific outputs.

Where a model changes during data collection or analysis, the change may need methodological consideration.

Differences in model behavior could affect the comparability of outputs.

The research record should make relevant changes identifiable.

Should Researchers Disclose Model Versions in the Manuscript?

That depends on the journal's policy and the model's role in the research.

For routine writing assistance, the required declaration may identify the tool and purpose without specifying an exact version.

For model-based analysis or studies evaluating AI systems, more detailed identification may be necessary.

Researchers should distinguish internal version records from the information required in a publication.

The question of naming AI tools and model versions in disclosures should be considered alongside the methodological reporting requirements.

How Long Should Model Version Records Be Retained?

Retention depends on the applicable research-records policy and the significance of the AI-assisted activity.

Where model information is necessary to interpret published findings, it may need to be retained with the broader research archive.

However, no universal retention period applies specifically to every AI model identifier.

Researchers should preserve the information according to institutional, funder, and project requirements.

Model records should remain connected to the prompts, outputs, and analytical materials they describe.

What If the Model Information Was Not Recorded?

Researchers may attempt to recover the information from saved interaction metadata, API logs, service records, or other contemporaneous documentation.

However, reconstruction should not involve unsupported assumptions.

If the exact identifier cannot be established, the researcher should record the known information and acknowledge the limitation where relevant.

For example, the record may identify ChatGPT as the application while noting that the underlying model version was not preserved.

A later model should not be presented as the original merely because it is currently available.

Accurate documentation of uncertainty is preferable to false precision.

When a Model Update Complicates Reproduction of AI-Assisted Coding

The example illustrates why model identification can strengthen the research record without guaranteeing identical outputs.

The missing identifier limits reconstruction, but it does not establish that a model update caused the differences.

Preserving actual outputs and human verification records remains important even when model information is available.

Misconceptions About Recording AI Model Versions

How Should You Record AI Model Information During Research?

Researchers should capture available model information at the time of consequential AI use rather than rely on later reconstruction.

The record should connect the model to the particular research activity and outputs it produced.

Researchers should also ensure that the documentation remains accessible through authorized research storage arrangements.

The appropriate standard is accurate, contemporaneous, and proportionate documentation of the AI system actually used.

What to Record About AI Models Used in Research

Frequently Asked Questions About Recording AI Model Versions

Record the Model That Produced the Research Output, Not the One Available Later

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.