How Much Documentation Is Enough to Defend What You Actually Did?

How Much Evidence Would You Need If Someone Asked What You Actually Did?

“Document your research carefully” sounds straightforward until you try to apply it. Should every analytical choice receive a written explanation? Must every intermediate dataset survive? Do you need records of every meeting, test run, failed procedure, and minor change?

At one extreme, sparse documentation leaves researchers unable to establish what happened. At the other, indiscriminate documentation produces thousands of poorly organized files that may make the authoritative record harder to find.

The useful standard lies between those extremes: preserve enough evidence to support a reliable reconstruction of consequential research activities.

Enough Documentation Means Enough to Establish the Consequential Facts

The Right Standard Is Reconstruction, Not Volume

There is no universal number of pages, files, screenshots, log entries, or metadata fields that makes research sufficiently documented. A small observational study and a regulated multicenter clinical trial plainly do not require identical documentation systems.

The better question is functional: what must the record allow someone to establish?

Start With What You Would Need to Prove About the Research

For consequential parts of a study, documentation should usually be capable of establishing several basic facts where they are relevant:

• what procedure, analysis, or activity occurred;
• which evidence, participants, samples, files, or materials were involved;
• who performed or authorized an important action;
• when the activity or decision occurred;
• which protocol, instrument, dataset, software, code, or other version was used;
• what consequential changes, deviations, exclusions, corrections, or transformations occurred;
• how important outputs connect to their underlying evidence.

You may not need every element for every activity. The point is to identify the facts that would become difficult to establish if the documentation disappeared.

Documentation Should Be Proportional to Consequence

A formatting change to a graph label and a decision to exclude 15% of the sample should not normally receive the same documentary treatment.

As the potential effect of an action on the study increases, the case for stronger documentation usually increases with it. Decisions affecting eligibility, primary outcomes, data transformations, protocol deviations, analytical specifications, coding frameworks, corrections, or interpretations generally deserve more attention than routine operational choices.

The Record Should Establish What Happened, Not Merely What Was Supposed to Happen

A protocol, preregistration, standard operating procedure, or analysis plan establishes intent. It does not by itself prove that the planned procedure was followed.

Defensible documentation therefore needs evidence of execution where execution matters. That evidence might be a laboratory entry, field record, instrument log, completed form, version-controlled script, processing log, qualitative memo, electronic system record, or another form appropriate to the research.

If the plan says one thing and the actual procedure changed, the record should preserve the difference rather than allowing the plan to masquerade as a description of what happened.

Document Enough Context to Make the Evidence Interpretable

NIH data-management guidance emphasizes that metadata and associated documentation allow users to understand how data were collected and how to interpret them. Examples include methodology, procedures, labels, variable definitions, and other information necessary to reproduce and understand the data.

This principle extends beyond datasets. A cryptic note, unexplained filename, unidentified specimen, undocumented variable code, or analysis script with unknown inputs may technically exist while providing little defensible evidence.

An adequate research record therefore needs enough context to remain meaningful after the immediate project context has faded.

Preserve Provenance Through Consequential Transformations

Research evidence often changes form. Raw observations become cleaned data. Variables are recoded. Images are processed. Qualitative material is coded. Samples receive identifiers. Analytical datasets are derived from source datasets.

NIH repository guidance treats provenance as the ability to record the origin, chain of custody, and modifications to datasets and metadata. This provides a useful principle for documentation more broadly: important outputs should not become disconnected from their origins.

You do not necessarily need to preserve every transient intermediate file. You do need enough information to understand consequential transformations and reproduce or verify them where appropriate.

Use Contemporaneous Records Instead of Future Memory

Documentation becomes less defensible when the only explanation is “I remember doing it that way.”

FDA good-documentation materials use the ALCOA framework in regulated settings: attributable, legible, contemporaneous, original, and accurate. FDA also describes additional characteristics including completeness, consistency, endurance, and availability. These are regulatory concepts and should not be converted into universal legal requirements for ordinary academic research. Still, they illustrate why timing, attribution, and preservation matter when records must establish what happened.

Record consequential activities when they occur or reasonably close to that time. If later reconstruction is necessary, identify it as reconstruction rather than creating the appearance of a contemporaneous record.

More Documentation Can Become Less Defensible If Nobody Can Identify the Authoritative Record

Keeping everything sounds safe. In practice, uncontrolled duplication can create its own problems.

Imagine finding seven datasets named “final,” five slightly different analysis scripts, screenshots with no dates, three conflicting codebooks, and an email saying “use the new version” without identifying which version that means. Nothing was deleted, yet the research history is still unclear.

Good documentation therefore also requires organization, naming, version control, relationships among records, and identification of authoritative files. Quantity cannot substitute for provenance.

Some Evidence Requires Stronger Documentation Because the Stakes Are Higher

The appropriate burden of documentation increases when errors or disputes could have substantial consequences, when procedures are difficult to reproduce, when records support regulatory or safety decisions, or when applicable policies explicitly prescribe documentation.

FDA requirements for regulated pharmaceutical activities, for example, demand substantially more formal documentation controls than many ordinary academic projects. Researchers should not imitate regulatory paperwork merely to appear rigorous, but neither should they use general academic norms when binding requirements apply.

A Defensible Record Does Not Mean a Record That Guarantees You Will Win an Argument

Documentation can establish what you did without establishing that what you did was correct.

A contemporaneous record may show that you used a particular exclusion rule, transformation, or statistical model. Another researcher may still conclude that the choice was methodologically inappropriate. That is a scientific disagreement, not necessarily a documentation failure.

You need both. Meticulous documentation cannot rescue poor methodology, but poor documentation can make sound methodology surprisingly difficult to demonstrate.

The Research Record Becomes Especially Important When Work Is Questioned

Under U.S. Public Health Service research-misconduct regulations, destruction, absence, or failure to provide research records adequately documenting questioned research can constitute evidence of misconduct when specified evidentiary conditions are met. This does not mean that every missing file implies misconduct. The regulation expressly establishes conditions that must be demonstrated.

The broader lesson is narrower and useful: once research is disputed, documentation that once seemed administrative can become primary evidence of what actually happened.

How Much Documentation Would Be Enough to Explain One Published Number?

The team does not need a narrative describing every click made during data cleaning. It needs sufficient evidence to account for the consequential transition from 800 collected records to 742 analyzed records.

That is the proportionality principle in practice.

Defensible Documentation Is Neither Minimalism nor Total Surveillance

Use the “Could Someone Reconstruct This?” Test

Select the most consequential parts of your study and imagine that you are unavailable. Could another knowledgeable person use the surviving records to determine what actually happened?

Do not ask whether that person could guess correctly. Ask whether the records provide evidence.

This framework also helps determine which research decisions deserve preservation even when they never appear in the paper. The more a choice affects the evidentiary chain, the more difficult it should be for that choice to disappear without a trace.

Finally, test your documentation before the project ends. Waiting until a manuscript is questioned is an unusually expensive way to discover that nobody knows which file produced Figure 3.

Can Your Documentation Establish What You Actually Did?

Common Questions About How Much Research Documentation Is Enough

Document Enough to Replace Memory With Evidence

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.