01 · The Question
If a Decision Never Appears in the Paper, Does It Still Need a Record?
A research paper presents a deliberately compressed version of a much longer process. It may tell readers which participants were analyzed, which model was used, or which coding framework guided interpretation, but it rarely reports every discussion and intermediate decision that led there.
During the actual project, researchers may reject analytical approaches, resolve ambiguous cases, revise coding rules, change procedures, exclude unusable observations, correct errors, or decide that a seemingly promising result should not be pursued.
Most of those decisions will never become sentences in the published article. Some should nevertheless remain part of the underlying research record.
03 · What You Need to Know
The Final Paper and the Research Record Serve Different Purposes
A journal article is designed to communicate a study within disciplinary and editorial constraints. The underlying research record serves a different function: it preserves evidence of how the work actually developed.
That difference explains why the publication cannot be used as the sole test of what deserves documentation. A methodological decision may be too granular for the article yet highly consequential if someone later needs to determine how a result was produced.
Document Decisions That Could Change the Evidence or Its Interpretation
A useful threshold is materiality. Ask whether a reasonable alternative decision could have changed the dataset, analysis, result, interpretation, or credibility of the research.
Depending on the study, consequential decisions may include:
excluding or retaining observations, participants, samples, cases, or variables;
changing an instrument, protocol, recruitment procedure, or sampling strategy;
resolving ambiguous eligibility or classification cases;
altering coding rules or a qualitative coding framework;
choosing among plausible analytical specifications;
transforming, recoding, aggregating, imputing, or otherwise processing data;
departing from a preregistration, protocol, or original analysis plan;
rerunning an experiment, assay, model, or analysis after an unexpected result;
deciding that an apparent anomaly is an error rather than a genuine observation;
changing an interpretation after contradictory evidence appears.
The exact threshold varies by methodology. What matters is the decision's effect on the evidentiary chain, not whether it eventually earns space in the manuscript.
Preserve the Rationale, Not Merely the Outcome
A note saying “Cases 17–23 excluded” establishes what happened but not why. If the exclusion later becomes important, the missing rationale may be precisely what another researcher needs to evaluate.
Useful decision documentation therefore usually identifies the decision, its timing, the person or team responsible where relevant, and enough rationale to understand why the choice was made. It may also identify the evidence, protocol provision, analytical criterion, or discussion on which the decision was based.
This need not become a miniature essay. “Excluded because participant did not satisfy the prespecified age criterion in protocol v2.1” may be far more useful than several paragraphs written years later from memory.
Timing Changes the Evidentiary Value of a Decision Record
A rationale documented when a decision is made generally provides stronger evidence than a rationale reconstructed after the result becomes controversial.
Regulatory good-documentation frameworks illustrate the principle. FDA materials describe contemporaneous recording as recording results, measurements, and data when work is performed. These requirements arise in particular regulated environments and should not be treated as universal rules for all academic research, but the underlying logic is broadly useful: contemporaneous records reduce dependence on retrospective memory.
If a decision must be reconstructed later, label the reconstruction honestly. Do not make a later explanation appear to have existed at the time of the original decision.
Unsuccessful and Rejected Paths Can Sometimes Matter
Suppose a researcher tries three reasonable analytical specifications before selecting one. Does every discarded output need permanent preservation? Not necessarily.
But if the choice among those analyses could materially affect the conclusion, documentation of the analytical history may be important. The same applies when a procedure is abandoned because it failed, an observation is rejected as invalid, or an interpretation changes after contradictory evidence appears.
The aim is not to preserve an archaeological site containing every abandoned idea. It is to prevent the research record from becoming a retrospective story in which only the successful path appears ever to have existed.
Some Decisions Belong in the Paper as Well as the Record
Keeping a decision in the underlying record does not excuse researchers from reporting information that readers need to evaluate the study.
If a protocol deviation, exclusion rule, analytical choice, methodological change, or other decision is necessary for understanding or assessing the published work, applicable reporting standards and journal requirements may require disclosure in the manuscript. A private record is not a substitute for transparent reporting.
The distinction is therefore not “paper or record.” Some consequential decisions appropriately belong in both.
Publication record
Reports the methods, decisions, results, limitations, and other information readers need under applicable scholarly and reporting standards.
Underlying research record
Preserves a more detailed evidentiary history that can include consequential decisions and supporting documentation not reproduced in the publication.
Decision Records Help Preserve Provenance
NIH defines metadata as information that helps make scientific data interpretable and reusable, including methodology, data provenance, and data transformations. Its repository guidance similarly identifies provenance mechanisms that record the origin, chain of custody, and modifications to datasets and metadata.
A consequential decision can form part of that provenance. If 30 observations disappear between the source dataset and the analytical dataset, for example, the ability to identify the exclusion rule and when it was applied can be essential to understanding the transformation.
This is why a broader research audit trail often includes decisions as well as data.
Decision Records Can Protect Researchers as Well as Challenge Them
Documentation is sometimes imagined primarily as evidence that can be used against a researcher. That is only half the picture.
A contemporaneous decision record may demonstrate that an exclusion rule preceded knowledge of the final result, that a methodological change responded to an equipment failure rather than an inconvenient finding, or that an unusual analytical choice had a defensible rationale.
Conversely, missing documentation can make legitimate decisions much harder to establish later. Under U.S. Public Health Service research-misconduct regulations, for example, destruction, absence, or failure to provide research records adequately documenting questioned research can become evidence in specified circumstances. That regulatory rule is context-specific, but it illustrates the evidentiary importance records can acquire when research is questioned.
Not Every Decision Needs the Same Level of Documentation
A project can generate hundreds of minor choices. Treating every one as equally important would create an unusable record.
Documentation should therefore be proportional. Decisions with greater potential to alter the evidence, analysis, interpretation, compliance status, or reproducibility of the work generally deserve stronger documentation. Routine choices with no meaningful effect may require little or no separate record.
The harder cases are decisions that seem trivial when they are made . A practical strategy is to document a choice when you would have difficulty explaining an important later consequence without knowing why that choice occurred.
Watch Out
Do not retrospectively manufacture rationales for decisions after seeing their consequences. If the original rationale was not documented, distinguish clearly between what contemporaneous records establish and what you now remember, infer, or reconstruct.
04 · A Practical Example
A Decision Can Matter Even When the Paper Gives It Only One Sentence
Hypothetical Example
An ambiguous group of survey responses
A research team collects 900 survey responses. During cleaning, it discovers 34 submissions completed unusually quickly. The protocol specifies several quality checks but does not establish an automatic minimum completion time.
Problem
The team must decide whether unusually fast completion alone is sufficient grounds for exclusion.
Decision
Before running the primary outcome analysis, the team decides not to exclude cases based solely on completion time. Instead, it applies the prespecified quality criteria.
Record
A dated decision log identifies the issue, participants affected, evidence considered, people involved, decision, and rationale.
Publication
The methods section reports the actual exclusion criteria used. It does not reproduce the entire internal discussion.
Later question
A reviewer of a follow-up study asks whether fast respondents were removed after the researchers saw their effect on the results. The contemporaneous record shows when the decision occurred and what rationale supported it.
The unpublished decision record provides information that the concise published method cannot. It establishes chronology and rationale without requiring the researchers to remember the discussion years later.
06 · What This Means for You
Ask Whether Losing the Decision Would Break the Research Story
You do not need to create a bureaucratic decision log for every thought your research team has. Concentrate on choices that materially shape what evidence enters the study, what happens to it, and what conclusions can be drawn from it.
A simple decision framework
If the decision changes who or what enters the analysis
Document the decision, relevant rule or rationale, and timing.
If the decision changes how evidence is coded, transformed, or analyzed
Preserve enough information to understand the change and its effect on the workflow.
If the decision departs from an approved or prespecified plan
Document the departure and rationale, and determine whether it also requires formal approval or disclosure.
If reasonable researchers could make different choices with meaningful consequences
Consider preserving why your team chose one approach.
If the choice is routine and has no consequential effect
A separate permanent decision record may not be necessary unless applicable procedures require one.
Where possible, integrate decision documentation into tools you already use: electronic laboratory notebooks, version-control systems, analysis scripts, coding memos, deviation logs, project records, or structured decision logs. Documentation is more likely to be reliable when it is part of the workflow rather than an administrative reconstruction at the end.
The resulting records should contribute to an adequate research record , not simply create more files.
07 · A Quick Checklist
Before You Let a Research Decision Disappear
Ask whether the decision:
Changes which participants, observations, samples, variables, or evidence enter the research or analysis.
Changes how important evidence is collected, coded, cleaned, transformed, analyzed, or interpreted.
Represents a departure from a protocol, preregistration, approved procedure, or analysis plan.
Could materially affect the findings or how another researcher evaluates them.
Would be difficult to explain later without knowing the rationale and timing.
Has been documented close enough to the decision that the record does not depend primarily on later recollection.
Needs disclosure in the publication, report, ethics documentation, or another formal record in addition to internal preservation.
11 · Cite this Guide
How to Cite This Guide
This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.
Recommended (Field Guide)
APA
MLA
Chicago
Copy Citation
