Should Researchers Collect and Document Reasons for Missing Data?

Should You Record Why a Research Measurement Is Missing?

Suppose you are conducting a longitudinal study of students' academic performance. At the end of the semester, several participants have no final assessment scores. Some were absent because of scheduling conflicts, others discontinued participation, and a few experienced technical problems during the assessment.

You could record all these observations simply as missing values. After all, the statistical dataset may represent each unavailable score with the same missing-value indicator.

However, the circumstances surrounding missingness may help explain whether the problem was preventable, whether some outcomes can still be obtained, and which assumptions are plausible during analysis.

Should researchers systematically document those circumstances? How detailed should the records be, and can the stated reasons establish whether data are missing completely at random, missing at random, or missing not at random?

Document Missingness Reasons, but Do Not Treat Them as Proof of a Missing-Data Mechanism

How to Document Missingness Without Overinterpreting Its Causes

Why are reasons for missing data scientifically useful?

Missing-data documentation provides information about how the observation process operated. It can help researchers distinguish a technical failure from a missed appointment, identify recurring collection problems, and determine whether additional follow-up is appropriate.

For example, a missing examination score caused by a failed database transfer may be recoverable from an authorized institutional record. A score that was never generated because the student did not take the examination presents a different problem.

Without documentation, these situations may appear identical in the analytical dataset.

Recorded reasons may also help researchers evaluate whether missingness differs across study groups or measurement occasions. This is especially important when the observation process may depend on exposure, participant characteristics, or outcomes.

The National Research Council's work on missing data in clinical trials emphasizes prevention, continued outcome collection where appropriate, and careful attention to the circumstances producing incomplete information.

What kinds of reasons should researchers record?

The categories should reflect the study design, population, measurement procedures, and ethical requirements. A practical classification may distinguish the following situations.

These categories are illustrative rather than a universal coding standard. Researchers should adapt them to the actual collection process and define them in a data dictionary.

Why distinguish the immediate reason from the underlying cause?

The immediate reason for missingness is not always its underlying causal explanation.

For example, a researcher may record that a student missed an examination because the student did not attend the scheduled session. That documents the observation event but does not explain why the student was absent.

The absence might reflect illness, transportation problems, academic difficulties, or other circumstances.

Similarly, a participant may report being too busy to complete a questionnaire. That explanation may be accurate, but it does not establish whether workload is associated with the missing outcome.

Researchers should therefore distinguish observed collection events, voluntarily provided explanations, and hypotheses about the underlying missingness process.

This distinction prevents administrative categories from being interpreted as verified causal mechanisms.

Can recorded reasons establish MCAR, MAR, or MNAR?

No. The conventional missing-data mechanisms are defined by statistical relationships between missingness and observed or unobserved data.

Under missing completely at random (MCAR), missingness is independent of the relevant data values. Under missing at random (MAR), missingness does not additionally depend on missing values after conditioning on appropriate observed information. Under missing not at random (MNAR), dependence on unobserved information remains.

A recorded technical failure may make MCAR plausible if the failure was genuinely independent of participant characteristics and outcomes. However, a system failure concentrated among participants with poor internet access may not satisfy that assumption.

Likewise, a documented refusal does not automatically establish MNAR. The refusal may depend on observed circumstances, unobserved outcomes, or both.

Rubin's foundational framework makes clear that missing-data mechanisms concern the relationship between observation and data values, not merely the labels assigned to nonresponse events.

Researchers should use recorded reasons to inform their assumptions without claiming that those reasons prove the assumptions.

Should reasons be recorded at the participant or measurement level?

The appropriate level depends on the missingness event.

For item nonresponse, the relevant record may concern a specific question. For a missed assessment, it may concern an entire measurement occasion. For participant attrition, it may concern several future observations.

A participant-level label such as "withdrawn" may be insufficient when the participant completed baseline measurements, declined one questionnaire, and subsequently permitted an administrative outcome to be collected.

Researchers should therefore distinguish participant status from the availability of each measurement.

Where feasible, records should identify the relevant variable or assessment, scheduled date, observation status, documented reason, and whether a valid value was later recovered.

This allows analysts to distinguish intermittent nonresponse from permanent loss of follow-up.

How should unknown reasons be handled?

Unknown reasons should be recorded explicitly rather than replaced with assumptions.

For example, if a participant does not respond to an authorized follow-up invitation, the research team may know that contact was unsuccessful but not why.

Assigning the reason "lack of interest" or "poor performance" would introduce unsupported information into the dataset.

An unknown category may be necessary even in studies with extensive monitoring procedures.

Researchers can report the proportion of missing observations with known and unknown reasons, recognizing that the availability of reasons may itself differ across groups.

Can documenting reasons help prevent additional missingness?

Yes. Systematic records may reveal recurring problems that can be addressed while data collection is ongoing.

For example, repeated technical failures on a particular device type may justify software testing. Frequent scheduling conflicts may indicate that assessment windows are too restrictive.

However, procedural changes should preserve the intended outcome definition and measurement comparability.

If one study group receives substantially different follow-up opportunities after a problem is identified, researchers should document the change and assess its implications.

Missingness monitoring should therefore be connected to the study's design-stage missing-data prevention strategy.

Should researchers ask participants why they declined a question?

Not automatically. Participants may decline to disclose information without providing an explanation.

Requesting reasons can be appropriate when voluntary, minimally burdensome, and relevant to improving the study. However, repeated questioning may feel intrusive or coercive, especially for sensitive topics.

Researchers should avoid making participation or withdrawal conditional on explaining a refusal.

Where an optional reason is collected, the consent materials and data-management plan should address how that information will be used and protected.

The scientific value of additional information does not override participant autonomy.

How can missingness records support statistical analysis?

Documented reasons may help researchers identify observed variables associated with nonresponse, distinguish recoverable records from unavailable outcomes, and develop plausible sensitivity scenarios.

For example, if missed assessments are frequently associated with documented scheduling conflicts, the team may examine whether those conflicts are related to observed baseline characteristics.

However, missingness categories should not automatically be included in an imputation model without considering how they were generated and whether they are observed for all relevant participants.

Some reason variables may be consequences of exposure or selection processes. Their analytical use requires a defensible causal and statistical rationale.

When MNAR is plausible, recorded reasons may inform sensitivity parameters, but the values of those parameters generally remain assumption-dependent.

Why should reasons be compared across study groups?

Different groups may experience different barriers to outcome collection.

For example, an intensive intervention may increase scheduling conflicts, while a comparison condition may have fewer contact opportunities.

Researchers should report missingness counts and available reasons by group and measurement occasion when relevant.

Nevertheless, a higher frequency of a particular reason does not necessarily establish that the intervention caused it. Baseline differences, collection procedures, and incomplete reason reporting may contribute.

Understanding why missingness patterns differ between study groups helps researchers interpret these records without overclaiming.

How should missingness reasons be reported in a manuscript?

The methods section should describe how missingness was recorded, which categories were used, and whether reasons were obtained from administrative events, researcher observations, or participant reports.

The results should present relevant counts and percentages, distinguishing unavailable outcomes from other missing measurements.

For longitudinal studies, participant flow diagrams or tables may help readers understand missed occasions, withdrawal, and outcome availability.

The discussion should evaluate whether the documented reasons support the plausibility of the primary analytical assumptions and identify uncertainty that remains.

Reporting guidance such as STROBE and CONSORT supports transparent descriptions of missing data and participant flow, although the specific requirements depend on the study design.

Researchers should avoid stating that missingness was random solely because participants reported scheduling problems or other apparently unrelated reasons.

Creating a Missing-Data Reason Log for a Longitudinal Study

The numbers are hypothetical. They demonstrate how a reason log can support operational recovery and transparent reporting without converting administrative explanations into verified statistical mechanisms.

Common Misconceptions About Documenting Missingness

Create a Missingness Log That Supports Both Research Operations and Interpretation

Before collection begins, establish a consistent set of missingness categories and specify how the research team will record relevant events.

Separate the analytical missing-value indicator from the operational reason log. A missing value indicates that the required measurement is unavailable; the reason log documents what is known about the circumstances.

Where appropriate, establish a process for recovering critical outcomes from valid backup sources.

The missingness log should be sufficiently detailed for audit and interpretation but should not collect unnecessary sensitive information. Its purpose is to document the observation process, not to create a speculative explanation for every absent value.

Before Recording Reasons for Missing Data

Frequently Asked Questions About Missing-Data Reasons

Document What Is Known About Missingness Without Inventing Its Cause

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.