Should Missing-Data Handling Be Planned Before Data Collection?

Should You Decide How to Handle Missing Data Before You Know What Will Be Missing?

You are preparing a research study involving questionnaires, assessments, interviews, institutional records, or repeated measurements. You expect to collect the information needed for your planned analyses, but some observations may be incomplete.

Participants may skip sensitive questions, miss follow-up assessments, withdraw from the study, or become unreachable. Administrative records may lack particular variables, and technical failures may prevent measurements from being recorded.

Should you wait until data collection is complete before deciding how to handle these missing values? After all, you cannot know precisely which observations will be missing.

The difficulty is that missing-data handling is not merely a statistical decision made after collection. The design influences why information becomes missing, which variables can help explain missingness, and whether the intended analysis remains credible.

Researchers should therefore consider missingness before it occurs, even though the final missing-data patterns cannot be known in advance.

Yes, Missing-Data Prevention and Analytical Handling Should Be Planned Prospectively

Planning Missing-Data Prevention, Analysis, and Sensitivity Assessment

What Are Missing Data?

Missing data occur when a value intended for collection or analysis is unavailable for a particular observation.

For example, a student may provide baseline writing scores but fail to complete the final assessment. A questionnaire respondent may leave an income item unanswered. A school record may lack a relevant demographic variable.

Missingness can occur at the item, measurement occasion, participant, cluster, or data-source level.

These patterns have different implications. Missing one questionnaire item is not necessarily equivalent to losing all follow-up outcomes for a participant.

Researchers should distinguish missing information from values that are genuinely inapplicable, structurally unavailable, or outside the study's defined measurement requirements.

Why Can Missing Data Threaten Research Validity?

Missing observations reduce the information available for analysis and may increase statistical uncertainty.

More importantly, missingness can introduce bias when the availability of observations is related to the outcome, exposure, or other variables in ways not adequately addressed by the analytical method.

For example, students who struggle academically may be less likely to complete a final assessment.

If researchers analyze only students with complete outcomes, the resulting estimates may misrepresent the intended population.

However, missingness does not always produce substantial bias. Its consequences depend on the missingness mechanism, research design, target quantity, and analytical assumptions.

What Are MCAR, MAR, and MNAR?

Missing-data theory commonly distinguishes three mechanisms.

These categories describe assumptions about a missing-data process, not labels that can always be conclusively assigned through a statistical test.

In particular, MAR and MNAR generally cannot be distinguished definitively using only the observed data.

Can Researchers Test Whether Data Are MCAR, MAR, or MNAR?

Some statistical procedures can evaluate implications of MCAR under particular assumptions.

For example, Little's MCAR test examines whether observed patterns are compatible with a specified MCAR framework.

However, failing to reject MCAR does not prove that data are missing completely at random.

Likewise, observing associations between missingness and recorded variables may provide evidence against MCAR but does not automatically establish MAR.

The distinction between MAR and MNAR depends on relationships involving unobserved information, which cannot generally be established from observed records alone.

Researchers should use substantive knowledge, study procedures, available predictors, and sensitivity analyses to evaluate plausible mechanisms.

What Types of Missingness Should Be Anticipated During Design?

Prevention strategies should be proportionate to the research setting and consistent with participant autonomy and approved procedures.

Why Is Preventing Missing Data Often Better Than Repairing Them Statistically?

Statistical methods cannot fully replace information that was never observed.

Methods such as multiple imputation use observed information and assumptions to represent uncertainty about missing values.

However, their validity depends on whether those assumptions are appropriate.

Reducing avoidable missingness through accessible procedures, clear instruments, appropriate scheduling, and reliable data systems can improve the evidence available for analysis.

Nevertheless, prevention should not compromise voluntary participation or create inappropriate pressure to disclose sensitive information.

Good research design balances data completeness with participant rights and reasonable burden.

Should Researchers Plan Follow-Up and Retention Procedures?

Yes, particularly in longitudinal and intervention studies.

Retention planning may include appropriate reminders, alternative assessment arrangements, updated contact procedures, and accessible participation options.

However, retention procedures should remain consistent with consent and ethics requirements.

Researchers should also distinguish discontinuation of an intervention from discontinuation of outcome assessment.

In some studies, participants who stop receiving an intervention may still provide valuable follow-up outcomes when continued assessment is authorized and acceptable.

The intended estimand and treatment of intercurrent events should inform these decisions.

What Is Complete-Case Analysis?

Complete-case analysis uses observations with all variables required for the specified analysis.

It is straightforward to implement, but it can reduce precision and introduce bias under some missingness mechanisms.

However, complete-case analysis is not universally invalid whenever data are not MCAR.

Its validity depends on the analysis, estimand, and relationships among missingness, outcomes, exposures, and covariates.

Researchers should not assume that excluding incomplete records is harmless merely because the remaining sample is large.

The decision should follow a defensible missing-data strategy rather than a software default.

What Is Multiple Imputation?

Multiple imputation generates several completed datasets by drawing plausible values for missing observations from an imputation model.

The intended analysis is performed separately on each dataset, and estimates are combined using appropriate pooling procedures that reflect within- and between-imputation uncertainty.

Multiple imputation can be useful when the imputation model is appropriate and the relevant missingness assumptions are defensible.

However, it does not automatically remove missing-data bias.

The imputation model should reflect the substantive analysis, variable types, relevant interactions, and important predictors of missingness and outcomes.

Researchers should not use imputation mechanically without considering model compatibility and the study's data structure.

What Are Likelihood-Based Missing-Data Methods?

Likelihood-based methods estimate model parameters using the available observations under specified distributional and missingness assumptions.

For example, certain longitudinal mixed-effects models can use incomplete repeated-measures records without requiring every participant to have measurements at every occasion.

However, these methods do not automatically solve informative dropout.

Their validity depends on assumptions concerning the outcome model and missingness process.

Researchers should distinguish the ability of software to process incomplete records from evidence that the resulting estimates are unbiased.

Should Missing Values Ever Be Replaced With the Mean?

Simple mean imputation is generally problematic for inferential analysis because it can distort variability, associations, and uncertainty.

Replacing missing values with a single fixed estimate treats the imputed information as more certain than it is.

Similarly, carrying the last observation forward in longitudinal studies can impose unrealistic assumptions about how outcomes change.

These procedures should not be treated as default solutions merely because they are easy to implement.

Researchers should choose methods according to the estimand, data structure, and plausible missingness mechanism.

Which Variables Should Be Collected to Support Missing-Data Analysis?

Researchers should consider collecting baseline and follow-up variables that are scientifically relevant predictors of missingness and outcomes.

For example, a longitudinal educational study might record baseline academic performance, digital access, and other justified characteristics associated with follow-up participation.

Such variables may help support an MAR-based analysis when the relevant assumptions are plausible.

However, collecting additional predictors does not guarantee MAR, and unnecessary sensitive data should not be collected solely because they might be useful.

Variable selection should balance analytical value, privacy, participant burden, and data minimization requirements.

Should Missing-Data Methods Be Specified in the SAP?

Yes, when missingness could materially affect the primary or important secondary analyses.

The statistical analysis plan should describe the anticipated missingness patterns, primary analytical method, relevant assumptions, and procedures for assessing missing-data consequences.

It may also specify the imputation model, auxiliary variables, number of imputations or criteria for determining it, and pooling procedures where appropriate.

However, some implementation details may require refinement when the observed missingness structure becomes known.

Such changes should be justified and documented, particularly in confirmatory research.

The broader statistical analysis plan should remain consistent with the study's estimand and design.

How Does the Estimand Affect Missing-Data Handling?

The estimand defines the quantity the study seeks to estimate, including how relevant intercurrent events are handled.

For example, a trial may seek the effect of treatment assignment regardless of whether participants discontinue the intervention.

In that case, collecting outcomes after treatment discontinuation may be important.

However, treatment discontinuation and missing outcome data are distinct concepts. A participant may discontinue treatment but still provide an observed outcome.

Researchers should define the scientific target before selecting a missing-data method.

The ICH E9(R1) framework emphasizes this relationship in clinical trials.

Can Missingness Affect Randomization?

Randomization can support baseline comparability between assigned groups, but missing outcomes may undermine the comparability of the analyzed observations.

For example, if participants with poorer outcomes are more likely to miss follow-up in one condition, a complete-case comparison may be biased.

Analyzing participants according to randomized assignment does not automatically resolve missing outcome information.

Researchers should therefore plan follow-up and missing-data handling as part of the trial design.

Appropriate analytical methods and sensitivity analyses remain necessary when missingness could affect the estimated treatment effect.

How Does Missingness Affect Observational Research?

Observational studies may experience missingness in exposures, outcomes, confounders, and other variables.

These patterns can affect the target population, confounding adjustment, and the interpretation of associations.

For example, missing socioeconomic information may complicate an analysis intended to address confounding between digital access and academic performance.

Researchers should examine whether the missingness process is related to exposure or outcome characteristics and whether the planned method supports the intended inference.

Imputation does not compensate automatically for unmeasured confounding or selection bias unrelated to the missing-data model.

How Should Missing Data Be Handled in Multilevel Research?

Multilevel studies may have missing observations at several levels, such as students, classrooms, institutions, or research sites.

For example, individual students may miss assessments while an entire school fails to provide follow-up records.

These patterns have different implications for statistical precision and bias.

Missing-data models should account appropriately for the hierarchical structure where relevant.

Single-level imputation that ignores important clustering may distort relationships or uncertainty.

Researchers should consider the possibility of cluster-level dropout during sample-size planning and data collection design.

What Is a Missing-Data Sensitivity Analysis?

A missing-data sensitivity analysis evaluates how conclusions change under alternative assumptions about unobserved values.

For example, a primary analysis may assume missingness is MAR conditional on recorded variables.

Researchers may examine scenarios in which missing outcomes are systematically higher or lower than predicted under that assumption.

Pattern-mixture models, selection models, and tipping-point analyses can support such investigations under appropriate conditions.

However, these analyses depend on assumptions that cannot generally be verified from the observed data alone.

Planning sensitivity analyses during research design helps ensure that the alternative scenarios are scientifically motivated rather than selected to preserve favorable findings.

How Should Missingness Be Described in Research Reports?

Reports should identify the extent and patterns of missing data for important variables and assessment occasions.

For intervention studies, participant flow information can help readers understand recruitment, follow-up, and analytical inclusion.

Researchers should describe the missing-data methods, assumptions, and relevant sensitivity analyses.

Where possible, they should explain how conclusions change under alternative plausible assumptions.

Reporting only the percentage of missing values is insufficient when the missingness mechanism may materially affect the estimates.

Researchers should also distinguish observed findings from values generated through statistical imputation.

What If Missingness Is Much Greater Than Expected?

Unexpectedly high missingness may reduce precision or undermine the assumptions supporting the planned analysis.

Researchers should investigate whether the problem arises from participant burden, measurement procedures, technical failures, or other identifiable causes.

Where data collection is ongoing, approved corrective procedures may improve completeness.

However, analytical changes made after examining outcomes require particular caution.

If missingness prevents the study from supporting its intended inference, researchers should acknowledge that limitation rather than assume a more sophisticated model can recover the missing information.

Can Missing-Data Handling Be Changed After Collection?

Potentially, when unforeseen patterns or analytical problems justify modification.

For example, an imputation model may require adjustment because a variable has substantially different properties from those anticipated.

However, the change should be justified according to the data structure and scientific assumptions, not selected because it produces a more favorable treatment effect.

Researchers should document the timing, rationale, and consequences of consequential changes.

Where appropriate, the original planned analysis and revised analysis can both be reported.

Transparency allows readers to evaluate how analytical decisions affected the conclusions.

Planning for Missing Writing Assessments in a Longitudinal Educational Study

Suppose 15% of participants in the intervention group and 8% in the comparison group have missing final writing scores. These are hypothetical percentages, not empirical findings.

The difference in missingness rates raises questions about the missingness process, but it does not by itself establish that the data are MNAR or that the treatment effect is biased.

Researchers should examine relevant observed predictors, apply the prespecified analysis, and assess plausible departures from its assumptions.

If conclusions change substantially under credible alternative scenarios, the final interpretation should acknowledge that dependence.

Common Misconceptions About Missing-Data Planning

How to Develop a Missing-Data Plan Before Collection Begins

Begin by identifying the variables and measurement occasions most important to the study's primary inference.

Then consider why observations may become missing, how avoidable missingness can be reduced, and which assumptions the planned analytical method requires.

A defensible missing-data plan combines prevention, appropriate statistical methods, and transparent evaluation of assumptions. None of these components can reliably substitute for the others.

Before Finalizing Missing-Data Handling

Frequently Asked Questions About Missing-Data Planning

Missing-Data Planning Begins Before Values Become Missing

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.