Can Loss to Follow-Up Introduce Selection Bias?

What Happens to Your Findings When Some Participants Disappear Before the Final Assessment?

You conduct a semester-long randomized study evaluating an AI-supported learning platform. At the beginning, 200 students are assigned to intervention and comparison groups. By the final examination, however, some students have withdrawn from the course, stopped responding, or become unavailable for assessment.

You analyze the students with complete outcome data and find that the intervention group performs better.

But what if students who struggled academically were more likely to disappear from one group? What if those who found the platform difficult to use were disproportionately absent from the final assessment?

The remaining participants may no longer provide the comparison established by the original design.

Loss to follow-up becomes a selection problem when the availability of outcome data depends on characteristics or processes that affect the relationship researchers intend to estimate. The difficulty is determining when that selection produces bias and what can reasonably be done about it.

Can Losing Participants During Follow-Up Bias a Study?

How Follow-Up Loss Changes the Sample and Potentially Distorts Research Findings

What Is Loss to Follow-Up?

Loss to follow-up occurs when participants initially enrolled or observed in a study no longer provide information required at a later assessment.

For example, students may stop responding to surveys, withdraw from a course, transfer institutions, or become unreachable before a scheduled outcome measurement.

In clinical and population research, participants may relocate, discontinue contact, or otherwise become unavailable for outcome ascertainment.

Loss to follow-up should be distinguished from other forms of missing data. A participant may remain enrolled but skip one questionnaire item, while another may withdraw completely from all subsequent assessments.

The distinction matters because the missingness mechanism and available information may differ.

Why Is Loss to Follow-Up a Selection Process?

When researchers analyze only participants with observed follow-up outcomes, they condition their analysis on remaining observable.

The resulting sample may differ from the original study population.

Suppose students with lower academic preparation are more likely to withdraw. If preparation also predicts final achievement, the observed sample may overrepresent stronger students.

Whether this produces bias depends on the estimand and how missingness relates to treatment, exposure, outcomes, and other variables.

Selection bias is not established merely because the final sample is smaller. It arises when selection changes the relevant relationship or estimate systematically.

How Is Loss to Follow-Up Different From Ordinary Missing Data?

Loss to follow-up is one mechanism through which data become missing. Missing data is the broader category.

These categories can overlap, but their causes may differ substantially.

Researchers should examine how missing data threaten internal validity rather than treating every incomplete observation as the same problem.

What Is Attrition Bias?

Attrition bias refers to systematic distortion associated with participants leaving a study or becoming unavailable for outcome measurement.

In randomized experiments, attrition may undermine the comparability created by treatment assignment when the analyzed participants are selected through postassignment processes.

For example, an intervention may cause some students to disengage, and disengagement may be associated with poorer potential outcomes.

If those students are excluded, the observed treatment effect may differ from the effect among everyone originally randomized.

However, attrition does not necessarily bias every estimate. The relevant missingness assumptions and analytical method determine whether the intended effect remains identifiable.

Does Differential Attrition Automatically Mean Selection Bias?

No. Differential attrition occurs when the proportion lost differs between study groups.

For example, 10% of the intervention group and 25% of the comparison group may lack final outcomes.

This imbalance raises concerns, particularly if missingness is associated with potential outcomes.

However, unequal loss rates alone do not establish the magnitude, direction, or even existence of bias.

Conversely, identical loss rates do not guarantee an unbiased comparison. The characteristics of participants lost may differ across groups despite equal percentages.

Researchers should examine how differential attrition threatens internal validity through the actual selection mechanism.

Can Equal Attrition Rates Still Produce Bias?

Yes. Suppose 20% of participants are lost from each treatment group.

In the intervention group, students with low potential achievement may be disproportionately missing. In the comparison group, missingness may occur primarily among students with high potential achievement.

The same overall attrition rate could produce a misleading observed treatment difference.

The reverse patterns could produce bias in the opposite direction.

Researchers therefore need more information than the percentage of missing outcomes.

Attrition rates are useful descriptive indicators, but they do not establish the causal consequences of selection.

How Can Conditioning on Follow-Up Completion Create Collider Bias?

Suppose treatment assignment influences whether a student remains in a study, while prior academic preparation also influences retention.

Follow-up completion is then a common effect of treatment assignment and academic preparation.

Conditioning on completion can create an association between assignment and preparation among the retained participants, even though randomization made them independent in expectation before selection.

If preparation affects the outcome, the retained treatment groups may no longer be causally comparable.

This is one possible form of collider stratification bias.

Not every loss-to-follow-up mechanism has this structure. Researchers should examine whether conditioning on participation opens a noncausal association rather than assuming every exclusion operates identically.

What Is Informative Censoring?

Censoring occurs when information about an outcome or event time becomes incomplete after a specified point.

In time-to-event analysis, informative censoring is a concern when the censoring process carries information about event risk that is not adequately accounted for by the analytical assumptions.

For example, students who discontinue a program because of serious academic difficulties may differ in their subsequent probability of course failure from students who remain under observation.

Standard survival methods often require independent or conditionally independent censoring assumptions.

If these assumptions are violated, estimates may be biased.

Informative censoring is related to loss-to-follow-up selection bias, although the appropriate terminology and analysis depend on the outcome and design.

What Do MCAR, MAR, and MNAR Mean?

Missing-data terminology helps describe assumptions about why observations are unavailable.

These mechanisms are assumptions about the joint data-generating process, not labels that can always be determined from the observed data.

In particular, MAR and MNAR generally cannot be distinguished conclusively from observed data alone.

The appropriate analysis depends on the estimand, data structure, and plausibility of the assumptions.

Can Complete-Case Analysis Be Unbiased?

Sometimes. Complete-case analysis uses only observations with the variables required for the analysis.

It can produce valid estimates under particular conditions, which may be less restrictive than MCAR for some models and estimands.

However, those conditions should be justified rather than assumed.

For example, excluding students whose final outcomes are missing may distort an average treatment effect if missingness depends on treatment and outcome-related characteristics.

Complete-case analysis also commonly reduces precision because available information is discarded.

Researchers should not equate a complete dataset with an unbiased analytical sample.

Can Intention-to-Treat Analysis Prevent Attrition Bias?

Intention-to-treat analysis preserves participants' original treatment assignments when estimating the effect of assignment.

However, simply labeling an analysis intention-to-treat does not resolve missing outcome data.

If outcomes are unavailable for some randomized participants, researchers still need a defensible strategy for estimation and sensitivity analysis.

Imputing missing values without suitable assumptions or excluding participants while retaining the intention-to-treat label does not automatically preserve validity.

Researchers should distinguish the treatment-assignment estimand from the methods used to address missing outcomes.

What Is Inverse Probability of Censoring Weighting?

Inverse probability of censoring weighting, often abbreviated IPCW, attempts to account for selective observation by weighting retained participants according to their estimated probabilities of remaining observable.

Under appropriate assumptions, participants who resemble individuals more likely to be lost receive greater weight.

The method requires suitable models for observation probabilities, relevant measured predictors, and positivity.

It also requires assumptions that censoring is conditionally independent of the relevant potential outcomes given the measured information.

Extreme weights can create unstable estimates, and unmeasured causes of both censoring and outcomes may remain problematic.

IPCW is therefore an analytical option, not an automatic correction for attrition.

Can Multiple Imputation Address Loss to Follow-Up?

Multiple imputation creates several completed datasets by drawing plausible values for missing observations from a specified statistical model.

Each dataset is analyzed, and the results are combined to account for uncertainty associated with missing values.

Under suitable assumptions, including a defensible imputation model and missingness mechanism, the approach can use information from participants with incomplete records.

However, imputation does not recover the actual unobserved outcomes or prove that missingness is ignorable.

Researchers should include appropriate predictors, respect the study design, and evaluate sensitivity to alternative assumptions.

Imputation should not be treated as a substitute for preventing unnecessary follow-up loss.

What Is a Sensitivity Analysis for Missing Outcomes?

Sensitivity analysis investigates how conclusions change when assumptions about missing outcomes are varied.

For example, researchers may examine scenarios in which participants lost from the intervention group have systematically poorer outcomes than predicted under an MAR model.

Pattern-mixture models, selection models, delta-adjustment approaches, and bounding methods can support different sensitivity questions.

These methods require careful specification and should be matched to the estimand and design.

A sensitivity analysis does not reveal the true missing outcomes. It shows how dependent the conclusions are on assumptions that cannot be verified directly.

Can Follow-Up Procedures Reduce Selection Bias?

Prospective retention planning can reduce the amount of missing outcome information.

Researchers may use appropriate contact schedules, accessible assessment formats, updated contact details, and follow-up opportunities that do not depend on intervention adherence.

For example, students who stop using a learning platform may still be invited to complete the final assessment, subject to consent and ethical requirements.

This separation helps prevent the outcome dataset from consisting only of successful or highly engaged intervention users.

Retention procedures should be proportionate, respectful, and consistent with participants' rights, including withdrawal.

High retention is desirable, but it does not by itself prove that remaining missingness is harmless.

Should Researchers Collect Outcomes After Intervention Discontinuation?

Often, yes, when the estimand concerns the effect of assignment or an intervention policy and continued outcome collection is ethically permitted.

A participant who stops using the intervention may still provide valuable outcome information.

For example, a student who discontinues an AI-supported learning platform can still complete a course examination.

Excluding such students may change the analysis from the effect of assignment to a selected comparison of adherent participants.

However, continued contact must respect consent conditions and any decision to withdraw from research.

Researchers should distinguish discontinuing treatment from withdrawing permission for further data collection.

How Should Loss to Follow-Up Be Reported?

Researchers should report the number initially enrolled, the number with outcome data at each relevant time point, and reasons for missing follow-up where known.

In comparative studies, these figures should be presented separately by group.

Describe observed characteristics associated with missingness, analytical assumptions, methods used to handle incomplete outcomes, and relevant sensitivity analyses.

For randomized trials, current CONSORT guidance emphasizes transparent participant flow and reporting of missing outcome data.

For observational studies, STROBE provides relevant reporting principles.

Researchers should avoid interpreting nonsignificant differences between retained and lost participants as proof that attrition was random.

How Selective Follow-Up Can Reverse the Apparent Effect of an Intervention

The example does not show that attrition normally reverses treatment effects. It demonstrates that equal follow-up rates cannot guarantee unbiased results.

Because the missing outcomes are unobserved, the actual effect cannot be recovered from the complete-case means alone without further assumptions or information.

Common Misconceptions About Loss to Follow-Up

How to Prevent and Evaluate Follow-Up Selection Bias

Plan outcome follow-up before the intervention begins. Specify the estimand, identify plausible reasons for missing outcomes, and establish procedures for retaining participants without compromising voluntary participation.

When outcomes are missing, investigate the selection mechanism rather than assuming that complete cases are representative.

For educational technology research, monitor whether disengagement, limited connectivity, course withdrawal, or platform difficulties influence outcome availability.

These factors may be closely connected to intervention effectiveness and should not disappear from the analysis merely because the affected students are harder to follow.

Loss-to-Follow-Up Selection Bias Checklist

Frequently Asked Questions About Follow-Up Attrition

Follow-Up Loss Matters Because It Can Change the Comparison Being Estimated

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.