03 · What You Need to Know
How Loss to Follow-Up Affects Cohort Validity
What Is Loss to Follow-Up?
Loss to follow-up occurs when researchers can no longer obtain required outcome or other follow-up information about a participant before the intended observation period ends.
For example, a student may stop completing questionnaires, leave the institution without an available outcome record, or become unreachable after changing contact information.
However, not every participant who leaves a cohort's active population is lost to follow-up. A student who experiences the prespecified outcome has reached an endpoint. Another who completes the planned observation period has completed follow-up. Graduation may represent an outcome or competing event, depending on the study's definitions.
Researchers should distinguish these situations rather than treating every departure as missing data.
The rules for starting and ending cohort follow-up determine which departures constitute loss for a particular analysis.
How Is Loss to Follow-Up Calculated?
For a cohort with a common fixed follow-up horizon, researchers may report the proportion of initially eligible participants whose required outcome status is unavailable at that horizon.
For staggered-entry cohorts, survival studies, or outcomes measured at different times, one overall percentage may conceal important information. Researchers may need to report follow-up completeness by wave, exposure group, and relevant time horizon, together with censoring and numbers at risk.
Why Is There No Universal Acceptable Threshold?
Researchers sometimes encounter suggested attrition thresholds in appraisal instruments or disciplinary conventions. Such thresholds may be useful screening prompts, but they should not be treated as universal rules of validity.
A study losing 5% of participants could be seriously biased if nearly all losses occur among exposed participants at particularly high risk of the outcome.
Another study losing 25% might retain useful evidence if reasons for loss are well documented, relevant predictors are measured, and sensitivity analyses indicate that conclusions are reasonably robust under plausible missing-data assumptions.
Neither scenario can be judged conclusively without examining the actual missingness mechanism.
The proportion lost is therefore a warning indicator, not a substitute for evaluating selection, outcome ascertainment, and uncertainty.
When Does Loss to Follow-Up Introduce Selection Bias?
Attrition can introduce selection bias when continued observation is related to exposure, outcome risk, or other relevant variables in ways that distort the comparison.
Suppose students experiencing declining academic engagement are more likely to stop responding to questionnaires. The remaining respondents may appear more engaged than the original cohort.
If students receiving AI literacy training are also more likely to continue participating, the observed exposure groups may become increasingly different during follow-up.
Analyzing only those with complete observations can then produce an association that differs from the relationship in the intended population.
Importantly, differential loss by exposure group is not the only source of concern. Similar overall loss percentages can conceal differences in the outcome risks of those lost.
Does Equal Loss Across Exposure Groups Mean There Is No Bias?
No. Suppose 15% of trained students and 15% of untrained students are lost.
The identical percentages do not establish that the losses occurred for similar reasons or among participants with comparable outcome risks.
In one group, missingness may be concentrated among highly engaged students. In the other, it may be concentrated among students experiencing academic difficulties.
The resulting bias could differ substantially despite identical attrition rates.
Researchers should therefore examine both the amount of loss and the characteristics and circumstances associated with it.
What Is Informative Censoring?
Censoring occurs when the outcome is not observed beyond a particular point. In survival analysis, participants may contribute valid observation time before becoming censored.
Informative censoring is a concern when censoring carries information about the subsequent event process that is not adequately addressed by the analytical model or conditioning variables.
For example, students with severe academic difficulties may be more likely to leave the university's recording system and also more likely to experience the outcome of interest.
Conventional survival estimators require appropriate assumptions about censoring. When those assumptions are violated, estimated risks or survival probabilities may be biased.
Administrative censoring at a prespecified study closing date is often easier to characterize than participant dropout, but it should not automatically be assumed independent of outcome risk, especially when calendar entry and follow-up duration differ systematically.
How Does Missingness Relate to MCAR, MAR, and MNAR?
Missing-data theory commonly distinguishes three mechanisms, although their relevance depends on the data structure and analytical model.
| Mechanism |
General Meaning |
Illustrative Situation |
| Missing completely at random (MCAR) |
Missingness is unrelated to observed or unobserved data under the formal definition |
Questionnaire records are lost through an unrelated technical failure |
| Missing at random (MAR) |
Missingness can depend on observed information but not additionally on missing values after conditioning appropriately |
Response depends on previously recorded engagement and course load |
| Missing not at random (MNAR) |
Missingness remains related to unobserved values after conditioning on observed information |
Students with unrecorded severe disengagement are more likely to stop responding |
These examples are illustrative. The formal classification depends on the joint data and missingness process.
MAR cannot generally be verified conclusively from observed data alone. MNAR analyses typically require additional assumptions or sensitivity parameters.
Moreover, the assumptions used for missing repeated outcomes and those used for censoring in survival analysis should be matched to their respective estimands and models.
Can Loss to Follow-Up Affect Statistical Precision Without Introducing Bias?
Yes. Missing observations reduce available information and may widen confidence intervals even when the missingness process does not systematically distort the target estimate.
For example, losing participants for reasons unrelated to the relevant outcomes may reduce the number of observed events and the precision of estimated risk differences.
However, precision and bias are separate issues. A highly precise estimate can be biased, while a relatively unbiased estimate may remain too imprecise for a useful conclusion.
Researchers should evaluate both rather than treating sample retention as the sole indicator of evidence quality.
Can You Replace Participants Who Are Lost?
Recruiting replacement participants does not ordinarily restore the missing longitudinal histories of those who left.
For example, a new student recruited during the second year cannot supply the first-year experiences of an original participant who discontinued responding.
Additional recruitment may be appropriate in an open cohort that permits new entrants, but it changes membership according to the study's design rather than repairing the missing observations of original members.
Replacing lost participants in a closed cohort without revising its protocol could also change the target population and follow-up structure.
Researchers should distinguish planned cohort replenishment from attempts to compensate for attrition after it occurs.
How Can Researchers Prevent Loss to Follow-Up?
Prevention is often more defensible than relying entirely on statistical corrections after substantial data have become unavailable.
Possible strategies include maintaining appropriate contact procedures, using manageable assessment schedules, documenting alternative contact channels with consent, and obtaining reliable administrative outcome information where permitted.
Researchers may also reduce participant burden, explain the value of continued participation, and monitor response patterns throughout the study.
Retention strategies should remain ethically appropriate. Participants must retain the right to withdraw, and incentives or contact practices should not become coercive.
The frequency of cohort measurement should be selected with participant burden and likely retention in mind.
Can Multiple Imputation Solve Attrition?
Multiple imputation can be useful for certain missing-data problems when its modeling and missingness assumptions are appropriate.
It creates multiple plausible completed datasets, analyzes them, and combines the resulting estimates while representing uncertainty about missing values.
However, imputation does not recreate the actual unobserved outcomes or automatically remove attrition bias.
Its validity depends on the imputation model, available auxiliary variables, compatibility with the substantive analysis, and assumptions about missingness.
Survival outcomes and longitudinal event histories require particular care. Simple imputation of an unobserved event status without accounting for event timing and censoring may be inappropriate.
What Is Inverse Probability of Censoring Weighting?
Inverse probability of censoring weighting, or IPCW, is an approach that can address certain forms of informative censoring under appropriate assumptions.
The method estimates probabilities of remaining uncensored conditional on relevant observed histories and assigns weights intended to represent participants who became unobserved.
For example, if students with previously measured low engagement are more likely to stop responding, observed students with similar histories may receive greater analytical weight.
Valid application requires appropriate measurement of predictors of censoring and outcome, sufficient positivity, and suitable modeling of censoring probabilities.
Extreme weights may produce unstable estimates. Unmeasured factors associated with both censoring and outcome may leave residual bias.
IPCW is therefore a potentially useful method, not a universal remedy.
Why Are Sensitivity Analyses Important?
Because missing outcomes cannot be directly observed, researchers should examine whether their conclusions depend heavily on assumptions about those outcomes.
For example, an analysis might explore how results would change if students lost to follow-up had substantially higher withdrawal risk than otherwise comparable observed students.
Pattern-mixture models, selection models, delta-adjustment approaches, and other sensitivity frameworks may be appropriate for different missing-data structures.
For simple binary outcomes, transparent best-case and worst-case bounds can illustrate how much uncertainty missing outcomes create, although such bounds may be too wide to be practically decisive.
Sensitivity analyses should use scientifically plausible assumptions where possible and clearly distinguish observed evidence from hypothetical scenarios.
How Should Loss to Follow-Up Be Reported?
Researchers should report the original cohort size, numbers contributing information at relevant follow-up points, reasons for missingness where known, and the distribution of losses across exposure groups.
They should distinguish events, administrative censoring, competing events, withdrawal of consent, and genuinely unavailable outcome information.
Where possible, characteristics of retained and lost participants should be compared, without interpreting nonsignificant differences as proof that attrition is harmless.
Statistical methods for handling missingness and censoring should be described, together with their assumptions and sensitivity analyses.
The STROBE recommendations emphasize reporting participant flow, missing data, follow-up information, and methods used to address potential bias.
