Should Participant Retention Rate Be an Outcome in a Feasibility Study?

Should Participant Retention Be Measured Before Conducting a Definitive Study?

You have designed a study requiring participants to remain involved over time. Perhaps students will complete questionnaires at several points during a semester, teachers will participate in a six-week instructional intervention, or participants will return for follow-up assessments after the intervention ends.

Recruitment may be feasible, but will participants continue participating long enough to provide the evidence needed for the definitive study?

Some participants may withdraw, become unreachable, miss assessments, or stop engaging with intervention activities. These events are related, but they do not necessarily mean the same thing.

Should participant retention rate be included as a formal feasibility outcome? What exactly counts as retention, how should it be calculated, and what level of retention would justify proceeding?

The answers depend on the study's participation requirements, its intended outcomes, and the consequences of losing participants or observations during follow-up.

Measure Retention When Continued Participation Is Essential to the Definitive Study

Understanding Participant Retention as a Feasibility Outcome

What Is Participant Retention?

Participant retention generally concerns the continued participation of individuals in a study over a defined period or until a specified research milestone.

For example, a longitudinal study may define retention as remaining enrolled through the final follow-up assessment.

Another study may define retention according to whether participants remain contactable and willing to provide follow-up information.

These definitions are not identical.

Researchers should specify the event that constitutes retention rather than assume that the term has one universal operational meaning.

The definition should reflect what continued participation contributes to the definitive research question.

Why Is Retention Important in Feasibility Research?

Retention can influence whether a study obtains the observations needed for its intended analysis.

For example, a longitudinal educational study may enroll enough students at baseline but obtain follow-up information from only a portion of them.

This may reduce statistical precision and create concerns about bias if missing outcomes are related to participant characteristics or unobserved outcomes.

Retention feasibility is therefore particularly important when the research requires repeated measurements, long-term follow-up, or sustained participation.

However, the consequences of attrition depend on the design and the reasons information becomes unavailable.

A single-session anonymous survey may have little need for a longitudinal retention outcome.

How Is Retention Different From Recruitment?

Recruitment concerns identifying and enrolling participants.

Retention concerns maintaining their participation or follow-up after enrollment.

A study can have excellent recruitment but poor retention.

For example, students may readily enroll in a semester-long study but withdraw when coursework becomes demanding.

Conversely, a study may recruit slowly while retaining nearly everyone who enrolls.

These findings suggest different feasibility problems and should be reported separately.

For the recruitment-specific question, see Should Recruitment Rate Be an Outcome in a Feasibility Study?.

How Do You Calculate Participant Retention Rate?

A commonly used retention proportion compares the number of enrolled participants retained at a specified follow-up point with the relevant original cohort.

This calculation is useful when the cohort and retention definition are clear.

However, studies involving staggered enrollment, variable follow-up periods, or competing events may require more detailed approaches.

The appropriate denominator and analysis should reflect the actual study design.

What Is Attrition Rate?

Attrition commonly refers to the loss of participants from a study over time.

When retention and attrition are defined as mutually exclusive and exhaustive outcomes for the same cohort and follow-up point, their proportions sum to 100%.

However, the relationship becomes more complicated when studies distinguish formal withdrawal, missed assessments, incomplete intervention participation, and continued enrollment without outcome data.

Researchers should not assume that every missing measurement represents participant attrition.

How Is Retention Different From Outcome Data Completeness?

Retention and outcome completeness are related but distinct.

A participant may remain enrolled and attend study activities while failing to complete the primary outcome assessment.

Conversely, a participant who discontinues an intervention may still provide outcome data if the protocol permits continued follow-up.

The operational definitions should be established before data collection.

For a focused discussion of outcome completeness, see Should Data Completeness Be an Outcome in a Feasibility Study?.

Should Retention Be a Primary Feasibility Outcome?

Retention may be the primary feasibility outcome when maintaining follow-up represents the central uncertainty.

For example, a study involving repeated assessments over an academic year may depend on whether students remain available throughout the observation period.

If the feasibility study is principally intended to evaluate follow-up procedures, retention may appropriately receive primary emphasis.

However, retention need not be primary when recruitment, intervention delivery, or another issue presents a greater threat.

The outcome hierarchy should follow the purpose of the feasibility study rather than a universal requirement.

When Is Retention Particularly Important in Educational Research?

Retention may be consequential in longitudinal educational studies, intervention evaluations, repeated-measures investigations, and research spanning several academic periods.

Students may face examination schedules, course withdrawal, changing enrollment status, or competing responsibilities.

Teachers may experience workload fluctuations, changes in teaching assignments, or institutional transitions.

For example, a study examining student engagement across a semester may find that follow-up completion declines during examination periods.

A feasibility study can help determine whether assessment scheduling and communication procedures are realistic.

However, findings from one academic term may not transfer directly to another.

Should Retention Be Measured at Multiple Time Points?

Yes, when the definitive study includes repeated follow-up and the timing of attrition matters.

For example, a six-month study may assess retention after one month, three months, and six months.

Reporting only final retention can conceal when participants become unavailable.

Early losses may suggest problems with initial expectations or onboarding, while later losses may reflect sustained burden or changing circumstances.

Researchers should distinguish scheduled assessments from continued enrollment and specify the denominator used at each time point.

Where follow-up durations vary, appropriate time-to-event methods may be considered rather than relying exclusively on simple proportions.

What Causes Participant Attrition?

Attrition may arise from scheduling conflicts, participant burden, loss of interest, technical difficulties, health or personal circumstances, relocation, or other reasons.

In educational research, examination periods, changing course enrollment, and competing academic demands may also be relevant.

However, researchers should avoid assuming that all withdrawals reflect dissatisfaction with the study.

Participants may discontinue for reasons unrelated to the intervention or research procedures.

Where ethically and practically appropriate, feasibility studies can document reasons for withdrawal or missed follow-up.

These explanations may inform modifications, although information from individuals who cannot be contacted may remain unavailable.

Should Reasons for Withdrawal Be Collected?

When appropriate, information about withdrawal reasons can help distinguish remediable procedural problems from circumstances beyond the research team's control.

For example, repeated reports of excessive questionnaire burden may justify revising the assessment schedule.

However, participants should not be pressured to explain their withdrawal.

The right to withdraw should be respected according to the applicable consent arrangements and ethical requirements.

Researchers should collect only information justified by the study and approved procedures.

Missing withdrawal explanations should be acknowledged rather than replaced with assumptions.

Can Retention Be Improved Through Reminders?

Reminders may support follow-up completion when they are appropriate, accessible, and consistent with participant preferences and consent arrangements.

For example, students may benefit from receiving advance notice of scheduled questionnaires.

However, repeated messages can become burdensome or intrusive.

Researchers should examine reminder frequency, communication channels, timing, and the possibility of unintended disclosure of participation.

A feasibility study may test whether reminder procedures are practical and acceptable.

It should not assume that increasing reminder frequency will always improve retention.

Should Participant Incentives Be Used to Improve Retention?

Incentives may be considered when ethically appropriate and permitted by the research protocol.

However, their use requires attention to fairness, voluntariness, participant burden, and the applicable institutional requirements.

For example, compensation for completing a lengthy follow-up assessment may be reasonable in some contexts.

Researchers should avoid arrangements that create inappropriate pressure or obscure the distinction between compensation and research benefits.

Retention feasibility should not be pursued at the expense of meaningful participant choice.

How Does Retention Affect Statistical Power?

Loss of participants or outcome observations may reduce the amount of information available for the definitive analysis.

In some designs, researchers account for anticipated attrition when planning recruitment targets.

For example, if 180 complete follow-up observations are required and the expected retention proportion is 90%, a simple planning calculation suggests enrolling 200 participants.

However, increasing initial enrollment does not automatically resolve bias arising from missing outcomes.

Sample size planning and missing-data strategies should reflect the actual study design.

Can High Attrition Introduce Bias?

Potentially. Missing outcomes may introduce bias when the probability of missingness is related to relevant participant characteristics or outcomes in ways not adequately addressed by the analysis.

For example, students experiencing difficulty with an intervention may be less likely to complete follow-up assessments.

If their outcomes are systematically missing, the observed results may provide a misleading picture of the intervention.

However, the amount of attrition alone does not determine the direction or magnitude of bias.

Researchers should consider why outcomes are missing and what assumptions are required by the planned analysis.

Can a Feasibility Study Establish the Missing-Data Mechanism?

Usually not with confidence, particularly when the preliminary sample is small.

Researchers may identify associations between observed characteristics and missingness or document practical reasons for missed follow-up.

However, distinguishing missing completely at random, missing at random, and missing not at random generally involves assumptions that cannot be established through simple descriptive comparisons alone.

A feasibility study can identify potential problems and inform the planned missing-data strategy.

It should not claim that future missingness will be ignorable merely because no clear pattern was observed in the pilot.

Should Retention Be Evaluated Separately by Study Group?

In randomized or comparative studies, retention may differ between intervention and comparison conditions.

For example, participants assigned to a more demanding intervention may withdraw more frequently.

Researchers may report retention descriptively by group to identify possible feasibility concerns.

However, small feasibility samples may provide imprecise estimates of between-group differences.

Researchers should not use preliminary significance tests to conclude that attrition is equivalent across groups.

The emphasis should remain on identifying risks and improving the definitive study.

How Should Retention Be Evaluated in Cluster Studies?

Cluster studies may experience loss at both the individual and cluster levels.

For example, a university class may remain in a study while individual students miss follow-up assessments.

Alternatively, an entire school or class may withdraw, resulting in the loss of multiple participants.

These events have different consequences for statistical precision and the integrity of the study design.

Researchers should therefore distinguish cluster retention from individual participant retention where relevant.

The number of clusters retained may be especially consequential for a cluster-randomized trial.

What Retention Rate Is Considered Acceptable?

There is no universally acceptable retention percentage for every feasibility study.

The appropriate threshold depends on the research design, follow-up duration, primary outcome, anticipated missing-data consequences, and resources available for the definitive study.

For example, the consequences of 15% attrition may differ between a short-term descriptive study and a small randomized trial with limited statistical precision.

Researchers should establish retention expectations based on relevant evidence and the needs of the proposed analysis.

Arbitrary thresholds should not replace study-specific methodological justification.

How Should Retention Progression Criteria Be Developed?

Progression criteria should connect observed retention to the feasibility of the definitive study.

For example, researchers may determine the minimum follow-up completion required to obtain adequate primary outcome information within the planned sample and resources.

A traffic-light framework may distinguish outcomes supporting progression from those requiring modification or substantial reconsideration.

However, thresholds should be justified in advance where possible.

Researchers should also consider uncertainty and whether observed attrition problems can be remedied.

A pilot estimate slightly below a threshold should not automatically determine the decision without examining the evidence and context.

How Much Uncertainty Is Associated With Pilot Retention Estimates?

Retention proportions from small samples may be imprecise.

For example, observing that 18 of 20 participants completed follow-up gives a retention proportion of 90%.

However, the small denominator means that the underlying retention probability remains uncertain.

Confidence intervals may help communicate this uncertainty when appropriate.

Researchers should avoid interpreting the observed percentage as a precise prediction for the definitive study.

The required precision depends on the progression decision and the consequences of an incorrect feasibility judgment.

Can Retention Findings Be Generalized Across Populations?

Only cautiously.

Retention may vary with participant characteristics, study burden, follow-up duration, institutional setting, and communication arrangements.

For example, a retention strategy tested with postgraduate students may perform differently among first-year undergraduates.

Similarly, retention in a short pilot may not predict participation over an entire academic year.

Researchers should examine whether the pilot conditions adequately resemble the definitive study.

Where important differences exist, additional evidence or conservative planning assumptions may be warranted.

What If Retention Is Lower Than Expected?

The research team should investigate when and why participants became unavailable.

For example, missed follow-up may result from scheduling conflicts, unclear communication, technical barriers, or excessive assessment burden.

Appropriate modifications might include revising assessment windows, improving participant communication, or reducing unnecessary procedural burden.

However, modifications should preserve the study's essential measurements and ethical safeguards.

Substantial changes may require further feasibility testing.

Low retention may justify redesign, but it does not automatically mean that the research question lacks value.

What If Retention Is High?

High retention is encouraging when continued participation is essential to the definitive study.

However, the finding should be interpreted according to the pilot's duration, sample, and implementation conditions.

For example, retention may be unusually high when pilot participants receive intensive personal support that cannot be maintained at scale.

Researchers should consider whether the definitive study will have comparable staffing and resources.

High retention also does not guarantee complete or valid outcome data.

Should Retention Be Evaluated Alongside Intervention Adherence?

Often, yes, particularly when the intervention requires repeated activities.

However, retention and adherence are different constructs.

A participant may remain enrolled and complete outcome assessments while attending few intervention sessions.

Conversely, a participant may complete the intervention but become unavailable for later follow-up.

Researchers should define each outcome separately and examine its implications for the definitive study.

See Should Intervention Adherence Be an Outcome in a Feasibility Study?.

Should Retention Rate Be Included in Every Feasibility Study?

No. Retention is most relevant when continued participation or follow-up is necessary.

For example, a feasibility study examining institutional data access may not involve repeated participant contact.

In that case, retention may contribute little to the central feasibility question.

Researchers should select outcomes according to the uncertainties that matter rather than include retention automatically.

The broader process is discussed in What Feasibility Questions Should You Answer Before Conducting a Definitive Study?.

How Should Retention Findings Be Reported?

Researchers should define retention, identify the baseline cohort, specify follow-up points, and report the numbers retained and lost.

Where possible, reports should distinguish withdrawal, missed assessments, intervention discontinuation, and loss to follow-up.

Reasons for attrition may be described when available and ethically appropriate.

In comparative studies, reporting by group may reveal relevant differences.

Researchers should explain the uncertainty of estimates and how the findings influence the proposed definitive design.

Retention should not be presented as an isolated percentage without methodological context.

Evaluating Retention in a Semester-Long Online Learning Study

Illustrative Retention and Outcome Completeness

The findings demonstrate why retention and outcome completeness should be assessed separately.

A 90% final retention rate may appear encouraging, but the definitive study also depends on obtaining usable primary outcome records.

The feasibility team should therefore consider both measures, investigate the reasons for incomplete assessments, and evaluate the findings against contextually justified progression criteria.

The pilot does not establish that the same retention pattern will occur in a larger study or that the intervention is effective.

Common Misconceptions About Participant Retention in Feasibility Studies

How to Decide Whether Retention Should Be a Feasibility Outcome

Begin by determining whether the definitive study depends on participants remaining available over time.

Then define the specific form of continued participation required and identify how attrition could affect the planned analysis.

Retention should be included because it addresses a consequential uncertainty, not because every feasibility study needs an attrition percentage.

Before Using Retention Rate as a Feasibility Outcome, Check:

Frequently Asked Questions About Retention Rate as a Feasibility Outcome

Measure Retention When Continued Participation Determines the Quality of the Evidence

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.