03 · What You Need to Know
Why the Length of Observation Can Change Research Findings
Follow-Up Duration Is Part of the Research Question
A treatment effect measured after one month is not necessarily the same quantity as an effect measured after one year.
Suppose an intervention improves mathematics achievement immediately after instruction. A comparison conducted at the end of the course estimates performance differences at that occasion. A comparison conducted during the following academic year examines performance after additional learning experiences and a longer period without the original intervention.
These are related but distinct scientific questions.
The International Council for Harmonisation's E9(R1) framework emphasizes that a treatment effect should be defined through an estimand, which specifies the treatment conditions, target population, outcome variable, handling of relevant intercurrent events, and population-level summary.
For outcomes defined at particular occasions, the time point is part of the outcome definition. Changing it can therefore change the estimand.
Researchers should not regard follow-up duration as merely an administrative feature of the study.
Intervention Effects May Increase, Decrease, or Reverse Over Time
An intervention's effect need not remain constant throughout observation.
Some interventions produce an immediate advantage that weakens after treatment ends. Others require repeated practice or sustained exposure before meaningful benefits emerge.
For example, a digital learning platform might improve performance on recently taught material while providing less advantage on a delayed retention assessment. Conversely, a program designed to develop independent learning strategies might produce greater differences after students have had time to apply those strategies.
Effects can also reverse direction if early benefits are accompanied by later disadvantages or if comparison participants eventually receive other effective support.
These possibilities explain why short-term and long-term outcomes may provide complementary evidence.
However, an apparent change in effect size across assessments does not establish that the underlying causal effect changed. Sampling variability, missing data, measurement error, and other design features must also be considered.
Longer Follow-Up Can Reveal Events That Shorter Studies Miss
For outcomes defined by the occurrence of an event, extending follow-up provides more opportunity for events to occur.
For example, a study examining hospitalization within 30 days addresses a different cumulative risk period from a study examining hospitalization within one year.
The proportion of participants experiencing an event may increase as observation continues, even if the underlying event rate remains unchanged.
This is especially important when comparing studies with different observation periods.
A cumulative event proportion should not be interpreted without considering the duration over which events were counted.
Distinguish Cumulative Risk From Incidence Rate
Two commonly used measures summarize event occurrence differently.
Cumulative incidence or risk
The probability or proportion of initially event-free individuals experiencing a specified event over a defined period, under the relevant assumptions.
Incidence rate
The number of incident events divided by the total person-time at risk, typically expressed as events per unit of person-time.
These measures are not interchangeable. Cumulative risk depends explicitly on the observation horizon, while an incidence rate summarizes events relative to time at risk.
For a simple illustration, consider 100 participants followed for one year. If 10 experience a first event, the observed one-year event proportion is 10% when all participants are completely followed and the event definition is appropriate.
If another study follows participants for only one month, its observed event proportion cannot be compared directly with the one-year proportion as though both represented the same risk period.
Time-to-event methods can account for unequal observation times and censoring under appropriate assumptions, but they do not make every follow-up duration equivalent.
Why Longer Follow-Up Does Not Automatically Increase Statistical Power
For event outcomes, longer observation may produce more events, potentially increasing the information available for an analysis.
However, this benefit depends on the event process, treatment effect, censoring, competing events, and analytical method.
For continuously measured outcomes, additional elapsed time does not automatically improve precision. Outcomes may become more variable, treatment effects may weaken, or missing data may increase.
Moreover, an analysis planned for a particular endpoint should not be assumed to gain power simply because participants are observed longer.
Sample size and power calculations should correspond to the intended follow-up duration and effect measure. CONSORT 2025 emphasizes reporting the assumptions underlying sample size calculations, including follow-up duration and event-related assumptions when relevant.
How Can Attrition Distort Longer-Term Results?
Attrition occurs when participants leave a study or fail to provide planned measurements. Longer observation can create more opportunities for missed assessments, although attrition rates vary substantially across studies.
Missing follow-up outcomes may reduce precision and introduce bias when the reasons for missingness are related to outcomes or treatment conditions.
For example, students with poor academic performance may be less likely to complete a later assessment. If only highly engaged participants remain, their observed results may overstate performance in the originally enrolled population.
Comparing only participants who completed every assessment can therefore produce misleading conclusions.
Researchers should report the number of participants with available outcome data at each time point and explain how missing data were addressed.
CONSORT 2025 explicitly emphasizes reporting outcome availability by group and assessment occasion. Appropriate missing-data methods and sensitivity analyses should reflect the plausible reasons for missingness.
What Happens When Other Exposures Occur During Follow-Up?
The longer participants are observed, the more opportunities they may have to experience other influences relevant to the outcome.
In educational research, these may include additional tutoring, changes in teaching staff, new learning resources, or exposure to similar interventions.
In clinical research, participants may receive additional treatments, discontinue the assigned intervention, or experience other health-related events.
Such events can change the observed treatment-group difference.
However, they should not automatically be treated as unwanted confounders. Some are consequences of treatment assignment or ordinary features of the intervention strategy being evaluated.
ICH E9(R1) describes approaches to intercurrent events, including treatment-policy and hypothetical strategies. The appropriate strategy depends on the treatment effect the study intends to estimate.
For example, estimating the effect of offering an intervention under usual conditions may appropriately include subsequent nonadherence and additional services. Estimating an effect under a hypothetical condition without those events requires different assumptions.
Researchers should define these choices prospectively rather than adjusting indiscriminately for every event occurring after treatment begins.
Can Follow-Up Duration Change the Interpretation of Statistical Significance?
Yes. Different assessment times can produce different effect estimates, standard errors, confidence intervals, and p-values.
Suppose an intervention produces a statistically significant difference after six weeks but not after six months.
This does not necessarily establish that the intervention effect disappeared. The later estimate may be smaller, less precise, or based on fewer observed participants.
Likewise, a nonsignificant early comparison followed by a significant later comparison does not prove that the treatment effect increased.
Researchers should examine the estimated effects and their uncertainty at both occasions and, when relevant, directly evaluate whether the treatment effect differs across time.
A difference in statistical significance is not the same as a statistically established difference between effects.
How Does Follow-Up Duration Affect Practical Importance?
The practical importance of an effect can depend on when it occurs and how long it lasts.
A temporary improvement may be valuable when the intervention is designed to address an immediate need. The same temporary improvement may be inadequate when the objective is lasting behavioral change.
For example, an intervention that improves students' examination performance during a particular course could have practical value even if the advantage does not persist into later semesters.
However, if the intervention is marketed as developing durable independent learning skills, later outcomes become central to evaluating that claim.
Researchers should interpret effect magnitude against the intended purpose, not assume that the longest-lasting effect is always the most important.
How Should Studies With Different Follow-Up Durations Be Compared?
Comparisons across studies require attention to the timing and definition of their outcomes.
Two studies may evaluate similar interventions but measure outcomes at substantially different intervals. Their effect estimates may therefore represent different stages of intervention response.
Researchers conducting evidence syntheses should examine whether follow-up times are sufficiently comparable to support pooling or whether results should be grouped by clinically or substantively meaningful periods.
For example, immediate post-intervention achievement and achievement measured one year later may warrant separate analyses.
Arbitrary pooling can obscure temporal differences, while excessive separation into numerous small time categories may reduce precision.
Follow-up categories should be justified by the outcome and intervention rather than selected solely according to the observed results.
Does Longer Follow-Up Always Improve External Validity?
No. Longer follow-up may reveal how an intervention performs under changing real-world conditions, but it can also introduce interpretive complications.
Participants may move to different institutions, receive other interventions, or encounter changes in the environment that substantially alter the conditions under which the original treatment was delivered.
Whether this improves applicability depends on the intended target setting and treatment strategy.
Longer observation is useful when those evolving conditions are part of the decision researchers want to inform. It is less useful when the additional period is unrelated to the intervention's intended effects.
The question of whether follow-up has become unnecessarily long therefore concerns scientific relevance as well as practical feasibility.
How Should Follow-Up Duration Be Selected and Reported?
The follow-up period should be chosen before outcome results are examined, using substantive knowledge about the intervention and expected outcome process.
Researchers should define the start of follow-up, the intended endpoint, allowable assessment windows, and the duration over which events or outcomes will be evaluated.
For time-to-event studies, it is important to distinguish administrative study closure from individual follow-up duration. Participants recruited at different dates may contribute different amounts of observation time.
Reporting should describe actual follow-up experience, including censoring and relevant summaries of observation duration.
CONSORT 2025 recommends reporting the periods of recruitment and follow-up, the outcome assessment time points, and the numbers with available data at each occasion. Its explanation and elaboration document provides detailed guidance.
Researchers should also consider whether the planned follow-up ends before meaningful effects can emerge.
