03 · What You Need to Know
How Within-Person Measurements Change Statistical Power and Sample Size
What Is a Repeated-Measures Design?
A repeated-measures design collects multiple observations from the same participant or experimental unit.
Measurements may occur at different time points, under different conditions, or across repeated tasks.
For example, a study may assess student writing performance before an AI feedback intervention, immediately after implementation, and at a later follow-up.
Alternatively, the same students may complete writing tasks under different feedback conditions.
Repeated measures can help researchers investigate within-person changes and account for stable individual differences.
However, repeated-measures designs are not all statistically equivalent. Their requirements depend on whether the primary objective concerns change, treatment differences, trajectories, or interactions.
How Is a Repeated-Measures Design Different from an Independent-Groups Design?
In an independent-groups design, different participants contribute observations to separate conditions.
In a repeated-measures design, the same participant contributes multiple observations, creating dependence among those measurements.
Independent observations
Measurements from different participants are treated as independent under the assumed sampling or assignment model.
Repeated observations
Measurements from the same participant may be correlated and must be analyzed using an appropriate dependence structure.
The distinction affects standard errors, statistical power, and the amount of information contributed by each observation.
Researchers should not calculate sample size by simply multiplying the number of participants by the number of measurement occasions.
Why Can Repeated Measures Improve Statistical Efficiency?
Participants often differ in stable characteristics such as prior knowledge, writing proficiency, or learning motivation.
When the same participant is measured repeatedly, some of these stable differences can be accounted for through within-person comparisons.
For example, comparing a student's writing score before and after an intervention may reduce the influence of stable differences between students.
If repeated measurements are positively correlated, the variability of within-person changes can be smaller than the variability implied by comparing unrelated observations.
This may improve precision and reduce the number of participants required for a specified paired-change objective.
However, the benefit depends on the correlation structure and the effect being estimated.
What Is Within-Person Correlation?
Within-person correlation describes the statistical association between measurements obtained from the same participant.
For example, students who perform well on a baseline writing assessment may also tend to perform well at follow-up.
The strength of this relationship influences how much information repeated observations provide about individual change.
A high positive correlation may reduce the variance of paired differences when measurement variances are comparable.
A low correlation may provide less of this efficiency advantage.
However, correlation can affect different longitudinal estimands in different ways. Its influence should be evaluated using the planned analysis rather than a universal rule.
How Does Correlation Affect the Variance of a Paired Difference?
Suppose each participant has a baseline score and a follow-up score.
The variance of the within-person difference depends on the variances of both measurements and their covariance.
When the two measurements have equal variance σ², the expression simplifies to:
This relationship explains why paired designs can be statistically efficient for certain change-related objectives.
It should not be applied unchanged to every repeated-measures analysis.
What Happens When Within-Person Correlation Increases?
For a paired difference with equal measurement variances, increasing positive correlation reduces the variance of the difference.
Suppose the standard deviation at each measurement is ten points.
If the correlation is 0.20, the variance of change is 2 × 100 × 0.80 = 160.
If the correlation is 0.80, the variance of change is 2 × 100 × 0.20 = 40.
The second scenario produces substantially more precise estimates of mean change for the same number of participants, under the stated assumptions.
However, this does not imply that higher correlation always improves every longitudinal comparison. For example, estimating a difference between independent treatment groups at a single follow-up involves different variance components.
Does Measuring Participants Twice Halve the Required Sample Size?
Not necessarily. The efficiency gain depends on the correlation between measurements and the intended statistical test.
For a paired comparison, repeated observations may reduce the variance of the estimated change relative to a comparison of independent observations.
However, the reduction is not universally 50%.
For a longitudinal intervention study, baseline adjustment, follow-up timing, and treatment-by-time effects may produce different requirements.
Researchers should calculate the sample size for the actual repeated-measures model rather than apply a fixed discount to an independent-groups calculation.
Does Adding More Measurement Occasions Always Improve Power?
Additional measurement occasions can provide useful information about changes over time, but their marginal contribution may diminish.
For example, measuring writing performance at baseline, immediately after an intervention, and three months later may help distinguish immediate gains from sustained improvement.
However, adding several closely spaced assessments may contribute relatively little new information if the measurements are highly redundant.
Additional assessments may also increase participant burden, practice effects, fatigue, and missing data.
The number of measurement occasions should therefore follow the research question and expected trajectory rather than the assumption that more measurements always produce greater statistical efficiency.
Why Does Measurement Timing Matter?
Measurement timing determines which aspects of change can be observed.
For example, an AI feedback intervention may produce an immediate improvement in writing performance that diminishes after several weeks.
A study measuring only baseline and immediate post-intervention performance cannot directly estimate the longer-term trajectory.
Conversely, collecting many measurements during a period when little change is expected may add limited information for the primary objective.
Time intervals also influence correlations among repeated observations and the form of the longitudinal model.
Researchers should select assessment occasions based on the expected process of change and the estimand of interest.
What Is the Difference Between Paired and Longitudinal Analysis?
A paired analysis commonly examines the difference between two measurements from the same participant.
A longitudinal analysis may examine several measurements and model changes over time, including individual trajectories and group differences.
For example, a paired t-test may evaluate whether the mean writing score changes between baseline and follow-up in one group.
A longitudinal mixed-effects model may evaluate whether writing trajectories differ between AI feedback and conventional feedback groups across several time points.
These analyses have different statistical objectives and sample size requirements.
Researchers should avoid treating every study with repeated measurements as though it could be planned using a simple paired t-test.
How Does Baseline Adjustment Affect Sample Size?
In randomized studies with continuous follow-up outcomes, adjusting for a prognostic baseline measurement can improve statistical precision.
For example, baseline writing proficiency may explain substantial variation in post-intervention writing scores.
Under a simplified linear model, with equal baseline and follow-up correlation structures and appropriate assumptions, the residual variance after baseline adjustment is proportional to 1 − ρ².
This does not mean that every study with a baseline correlation of 0.60 can automatically reduce its total sample size by exactly 36%.
The actual efficiency gain depends on the statistical model, allocation, baseline measurement reliability, and other design assumptions.
Is Analysis of Covariance Better Than Comparing Change Scores?
In many randomized studies with continuous outcomes, analysis of covariance using the follow-up outcome and adjusting for baseline can be more statistically efficient than analyzing raw change scores.
However, the comparison depends on the baseline-follow-up relationship and the assumptions of the models.
For example, change-score analysis directly models the difference between measurements, while covariance adjustment estimates a treatment contrast conditional on baseline.
These approaches may differ in precision and interpretation.
Researchers should select the primary analysis prospectively and calculate sample size for that method.
The methodological discussion by Vickers and Altman provides a useful foundation for comparing baseline-adjusted and change-score approaches.
How Does Repeated-Measures ANOVA Affect Sample Size?
Repeated-measures analysis of variance can evaluate changes across measurement occasions and interactions involving within-subject and between-subject factors.
For example, a study may examine whether writing performance changes differently over time between AI feedback and conventional feedback groups.
Sample size planning requires assumptions about the effect of interest, outcome variability, correlations among measurements, number of occasions, and relevant covariance structure.
Some repeated-measures ANOVA procedures rely on sphericity assumptions or use corrections when those assumptions are not satisfied.
The number of measurements alone is therefore insufficient to determine the required sample.
Researchers should ensure that the power calculation corresponds to the planned test and covariance assumptions.
What Is Sphericity and Why Does It Matter?
Sphericity concerns equality of the variances of differences between pairs of repeated measurement conditions.
It is relevant to certain repeated-measures ANOVA tests involving more than two within-subject levels.
When sphericity is violated, conventional F-test assumptions may be inappropriate without adjustment.
Corrections such as Greenhouse-Geisser or Huynh-Feldt may modify the degrees of freedom used for inference.
These corrections can affect power and therefore sample size planning.
However, sphericity is not a universal requirement for every longitudinal model. Mixed-effects and other covariance-based approaches may use different assumptions.
How Do Mixed-Effects Models Change Sample Size Planning?
Mixed-effects models can represent repeated observations nested within participants and may include individual-specific effects.
For example, a longitudinal study may model student writing trajectories with random intercepts and potentially random slopes.
The required sample depends on the number of participants, measurement occasions, variance components, correlations, fixed effects, and missing-data assumptions.
Random-slope models may require substantial information about between-person differences in change.
Simple formulas may be inadequate for complex models.
Simulation-based power analysis can be useful when the planned model includes several variance components or irregular measurement schedules.
What Is the Difference Between More Participants and More Measurements?
Increasing the number of participants adds independent experimental or sampling units when the design supports that interpretation.
Increasing the number of measurements adds information within existing participants.
For example, recruiting 50 additional students may improve the estimation of between-person variation and treatment effects.
Measuring the original students at another time point may improve understanding of their trajectories but does not create 50 additional independent participants.
The relative value of each strategy depends on the estimand and covariance structure.
Researchers should distinguish the number of participants from the total number of recorded observations when reporting sample size.
Can Repeated Measures Compensate for Very Few Participants?
Not necessarily. Repeated measurements cannot fully replace independent participants for every research objective.
For example, collecting 20 writing assessments from each of five students provides extensive within-person information but limited evidence about variability across students.
Such a design may be useful for examining individual trajectories or intensive longitudinal processes.
However, it may be inadequate for estimating population-average treatment effects or differences across participant characteristics.
The number of independent units remains consequential, particularly for between-person inference.
Researchers should avoid treating large numbers of repeated observations as proof of a large effective sample.
How Does Repeated Measurement Affect Participant Burden?
Repeated assessments may increase participant fatigue, time demands, and the risk of incomplete data.
For example, asking students to complete lengthy writing assessments every week may influence motivation or participation.
Repeated testing may also produce practice effects, particularly when the same or similar assessment tasks are administered.
These effects can complicate interpretation of observed changes.
Researchers should balance the statistical value of additional measurements against participant burden and possible measurement reactivity.
More observations are not necessarily better when the measurement process itself changes the phenomenon being studied.
How Does Attrition Affect Longitudinal Sample Size?
Longitudinal studies frequently experience attrition because some participants do not complete later assessments.
For example, students may complete baseline and immediate post-intervention writing assessments but miss a later follow-up.
The recruitment target should account for the expected number of participants providing data relevant to the primary analysis.
However, the impact of missing observations depends on their timing, pattern, and relationship to observed or unobserved outcomes.
Mixed-effects models may use incomplete longitudinal records under appropriate assumptions, but they do not automatically eliminate bias from informative missingness.
See adjusting sample size for dropout and attrition.
Does Every Missing Follow-Up Require Excluding the Participant?
No. The treatment of incomplete repeated measurements depends on the planned statistical analysis and missing-data assumptions.
For example, a complete-case analysis may exclude participants who lack one required assessment.
Other longitudinal methods may incorporate participants with partially observed trajectories.
However, using incomplete records appropriately does not mean that missing data have no effect on power or validity.
The researcher should plan the primary analysis and assess plausible missingness scenarios before data collection.
Sample size calculations should reflect the expected usable information rather than assume perfect follow-up without justification.
How Does Clustering Interact with Repeated Measures?
Educational studies may involve multiple levels of dependence.
For example, repeated writing scores may be nested within students, while students are nested within classrooms.
Measurements from the same student may be correlated, and students from the same classroom may also share contextual influences.
These dependence structures affect statistical precision and power.
A sample size calculation that models within-person correlation but ignores classroom clustering may overstate the amount of independent information.
See how clustering affects sample size for the additional design considerations.
How Do You Choose the Number of Measurement Occasions?
The number of occasions should follow the primary research question and the expected process of change.
For example, a study examining immediate learning gains may require baseline and post-intervention assessments.
A study investigating retention of writing skills may also require delayed follow-up.
A study examining nonlinear learning trajectories may require several strategically timed observations.
Researchers should consider the incremental information from each measurement, expected correlations, participant burden, and missing-data risks.
Measurement schedules should be scientifically meaningful rather than chosen solely to increase the number of recorded observations.
How Can Pilot Data Inform Repeated-Measures Sample Size?
Pilot data may provide preliminary estimates of outcome variability, within-person correlation, measurement completion, and attrition.
For example, a pilot may suggest that baseline and follow-up writing scores are strongly correlated.
However, correlation and variance estimates from small pilots can be unstable.
Researchers should examine uncertainty and consider sensitivity calculations across plausible covariance assumptions.
The planning model should also reflect whether the pilot's measurement schedule and participant population resemble the proposed study.
Using one optimistic pilot correlation without sensitivity analysis may underestimate the required sample.
When Is Simulation-Based Sample Size Planning Useful?
Simulation is particularly useful when the study involves complex longitudinal models, multiple time points, random slopes, irregular observations, or expected missing data.
Researchers can generate hypothetical datasets under specified variance and correlation structures, fit the planned model, and evaluate power or precision across candidate sample sizes.
For example, simulation may compare designs with 100 students measured four times against designs with 150 students measured three times.
The results depend on the assumed data-generating process.
Researchers should therefore evaluate plausible alternatives rather than treat one simulation scenario as definitive.
How Should Repeated-Measures Sample Size Be Reported?
The methodology should identify the primary estimand, number and timing of measurements, planned analysis, and assumptions about within-person correlation and outcome variability.
Researchers should state whether sample size refers to participants, clusters, or total observations.
For longitudinal studies, expected attrition and missing-data assumptions should be documented.
If simulation was used, the data-generating assumptions and criteria for evaluating power or precision should be described.
The final report should distinguish the planned measurement schedule from the observations actually obtained.
Transparent reporting helps readers evaluate whether the repeated-measures design provided sufficient information for the intended conclusions.
