Can Repeated Measures Increase the Information Obtained from a Small Study Population?

Can You Obtain More Useful Research Data by Measuring the Same Participants Several Times?

Suppose you are studying academic engagement among 18 students enrolled in a specialized university program. Because the entire eligible population is small, recruiting additional students is not possible without changing the research question.

You initially plan to measure engagement once at the beginning and once at the end of the semester. Then you consider collecting weekly measurements instead.

With 18 students measured over ten weeks, you could obtain as many as 180 observations. Would that provide the statistical information of a study involving 180 participants?

Repeated measurements may reveal trajectories, short-term fluctuations, and changes that two observations cannot capture. However, measurements from the same participant are related, and additional observations do not automatically provide independent information.

The question is when measuring participants more frequently improves the study and when it merely produces a larger dataset without resolving its fundamental limitations.

Repeated Measures Can Improve Information, but They Do Not Create More Independent Participants

How Repeated Measurement Changes the Information Available in Small Studies

What are repeated measures?

Repeated measures are observations collected from the same individual or research unit on more than one occasion or under more than one condition.

Examples include weekly student engagement scores, monthly patient symptom assessments, repeated laboratory measurements, and performance evaluations before and after an intervention.

Repeated measurements may be collected in observational longitudinal studies, randomized experiments, crossover designs, and single-case experimental research.

The defining feature is that observations are linked to the same unit.

Because repeated observations share participant-specific characteristics, they are generally correlated. The statistical design must recognize this dependence.

Why can repeated measures be useful when the population is small?

When recruiting additional individuals is impossible, repeated measurements may provide more detailed information about the individuals already available.

For example, a single engagement score cannot show whether a student's participation increased steadily, fluctuated, or declined after an intervention.

Weekly measurements may reveal those patterns.

Repeated observations can also help distinguish stable between-person differences from changes occurring within individuals.

However, the benefit depends on the estimand. A study of average change over time may benefit considerably from repeated measurement, while a study of differences between independent population subgroups may remain limited by the number of participants.

The measurement schedule should therefore follow the scientific question rather than a desire to increase the number of dataset rows.

Why are repeated observations not independent participants?

Measurements from the same person often resemble one another because of stable characteristics, previous outcomes, and shared circumstances.

For example, a student with consistently high academic engagement may receive relatively high scores across several weeks.

Those observations provide information about the student's trajectory, but they do not represent independent students drawn from the population.

Treating all repeated observations as independent is a form of pseudoreplication.

It can underestimate standard errors and produce misleading confidence intervals or hypothesis tests.

Researchers should preserve participant identifiers and use analyses appropriate to the dependence structure.

What kinds of information can repeated measures improve?

The benefits differ according to the research objective.

The strongest justification identifies the additional information needed for the estimand and explains how the proposed schedule will provide it.

How does within-person correlation affect statistical information?

Within-person correlation describes the degree to which repeated observations from the same participant resemble one another.

When measurements are highly correlated, additional observations may provide relatively little new information about a participant's stable average.

When measurements capture meaningful variation over time, they may provide more information about temporal patterns.

A simplified design-effect expression can illustrate the influence of correlation when estimating an overall mean from equally sized clusters under an exchangeable correlation model.

For 20 participants with ten measurements each, a naive total of 200 observations would correspond to approximately 200 / 6.4 = 31.25 independent-observation equivalents for the particular overall-mean calculation under these simplifying assumptions.

This does not mean the study contains 31 independent participants, nor that every longitudinal analysis has an effective sample size of 31.25.

Estimating change, treatment-by-time interactions, and individual slopes involves different information structures.

The calculation demonstrates why repeated observations cannot simply be counted as independent participants.

How can paired measurements improve precision?

Paired designs compare measurements from the same participant under two occasions or conditions.

When baseline and follow-up scores are positively correlated, analyzing within-person differences can remove some stable between-person variation.

For example, students may differ substantially in their general academic ability. Comparing each student's follow-up performance with their own baseline may provide a more precise estimate of average change than treating the two sets of scores as independent samples.

However, improved precision does not establish causation.

A pretest-posttest change may reflect maturation, practice effects, concurrent experiences, or other time-related factors.

Researchers should distinguish the statistical advantage of pairing from the causal validity of the study design.

How many measurement occasions are needed?

There is no universal optimal number of repeated measurements.

Two occasions may be sufficient for a simple paired comparison, but they cannot establish the detailed shape of a trajectory.

Three or more occasions may support examination of nonlinear patterns, although the amount of information depends on measurement timing and model complexity.

Frequent measurements may be necessary when the outcome changes rapidly, while less frequent assessments may be appropriate for slowly changing constructs.

Researchers should consider the expected temporal process, measurement reliability, participant burden, and the statistical model.

Vickers' discussion of repeated-measures design emphasizes that the value of additional measurements depends on their correlation and the intended comparative analysis.

More measurements are not automatically better when they are redundant or compromise data quality.

Why does measurement timing matter?

Repeated measures should be scheduled according to when meaningful change is expected.

For example, an instructional intervention may produce immediate changes in task performance but require several weeks before effects on learning retention become apparent.

Measuring every day may add little information if the underlying construct changes slowly.

Conversely, measuring only at the beginning and end of a semester may miss important short-term responses.

Unequally spaced measurements can be appropriate when they reflect the expected process, provided the analysis accounts for actual observation times.

The measurement schedule should be prespecified and justified scientifically.

Can frequent measurement change participant behavior?

Yes. Repeated assessment may influence the phenomenon being measured.

For example, students who repeatedly complete engagement questionnaires may become more conscious of their study habits.

Repeated testing can also create practice effects, fatigue, or changes in motivation.

These measurement effects may complicate interpretation, particularly when the study lacks a comparison condition.

Researchers should consider whether assessment itself functions as an intervention and whether alternative instruments or measurement intervals can reduce reactivity.

Frequent data collection is useful only when its informational benefit outweighs its potential influence on participants and outcomes.

What statistical methods can analyze repeated measures?

The appropriate method depends on the outcome type, number of participants, measurement occasions, missingness, and estimand.

For two paired observations, a paired t-test or another suitable paired procedure may be appropriate for a specific comparison.

Repeated-measures analysis of variance can be used in selected balanced designs, but its assumptions and limitations require consideration.

Mixed-effects models can represent participant-specific variation and accommodate certain unbalanced longitudinal data structures under appropriate assumptions.

Generalized estimating equations may be useful for population-averaged effects, but conventional sandwich standard errors can perform poorly when the number of independent participants or clusters is small.

Researchers should not assume that a mixed model is automatically reliable merely because it can process many repeated observations.

Small-sample inference may require appropriate degrees-of-freedom corrections, alternative variance estimation, or other design-specific procedures.

What is the difference between within-person and between-person effects?

Within-person effects concern changes or associations occurring within the same individual over time.

Between-person effects concern differences among individuals.

For example, students who generally report higher engagement may also have higher achievement. That is a between-person association.

Separately, a particular student's achievement may improve during periods when their engagement exceeds their own usual level. That is a within-person association.

These relationships need not have the same magnitude or direction.

Researchers should avoid interpreting a between-person association as evidence of a within-person process.

Longitudinal models may need to distinguish participant means from deviations around those means when the scientific question requires this separation.

Can repeated measures help estimate individual trajectories?

Yes. Repeated observations can support estimation of how an outcome changes for each participant.

For example, weekly engagement scores may reveal that some students improve steadily while others show temporary gains followed by decline.

However, estimating individual slopes and variation among slopes requires sufficient measurements and appropriate modeling assumptions.

When the number of participants is very small, population-level estimates of random-effect variances may be unstable.

Researchers should distinguish describing observed trajectories from making precise claims about the distribution of trajectories in a broader population.

Graphical presentation of individual time series may be especially informative when confidentiality permits.

Can repeated measures strengthen causal inference?

They can strengthen certain designs, but repeated measurement alone does not establish causality.

For example, observing stable baseline performance followed by a change after intervention introduction may provide useful evidence about temporal relationships.

However, history effects, maturation, seasonal patterns, and other time-varying influences may explain the change.

Experimental arrangements such as randomized treatment sequences, multiple baselines, or appropriately designed crossover studies can provide stronger causal leverage under suitable conditions.

Researchers should consider whether the intervention can be manipulated ethically and whether effects are reversible or persistent.

Repeated observations improve the description of change, while experimental control determines how convincingly that change can be attributed to an intervention.

How do repeated measures relate to single-case experimental designs?

Single-case experimental designs rely on repeated measurement across planned experimental phases or conditions.

For example, a multiple-baseline design may introduce an intervention at different times across participants or behaviors.

The repeated observations allow researchers to examine patterns of level, trend, variability, and changes associated with intervention introduction.

However, simply measuring one participant many times does not automatically create a single-case experiment.

The design must establish appropriate experimental contrasts and replication.

For research involving very few eligible individuals, single-case experimental research may be appropriate when the objective concerns intervention effects on repeatedly measurable outcomes.

What if participants miss some measurement occasions?

Missing repeated measurements can reduce information and introduce bias.

For example, students experiencing high academic stress may be less likely to complete weekly engagement questionnaires.

If missingness is related to unobserved engagement after accounting for available information, conventional missing-at-random analyses may be inadequate.

Some longitudinal models can use partially observed trajectories under suitable assumptions, but they do not automatically resolve informative missingness.

Researchers should plan measurement schedules that minimize unnecessary burden and document reasons for missing observations where feasible.

They should also consider whether changing the measurement schedule could reduce missing data without altering the intended construct.

When do additional measurements stop being useful?

Additional measurements may offer diminishing returns when successive observations are highly correlated or when they provide little new information about the estimand.

For example, measuring a stable characteristic daily may add little information beyond a smaller number of reliable assessments.

Excessive measurement can also increase fatigue, missingness, and operational costs.

Researchers should evaluate the expected improvement in precision or trajectory estimation against the burden of additional observations.

Simulation or design-specific calculations may help compare alternative measurement schedules.

The optimal schedule is the one that provides useful information about the intended outcome, not necessarily the one producing the largest number of records.

When should researchers prioritize more participants instead?

Recruiting additional independent participants may be more valuable when the primary objective concerns between-person effects, population heterogeneity, or generalizability.

For example, estimating how an intervention effect varies across demographic or institutional groups requires sufficient independent individuals within those groups.

Repeatedly measuring the same small set of participants cannot create representation of groups that were never recruited.

However, when the entire eligible population is genuinely small, recruiting more participants may not be possible without changing the target population.

Researchers should then reconsider the estimand or select a design that uses the available population appropriately.

This decision should be grounded in research design principles for small populations.

Comparing Two Measurements with Weekly Follow-Up in a Small Educational Study

The hypothetical example illustrates a situation in which additional measurements are scientifically justified because the research question concerns trajectories.

If the objective instead required precise estimates of differences between independent student subgroups, ten weekly measurements might not resolve the limited number of individuals in those subgroups.

Common Misconceptions About Repeated Measures in Small Studies

Use Repeated Measurement When Time and Within-Person Change Matter

Begin by identifying whether the research question concerns within-person change, individual trajectories, or differences between independent participants.

Then choose measurement occasions according to the expected process and evaluate the additional information against participant burden.

When an intervention can be administered under more than one condition, consider whether a crossover design is suitable for the small population.

Repeated measurement should be used because it improves the evidence relevant to the question, not because it makes the dataset appear larger.

Before Adding Repeated Measurements to a Small Study

Frequently Asked Questions About Repeated Measures in Small Populations

Repeated Measures Add Information, Not Independent Participants

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.