03 · What You Need to Know
Understanding the Methodological Trade-Offs Between Within- and Between-Participant Designs
What Is a Within-Participant Comparison?
A within-participant comparison examines two or more measurements or conditions involving the same individual. The participant serves as the basis for comparing their own observations.
For example, students may complete assessments before and after an instructional intervention. Alternatively, the same students may experience two feedback conditions in a counterbalanced experiment.
These arrangements share repeated observations, but they do not have identical inferential properties. A one-group pretest-posttest study measures change over time, whereas a randomized crossover experiment can compare assigned conditions under additional assumptions.
Researchers should therefore distinguish repeated measurement from experimental manipulation. Using the same participants does not automatically establish a causal design.
What Is a Between-Participant Comparison?
A between-participant comparison examines observations from different individuals or groups.
For example, one group of students may receive AI-supported feedback while another receives conventional feedback. The researcher compares outcomes across the groups.
When participants are randomly assigned to conditions, the assignment process can support causal inference about the effect of assignment, subject to relevant assumptions and implementation conditions.
When groups are formed through voluntary participation or existing characteristics, differences may reflect selection processes or confounding.
The term between-participant describes the comparison structure, not whether the study is randomized.
How Do the Two Approaches Differ?
| Feature |
Within-participant comparison |
Between-participant comparison |
| Participants |
The same individuals provide repeated observations. |
Different individuals contribute observations to different groups or conditions. |
| Individual differences |
Stable individual characteristics can be controlled through paired comparison. |
Differences between individuals contribute to variation unless addressed through design or analysis. |
| Repeated exposure |
May create practice, fatigue, order, or carryover effects. |
Participants need not experience multiple conditions. |
| Statistical dependence |
Repeated observations are correlated and require appropriate analysis. |
Observations may be independent, unless clustering or other dependencies exist. |
| Common applications |
Longitudinal measurement, crossover experiments, repeated-condition studies. |
Parallel-group experiments, observational group comparisons. |
| Principal design concern |
Whether changes or earlier conditions affect later observations. |
Whether the groups provide a defensible contrast. |
These are general distinctions. A research design may combine both structures, such as a randomized parallel-group trial that measures every participant repeatedly over time.
Why Can Within-Participant Designs Improve Statistical Precision?
When observations from the same person are positively correlated, analyzing their paired difference can remove some variability attributable to stable individual characteristics.
Consider two writing assessments administered to the same students. Students differ in prior writing ability, but those relatively stable differences may contribute less to variation in individual changes than to variation in scores across unrelated students.
For paired observations, the variance of a difference is:
For example, if both measurements have variance 100 and covariance 70, the variance of the paired difference is 60. If covariance were zero, it would be 200.
This illustrates a potential precision advantage, not a guarantee that within-participant designs always require fewer participants. Sample requirements also depend on the outcome, effect size, correlation, missing data, and design assumptions.
Does Using the Same Participants Eliminate Confounding?
No. Within-participant comparisons can control for characteristics that remain stable across observations, but they do not automatically address factors that change over time.
For example, students may improve their writing because of regular coursework, feedback from other sources, practice, or maturation.
If all students are measured before and after receiving AI-supported feedback, those influences may be difficult to distinguish from the intervention effect.
Similarly, in a crossover study, the effect of the first condition may persist into the second period.
Researchers should identify time-varying confounders, period effects, and relevant intervention histories before interpreting within-person differences causally.
What Are Order Effects?
Order effects occur when the sequence in which participants experience conditions influences their outcomes.
For example, students who first complete a writing task with AI-supported feedback may become more familiar with the assessment procedure before completing a task with conventional feedback.
Improved performance in the second condition could partly reflect practice rather than the feedback approach.
Fatigue, motivation, and changing familiarity may also affect later observations.
Randomizing or counterbalancing condition order can help distribute certain order effects across participants.
However, counterbalancing does not automatically eliminate effects that depend on the specific sequence or persist after the original condition ends.
What Are Carryover Effects?
Carryover occurs when exposure to one condition influences responses under a subsequent condition.
For example, students may learn revision strategies while using AI-supported feedback and continue applying those strategies when later receiving conventional feedback.
The second condition would then be evaluated among students whose skills had already been influenced by the first.
Carryover is particularly important when interventions produce lasting learning, behavioral, or psychological changes.
In some settings, a washout interval may reduce temporary effects. However, learned knowledge and skills may not be reversible, making a conventional crossover design unsuitable.
Researchers should not assume that a waiting period restores participants to their original state.
When Is a Crossover Design Appropriate?
A crossover design allows participants to receive multiple conditions in different periods, often with randomized condition order.
Such designs may be useful when the outcome is relatively stable, intervention effects are temporary, and participants can reasonably return to a comparable state before experiencing another condition.
For example, researchers might compare immediate responses to two interface layouts using counterbalanced task sequences, provided learning and carryover are adequately addressed.
However, crossover arrangements may be unsuitable for interventions intended to produce durable educational learning.
Senn (2002) provides a detailed treatment of crossover trial design, including period and carryover considerations.
Researchers should distinguish crossover experiments from simple repeated-measures studies because their causal assumptions and analytical requirements differ.
When Is a Between-Participant Design Preferable?
Between-participant designs are often preferable when the intervention cannot be reversed, repeated exposure would alter subsequent performance, or participants cannot reasonably experience every condition.
For example, a semester-long instructional intervention may produce lasting learning. Students cannot be returned to their original knowledge state before experiencing another instructional condition.
A parallel-group design may therefore provide a clearer comparison.
Between-participant designs may also reduce participant burden when each condition involves substantial time or effort.
However, the researcher must address differences between groups, especially when assignment is not randomized.
How Does Randomization Affect the Comparison?
Randomization can be used in either within-participant or between-participant experimental designs.
In a parallel-group trial, participants may be randomly assigned to different conditions.
In a crossover trial, the order of conditions may be randomized.
These procedures address different sources of potential bias.
Random assignment can support comparability of groups in expectation, while randomized condition order can help address systematic sequence differences.
Neither procedure removes every concern about adherence, missing outcomes, carryover, measurement, or implementation.
The causal claim must correspond to the actual randomization and intervention structure.
Can a Study Use Both Within- and Between-Participant Comparisons?
Yes. Many educational intervention studies combine the two.
For example, students may be randomly assigned to AI-supported or conventional feedback, with writing performance measured at baseline and after the intervention.
The between-participant dimension concerns assigned feedback condition. The within-participant dimension concerns repeated measurement over time.
The primary analysis may estimate the difference between groups at follow-up while accounting for baseline performance, or use another appropriately specified repeated-measures approach.
Vickers and Altman (2001) discuss why analysis of covariance may be preferable to simple change-score comparisons in randomized trials with baseline and follow-up measurements.
Researchers should distinguish the intervention contrast from within-person change and avoid assuming that the latter independently establishes treatment effectiveness.
Which Statistical Analyses Are Appropriate?
The analysis depends on the outcome, number of conditions, repeated measurements, distributional assumptions, and assignment structure.
Paired comparisons may be analyzed using paired-sample procedures when their assumptions are appropriate. Independent-group comparisons may use corresponding independent-sample methods.
Repeated-measures analysis, mixed-effects models, generalized estimating equations, or other approaches may be appropriate for more complex structures.
For example, multiple observations nested within students and students nested within classrooms may require explicit consideration of both levels of dependence.
The choice should not be made solely by counting groups or measurement occasions.
Researchers should specify the intended estimand and select analytical procedures consistent with the design and data structure.
How Do Missing Data Affect Within-Participant Designs?
Repeated-measures studies may lose information when participants complete only some measurement occasions.
For example, students who provide baseline scores but miss the final assessment may differ systematically from those who complete both.
A complete-case paired analysis may exclude these students and introduce bias when missingness is related to relevant characteristics or outcomes.
Some longitudinal modeling approaches can use incomplete repeated observations under appropriate assumptions, but they do not automatically eliminate missing-data bias.
Researchers should anticipate attrition, document missingness, and consider appropriate sensitivity analyses.
How Does Participant Burden Affect Design Choice?
Within-participant designs may require individuals to complete multiple tasks, assessments, or intervention periods.
This can increase fatigue, dropout, or response changes caused by repeated participation.
Between-participant designs may reduce individual burden because each person experiences fewer conditions, although they may require recruiting more participants.
The balance depends on the duration, complexity, and acceptability of the procedures.
Researchers should consider both methodological validity and ethical proportionality when selecting the comparison structure.
How Do You Choose the Appropriate Design?
Begin by identifying the exact comparison needed to answer the research question.
Then determine whether the same participants can experience all relevant conditions without unacceptable learning, carryover, or period effects.
If repeated exposure is meaningful, a within-participant arrangement may provide an efficient comparison.
If the intervention produces lasting changes or repeated exposure would compromise the contrast, a between-participant design may be preferable.
The choice should also consider recruitment feasibility, statistical precision, missing data, and the intended causal interpretation.
The broader question of identifying the exact comparison required by the research question should be resolved before selecting a statistical procedure.
Watch Out
Do not assume that within-participant designs are automatically more rigorous because participants serve as their own controls. Stable individual differences may be reduced, but time-related confounding, practice, carryover, and irreversible learning can undermine the intended comparison.
