03 · What You Need to Know
What Counts as Replication and How Much Evidence Is Convincing?
What does replication mean in single-case experimental research?
Replication refers to obtaining additional evidence that an intervention produces a functional relationship with an outcome under planned experimental conditions.
In single-case research, replication can occur through repeated changes in experimental conditions within one case or through demonstrations across several cases, behaviors, or settings.
For example, a reversal design may repeatedly introduce and remove an intervention. A multiple-baseline design may introduce the same intervention at staggered times across different participants.
Both arrangements can create opportunities to determine whether outcome changes correspond systematically with intervention manipulation.
However, replication requires an interpretable experimental contrast. Merely collecting additional observations under unchanged conditions does not establish another demonstration of effect.
What is a demonstration of effect?
A demonstration of effect occurs when the observed outcome pattern provides evidence that a change in experimental condition is associated with the predicted change in behavior or performance.
For example, a student may show relatively consistent task completion during baseline and a substantial improvement after prompting begins.
That pattern can contribute to a demonstration, but its interpretation depends on baseline trend, variability, immediacy, overlap, and alternative explanations.
When similar intervention-related changes appear at different planned points in time, the cumulative pattern may strengthen experimental control.
Researchers should distinguish the opportunity to demonstrate an effect from an effect that is actually demonstrated convincingly.
Why do methodological standards emphasize three demonstrations?
The What Works Clearinghouse single-case design framework identifies at least three attempts to demonstrate an intervention effect at three different points in time as a central design requirement.
This reflects the importance of replication for distinguishing intervention effects from coincident events, maturation, and other alternative explanations.
For example, one intervention introduction may coincide with a change in teacher behavior or task difficulty. Replicating the intervention-related pattern at additional times makes some alternative explanations less plausible.
However, three is not a universal statistical threshold proving causation.
Design standards also address matters such as phase observations, experimental manipulation, outcome measurement, and the arrangement of intervention changes.
Researchers should consult the applicable version of the standards rather than applying the number three without regard to the design.
Are three participants equivalent to three demonstrations?
No. Three participants do not automatically provide three demonstrations of effect.
Suppose three students receive an intervention simultaneously and all improve afterward.
Although three students contributed observations, the intervention began at one common point in time. A shared historical event could plausibly explain their simultaneous improvement.
By contrast, a multiple-baseline design may introduce the intervention at three sufficiently separated times while continuing baseline measurement in untreated tiers.
The staggered introductions can create distinct opportunities to evaluate the intervention-outcome relationship.
The important feature is the experimental structure, not simply the participant count.
How do replications differ across single-case designs?
| Design |
Potential replication structure |
Important limitation |
| AB |
One baseline-to-intervention transition. |
Generally insufficient for replicated experimental control. |
| ABA |
Intervention introduction followed by withdrawal. |
Limited replication and no second treatment introduction. |
| ABAB |
Introduction, withdrawal, and reintroduction. |
Requires meaningful reversibility and adequate phase patterns. |
| Multiple baseline |
Staggered intervention introduction across tiers. |
Requires sufficiently distinct timing and interpretable untreated tiers. |
| Alternating treatments |
Repeated comparisons across systematically arranged treatment conditions. |
Requires careful control of sequence, carryover, and treatment differentiation. |
Each design has its own experimental logic. Researchers should not count phase transitions mechanically without considering whether they constitute meaningful demonstrations.
Does an ABAB design provide three demonstrations of effect?
An ABAB design can provide three opportunities to observe changes associated with experimental condition transitions: intervention introduction, withdrawal, and reintroduction.
For example, task completion may improve when prompting begins, decline when prompting is withdrawn, and improve again after prompting returns.
When those patterns are sufficiently clear, they may provide convincing evidence of a functional relationship within the studied case.
However, the phases must contain adequate observations, and the outcome must be capable of meaningful reversal.
An ABAB sequence that produces no interpretable change after withdrawal may fail to provide the intended replication.
The distinctions are discussed in AB, ABA, and ABAB single-case experimental designs.
How does replication work in a multiple-baseline design?
Multiple-baseline designs introduce an intervention at different points across participants, behaviors, or settings.
For example, three students may begin treatment after different numbers of baseline sessions.
If each student's outcome changes following their respective intervention introduction, while untreated tiers remain consistent with their existing patterns, the design may provide replicated demonstrations.
However, the timing must be sufficiently staggered to distinguish the introductions.
Intervention contamination, shared historical events, or outcomes already improving before treatment can weaken the interpretation.
See when multiple-baseline designs are preferable for the underlying experimental logic.
Do all three demonstrations need to show identical effects?
No. Exact numerical equality is not required.
Participants may differ in baseline levels, response magnitudes, variability, or the time required for an intervention effect to emerge.
For example, one student may improve immediately while another improves gradually.
These differences do not necessarily invalidate the experimental relationship, particularly when the expected response process allows such variation.
However, inconsistent patterns can weaken the evidence when they do not correspond with intervention manipulation or when alternative explanations remain plausible.
Researchers should examine consistency in the functional relationship rather than demand identical effect sizes.
What if only two of three tiers show convincing effects?
The evidence may be weaker than intended, but the interpretation depends on the full design and observed patterns.
For example, two participants may show clear changes following intervention introduction, while a third continues improving from an existing baseline trend.
The third tier may not provide an unambiguous demonstration.
Researchers should not describe three successful demonstrations merely because three intervention introductions occurred.
Nor should they automatically discard the third participant to make the results appear consistent.
The complete pattern should be reported, including uncertainty about the functional relationship and possible differences in intervention response.
Why do phase length and measurement quality matter?
Experimental demonstrations depend on sufficiently informative observations before and after condition changes.
For example, a phase containing only one or two observations may not adequately characterize natural variability or trends.
Methodological standards therefore include requirements concerning the number of data points within phases, although specific requirements depend on the design and standards version.
Measurement reliability also matters. Inconsistent scoring, changing outcome definitions, or observer drift can produce apparent effects unrelated to treatment.
Researchers should assess baseline stability and interpretability and preserve comparable measurement procedures across phases.
Additional replications cannot compensate automatically for systematically invalid measurements.
Can statistical significance establish successful replication?
Not by itself. A statistically significant result does not necessarily demonstrate that the outcome changed systematically with the intervention across the required experimental contrasts.
Similarly, a nonsignificant result does not automatically establish the absence of an intervention effect.
Visual analysis commonly considers level, trend, variability, immediacy, overlap, and consistency across comparable phases.
Statistical methods may supplement that assessment when appropriate to the design.
However, repeated observations may be autocorrelated, and ordinary independent-observation tests may produce misleading inference.
Researchers should interpret statistical results in conjunction with the experimental structure and observed patterns.
What is the difference between within-case and across-case replication?
Within-case replication occurs when intervention effects are demonstrated repeatedly through condition changes within the same case.
For example, an ABAB sequence may provide several intervention-related contrasts for one student.
Across-case replication examines whether comparable effects occur in additional participants or experimental units.
For example, several students may show improvements following separately staggered intervention introductions.
Within-case replication can strengthen causal inference for the individual studied.
Across-case replication can also contribute evidence about whether the intervention works beyond one case, although it does not automatically establish population representativeness.
What is the difference between direct and systematic replication?
Direct replication examines whether a finding can be reproduced under closely comparable conditions.
Systematic replication deliberately varies relevant participant, intervention, or contextual characteristics to investigate whether the effect persists.
For example, researchers may first replicate a prompting intervention with students having similar learning needs, then examine students from different academic programs.
Direct replication supports reproducibility under similar conditions. Systematic replication helps identify the scope and boundaries of the intervention effect.
These forms of replication contribute to broader evidence but should not be confused with the minimum demonstrations required within one experimental design.
How many studies are needed to establish an evidence-based practice?
There is no single requirement that applies across all evidence frameworks.
Horner and colleagues proposed influential criteria for identifying evidence-based practices from single-subject research in special education, emphasizing methodologically sound studies, multiple research teams, and replicated effects across participants and settings.
Those evidence-synthesis criteria are different from the minimum demonstrations needed to establish experimental control within one study.
Researchers should consult the particular framework being used rather than treating three demonstrations in one experiment as sufficient to establish an intervention as an evidence-based practice.
Broader recommendations require a cumulative assessment of research quality, consistency, and applicability.
Can additional replications compensate for weak initial demonstrations?
Additional well-designed replications may strengthen the evidence base, but merely repeating a weak design does not automatically eliminate its limitations.
For example, several AB studies may show improvement after intervention introduction, yet each may remain vulnerable to coincident historical events.
Consistent results across independent studies can be informative, but the absence of strong experimental control should still be acknowledged.
Researchers should improve design quality and measurement reliability rather than rely exclusively on accumulating more cases.
The relevant objective is repeated credible evidence, not a large count of methodologically ambiguous observations.
What if intervention effects differ across replications?
Heterogeneous responses may reveal important information about the intervention.
For example, digital prompting may improve task completion among students who struggle with task initiation but offer little additional benefit to students already performing independently.
Researchers should examine whether differences correspond to baseline performance, participant characteristics, intervention fidelity, or contextual conditions.
However, explanations developed after observing inconsistent results should be treated cautiously and, where possible, evaluated in subsequent studies.
Reporting unsuccessful replications is important because selective presentation can exaggerate the apparent consistency of intervention effects.
How does replication relate to generalizability?
Replicated demonstrations can strengthen confidence that an effect is not unique to one experimental transition or case.
However, replication within one participant does not automatically establish effectiveness across a population.
Similarly, effects reproduced among three closely similar participants may not extend to substantially different individuals or settings.
Researchers should distinguish experimental replication from statistical representativeness.
See whether single-case findings can generalize beyond the individuals studied for the broader inferential question.
How should replication be planned before data collection?
Researchers should identify the experimental design, the intended demonstrations of effect, and the number and timing of condition changes.
The protocol should establish outcome definitions, observation schedules, phase-change rules, intervention fidelity procedures, and planned analyses.
Where relevant, intervention start points may be randomized within prespecified permissible windows.
Researchers should also consider whether participant welfare permits the proposed withdrawal, delay, or alternation of intervention conditions.
The design should satisfy applicable methodological standards while remaining scientifically meaningful and ethically acceptable.
Replication is strongest when it is built into the experimental logic rather than declared retrospectively after favorable results appear.
