03 · What You Need to Know
Understanding Variation, Adaptation, and Fidelity in Multisite Intervention Research
What Does Intervention Delivery Variation Mean?
Intervention delivery variation refers to differences in how an intervention is provided across research sites, practitioners, participant groups, or implementation periods.
Variation may concern delivery format, scheduling, personnel, communication channels, supplementary materials, or the sequence of activities. It may also involve differences in intervention intensity, duration, or content.
These differences are not methodologically equivalent. Changing the platform used to deliver an instructional message may preserve the intended intervention, while reducing the number of instructional sessions could alter its dosage and potentially its effects.
Researchers should therefore describe variation in terms of what changed, why it changed, and whether the modification affects an essential intervention function.
Why Might Different Research Settings Require Different Delivery Arrangements?
Research settings differ in their organizational structures, resources, populations, and established practices.
For example, universities may differ in academic calendars, instructional technologies, class sizes, faculty workloads, and student access to digital resources. Healthcare organizations may differ in staffing, referral pathways, equipment, and clinical workflows.
An intervention developed under one set of conditions may require adaptation to operate appropriately elsewhere.
These adaptations may improve compatibility with local systems, but they should not be assumed harmless. The research team must determine whether they preserve the mechanisms and activities believed to produce the intended outcomes.
How Is Standardization Different From Fidelity?
Standardization
Maintaining specified uniform procedures or conditions across intervention delivery.
Intervention fidelity
The extent to which delivery conforms to the intervention's intended specifications and essential functions.
Adaptation
A deliberate modification to intervention content, delivery, or implementation arrangements intended to improve fit with a particular context.
Implementation context
The organizational, social, technological, and other conditions that influence intervention delivery.
Standardization and fidelity are related but not identical. An intervention may retain fidelity while permitting different delivery arrangements if those arrangements preserve its essential functions.
Conversely, identical procedures may be implemented inconsistently or fail to produce the intended intervention exposure across settings.
Researchers should identify core intervention components and adaptable components before determining what constitutes acceptable delivery variation.
What Types of Variation May Be Acceptable?
The acceptability of a modification depends on whether it preserves the intervention's intended function and the conditions required for valid evaluation.
| Delivery dimension |
Illustrative variation |
Scientific consideration |
| Communication channel |
Feedback through a learning platform or scheduled consultation |
Does each channel provide the intended feedback function? |
| Scheduling |
Activities delivered on different days |
Are timing and frequency essential to the intervention? |
| Personnel |
Delivery by instructors with different professional backgrounds |
Are the necessary competencies and training maintained? |
| Language |
Materials translated for local participants |
Are meaning, comprehension, and intervention content preserved? |
| Examples |
Context-specific examples used to illustrate common principles |
Do examples preserve the instructional or behavioral mechanism? |
| Technology |
Different platforms used for equivalent activities |
Are access, functionality, and participant experiences comparable? |
| Dosage |
Different numbers or durations of sessions |
Does the modification change intervention exposure or the treatment contrast? |
| Core content |
Removal or replacement of essential intervention activities |
Does the modified intervention remain substantively equivalent? |
The table illustrates questions requiring evaluation rather than prescribing universally acceptable adaptations. Even a change in communication channel may be consequential when interpersonal interaction is central to the intervention mechanism.
When Should Intervention Delivery Be Strictly Standardized?
Strict standardization may be appropriate when the research question concerns a precisely defined intervention or when variation would undermine the intended comparison.
For example, a study evaluating the effect of a specific instructional sequence may need to preserve the order, duration, and content of activities.
Similarly, clinical interventions may require standardized dosage, administration, or monitoring for safety and scientific reasons.
Standardization may also be important during early efficacy research when investigators are establishing whether an intervention can produce the intended effect under specified conditions.
However, researchers should distinguish scientifically necessary uniformity from procedural details maintained solely for administrative convenience.
When Is Flexible Delivery More Appropriate?
Flexible delivery may be appropriate when the objective is to evaluate an intervention under routine conditions where some variation is expected.
For example, a pragmatic educational intervention study may allow instructors to integrate specified learning activities into their existing course schedules.
Such flexibility can improve the relevance of the findings to ordinary practice, provided essential intervention functions remain identifiable and delivery is adequately documented.
Researchers should consider whether the intervention study should reflect real-world practice when selecting its degree of standardization.
A flexible intervention should not be confused with an undefined intervention. The protocol must still specify what is being evaluated.
How Can Researchers Identify Essential Intervention Functions?
Essential functions are the processes or activities believed to be necessary for an intervention to achieve its intended effects.
Researchers may identify them through intervention theory, prior empirical evidence, developmental research, expert consultation, and examination of the mechanisms the intervention is intended to influence.
For example, an educational intervention may depend on students receiving timely, actionable feedback. The feedback function may be essential even if the communication channel can vary.
However, identifying a plausible mechanism does not establish that it has been empirically demonstrated. Researchers should distinguish theoretically proposed components from components supported by causal evidence.
The resulting specifications should explain which modifications preserve the intended intervention and which would create a substantively different treatment.
What Is the Difference Between Planned Adaptation and Protocol Deviation?
Planned adaptation involves modifications anticipated and permitted within the intervention design. Protocol deviation involves departures from approved or specified procedures that are not authorized within the protocol.
For example, a protocol may permit instructors to select examples relevant to their disciplines. Using different examples within the approved boundaries would be planned variation.
Omitting an essential intervention session without authorization would ordinarily be a deviation rather than a permissible adaptation.
Researchers should document both planned adaptations and deviations, but their implications may differ.
Material modifications may require ethics, institutional, sponsor, or regulatory approval before implementation, depending on the study and jurisdiction.
How Should Adaptations Be Documented?
Researchers should record what was modified, when the change occurred, who initiated it, why it was made, and whether it affected intervention content, delivery, or context.
The Framework for Reporting Adaptations and Modifications-Enhanced, known as FRAME, provides a structured approach to documenting modifications to evidence-based interventions.
Stirman and colleagues developed this framework to improve the characterization of modifications rather than treating adaptation as a single undifferentiated category.
Documentation should be proportionate to the study. Not every minor administrative adjustment requires an extensive qualitative record, but consequential changes should be identifiable and interpretable.
How Can Intervention Fidelity Be Measured Across Sites?
Fidelity assessment should correspond to the intervention's essential specifications.
Possible methods include structured observations, delivery checklists, recordings, administrative logs, participant reports, or independent ratings.
Researchers should distinguish whether a component was delivered from whether it was delivered competently and whether participants received or engaged with it as intended.
For example, a course may record that feedback was issued, but that record alone may not establish whether the feedback was timely, actionable, or understood.
Fidelity indicators should be defined prospectively where possible, with attention to measurement reliability and consistency across sites.
Can Different Delivery Modes Produce Different Effects?
Yes. Delivery arrangements may influence participant exposure, engagement, comprehension, or the mechanisms through which an intervention operates.
For example, synchronous coaching may provide opportunities for immediate clarification that are absent from asynchronous written feedback.
However, observing different outcomes across delivery modes does not automatically establish that the mode caused those differences.
Participants, providers, institutions, and resources may also differ. Appropriate comparative designs and analyses are needed when the objective is to estimate causal effects of delivery variation.
How Does Site-Level Variation Affect Statistical Analysis?
Participants within the same research site may share characteristics, providers, institutional policies, or delivery conditions. Their outcomes may therefore be correlated.
Ignoring this dependence can lead to inappropriate standard errors or misleading precision estimates.
Depending on the design, researchers may use multilevel models, generalized estimating equations, cluster-level analyses, or other appropriate methods.
Site-level variation may also be examined as a potential effect modifier, although subgroup and interaction analyses require adequate information and careful interpretation.
A small number of sites may limit the precision and reliability of estimates of between-site variation.
Should Researchers Treat Every Site as Delivering the Same Intervention?
Not automatically. The assumption of intervention equivalence should be supported by the protocol, delivery records, and relevant fidelity evidence.
If sites implement substantially different components, the study may be evaluating a family of related interventions rather than one uniform treatment.
That may be scientifically appropriate when the research question concerns flexible implementation, but the intended intervention and estimand should be defined accordingly.
Researchers should avoid interpreting a pooled average effect as though every site necessarily experienced the same treatment or obtained the same benefit.
Can Site-Level Adaptation Improve Equity and Accessibility?
Adaptation may allow interventions to accommodate differences in language, disability-related needs, technology access, or local resources.
For example, providing equivalent instructional materials through accessible formats may allow students who use assistive technologies to participate.
However, accessibility modifications should preserve the intervention's intended function and be evaluated when they alter measurement or delivery.
Researchers should distinguish adaptations that remove unnecessary barriers from modifications that change the active intervention.
Where appropriate, inclusive research design can strengthen the relevance of intervention evidence.
How Can Researchers Evaluate Contextual Influences?
Contextual information may include staffing, organizational leadership, technology, institutional policies, participant characteristics, and competing operational demands.
These factors can influence whether an intervention is delivered as intended and how participants experience it.
Researchers may collect contextual data through interviews, organizational records, observations, or structured implementation assessments.
However, context should not become an unrestricted list of explanatory variables. The selected factors should be linked to plausible mechanisms or specific research questions.
Researchers should consider how implementation context may help explain intervention outcomes without assuming that every observed association is causal.
Should Variation Be Considered During Sample Size Planning?
Yes, when the design involves multiple sites or delivery conditions that may influence outcomes.
Clustered data, variation in intervention effects, unequal site sizes, and planned interaction analyses can affect sample size and precision.
For example, a multisite study with substantial within-site correlation may provide less independent information than a simple calculation based only on the total number of participants suggests.
Researchers should use design-specific sample size methods and plausible assumptions about clustering and variation.
Broadening the number of sites does not automatically provide adequate power to estimate effects separately for each institution.
How Does Intervention Variation Affect External Validity?
Including multiple settings and realistic delivery arrangements may improve the relevance of findings to practice.
However, broader variation does not automatically establish generalizability. Sites may still be selected from a narrow range of institutions or may differ from the intended implementation population.
Researchers should describe the settings represented, the intervention versions delivered, and the conditions under which outcomes were observed.
When findings are applied to other settings, assumptions about contextual similarity and intervention equivalence should be considered explicitly.
When Does Delivery Variation Become Too Great?
Variation becomes problematic when it undermines the definition of the intervention, removes essential components, introduces unacceptable safety risks, or prevents the study from answering its intended question.
For example, if one site delivers a complete instructional program and another delivers only an introductory session, pooling outcomes under a single intervention label may conceal consequential differences.
The relevant threshold is not a universal percentage of variation. It depends on which components changed and whether those changes affect the intervention's function.
Researchers should evaluate whether excessive control limits implementation without replacing it with uncontrolled variation.
How Should Variation Be Reported?
Reports should describe the intervention's intended components, permissible adaptations, actual delivery, provider characteristics, and relevant contextual differences.
The Template for Intervention Description and Replication, known as TIDieR, provides guidance for reporting interventions in sufficient detail to support interpretation and replication.
Its items address intervention materials, procedures, providers, delivery modes, location, dosage, tailoring, modifications, and fidelity.
Reporting should distinguish planned flexibility from unplanned deviations and describe important differences across sites.
Transparent reporting supports interpretation but does not independently establish that all variations preserved intervention equivalence.
Watch Out
Do not assume that every local adaptation preserves the same intervention. Changes in dosage, essential content, delivery competence, or participant exposure may alter the treatment being evaluated. Conversely, requiring identical procedures where flexibility is scientifically acceptable may create unnecessary implementation barriers.
