03 · What You Need to Know
Balancing Intervention Consistency and Delivery Flexibility in Pragmatic Research
Why Is Intervention Standardization Relevant to Pragmatic Trials?
Intervention standardization establishes which aspects of treatment delivery should remain consistent across participants, providers, and study sites.
Without sufficient specification, different providers may implement substantially different activities under the same intervention label.
For example, an AI-assisted learning intervention might involve structured diagnostic feedback at one university, unrestricted chatbot access at another, and conventional tutoring at a third.
If these arrangements are not part of a deliberately defined treatment strategy, the resulting comparison may be difficult to interpret.
Standardization can therefore support treatment identification, provider preparation, intervention fidelity, and replication.
However, requirements that substantially depart from ordinary implementation may reduce the trial's relevance to the practical decision it is intended to inform.
Does Pragmatic Mean Minimally Standardized?
No. Pragmatism concerns the alignment between trial design and decisions made under conditions relevant to usual practice.
Some interventions are inherently standardized in ordinary use. A digital platform may deliver the same core functions to every participating institution. A clinical procedure may require consistent technical and safety standards.
Evaluating these interventions under their normal standardized arrangements can be entirely compatible with a pragmatic objective.
Conversely, imposing identical procedures on an intervention ordinarily tailored to individual needs may change the strategy being evaluated.
The appropriate question is not how little standardization the researchers can impose, but which delivery arrangements correspond to the intended use of the intervention.
How Does PRECIS-2 Address Intervention Delivery?
The PRECIS-2 framework includes flexibility of delivery as one of nine domains used to assess whether trial design choices align with explanatory or pragmatic objectives.
The domain concerns the extent to which intervention delivery resembles the flexibility available in usual care.
A more explanatory trial may impose restrictions beyond ordinary delivery to clarify treatment efficacy under specified conditions.
A more pragmatic trial may allow providers to deliver the intervention using the discretion they would ordinarily exercise.
However, PRECIS-2 does not establish that greater flexibility is always preferable.
Its purpose is to help trial designers assess whether their choices fit the trial's intended question.
What Is the Difference Between Standardizing Components and Standardizing Delivery?
Standardizing components concerns preserving the elements or functions that define the intervention.
Standardizing delivery concerns specifying how those components must be provided.
Component standardization
Ensuring that defining intervention content, functions, or decision rules remain present.
Delivery standardization
Requiring particular schedules, formats, providers, sequences, or procedures for delivering the components.
For example, a research writing intervention may require every participant to receive opportunities for diagnostic feedback and guided revision.
One university might provide these through weekly workshops, while another integrates them into ordinary course meetings.
Such variation may be acceptable if the intervention strategy permits both arrangements and there is a defensible basis for expecting them to preserve its essential functions.
Researchers should not assume that alternative delivery formats are equivalent merely because their stated purposes are similar.
Which Intervention Elements May Need Standardization?
The requirements depend on the intervention's theory, safety profile, and intended implementation strategy.
| Element |
Potential standardization |
Potential flexibility |
| Core content |
Required concepts or treatment components |
Contextual examples and supporting materials |
| Essential functions |
Diagnostic feedback and opportunities for revision |
Format through which those functions are delivered |
| Provider preparation |
Minimum intervention-specific competence |
Training arrangements suited to the institution |
| Timing |
Necessary sequence or treatment window |
Scheduling within permitted limits |
| Dose |
Prescribed dose or adaptive decision rules |
Additional exposure permitted by the strategy |
| Safety requirements |
Essential safeguards and escalation procedures |
Local implementation consistent with those requirements |
| Implementation documentation |
Common definitions and reporting requirements |
Locally feasible methods of collecting records |
These are illustrative distinctions rather than universal rules. An element that can vary in one intervention may be essential in another.
How Do Researchers Identify Core Intervention Components?
Researchers should begin with the intervention's theory of change and identify the components believed necessary to activate its intended mechanisms.
Prior empirical evidence, intervention development studies, feasibility testing, and consultation with practitioners may inform this process.
For example, a formative feedback intervention may depend on providing information about performance gaps and opportunities to act on that information.
The exact examples used during feedback may be adaptable, while omitting opportunities for revision could alter the intended learning process.
However, identifying a component as core does not prove that it is causally necessary.
The distinction should be justified and revised when stronger evidence becomes available.
Can Pragmatic Trials Require Fixed Intervention Doses?
Yes. A fixed dose may be part of the intervention that institutions would ordinarily adopt.
For example, a program may consist of six scheduled training sessions, each containing essential activities.
Evaluating that fixed package under ordinary delivery conditions can be pragmatic if the package corresponds to the intended implementation strategy.
Other interventions may deliberately vary dose according to participant need.
In those cases, researchers should specify the decision rules governing variation rather than require identical exposure.
Distinguishing dose delivered from dose received remains important regardless of trial orientation.
How Much Provider Discretion Is Appropriate?
Provider discretion should correspond to the intervention's intended use and the competence required to deliver it.
For example, ordinary instructors may be permitted to select disciplinary examples while following common feedback principles.
However, allowing instructors to replace diagnostic feedback with unrelated activities may change the intervention being evaluated.
Researchers should establish clear boundaries for permissible modifications and identify changes requiring review.
They should also consider whether ordinary providers have the skills necessary to make appropriate adaptations.
The degree of provider adaptation should be specified prospectively where feasible.
Should Every Site Use the Same Intervention Manual?
A common intervention manual can support consistency, but its level of detail should match the intended strategy.
For a fixed intervention, the manual may prescribe exact procedures, sequences, and delivery requirements.
For a flexible intervention, it may define essential functions, acceptable alternatives, and adaptation decision rules.
Using the same manual does not necessarily mean that every participant receives identical activities.
The manual should provide sufficient guidance for providers to implement the intended strategy without imposing unnecessary restrictions.
Can Intervention Standardization Improve Internal Validity?
Standardization may reduce unintended implementation variation and clarify the treatment contrast.
For example, ensuring that all intervention providers deliver required components can reduce uncertainty about what participants were offered.
However, internal validity is not determined by standardization alone.
Randomization, recruitment, missing outcome data, measurement, and analysis also affect the credibility of causal inference.
Moreover, ordinary variation that belongs to the specified treatment strategy does not automatically create bias for the effect of assignment to that strategy.
Researchers should distinguish implementation heterogeneity from methodological failures that compromise the randomized comparison.
Can Excessive Standardization Reduce Pragmatic Relevance?
Yes. Highly restrictive delivery requirements may create conditions unlike those under which the intervention would ordinarily operate.
For example, requiring expert facilitators, identical schedules, and weekly research-team supervision may produce an implementation package unavailable to most institutions.
If those supports are not part of the intended adoption strategy, the resulting effect may have limited relevance to routine implementation.
Researchers should consider how excessive standardization can reduce real-world applicability.
However, necessary training and support should not be removed merely to make a trial appear more pragmatic.
Can Excessive Flexibility Make a Pragmatic Trial Difficult to Interpret?
Yes. When providers make substantial modifications without preserving common functions, the intervention may become insufficiently defined.
For example, a program described as AI-assisted academic support might involve structured critical evaluation at one site and unrestricted chatbot use at another.
The trial may still estimate an assignment effect for the heterogeneous package as implemented, but the result may be difficult to interpret or reproduce elsewhere.
Researchers should distinguish appropriate flexibility from changes that undermine treatment fidelity.
Pragmatism does not remove the need for an identifiable treatment strategy.
How Should Fidelity Be Assessed in a Pragmatic Trial?
Fidelity should be assessed against the intended intervention, including any permitted flexibility.
For a fixed intervention, fidelity assessment may focus on whether required procedures were completed.
For a flexible intervention, it may examine whether essential functions were preserved and whether providers followed adaptation rules.
Possible methods include structured observation, provider records, digital logs, and adaptation documentation.
Researchers should avoid treating every difference in delivery as a fidelity failure.
The assessment should establish what was delivered without requiring uniformity that contradicts the intended strategy.
Should Fidelity Monitoring Be Less Intensive in Pragmatic Trials?
Not necessarily. Monitoring may be needed to understand implementation, ensure safety, or establish whether essential components were delivered.
However, monitoring can influence provider behavior, particularly when it includes frequent corrective coaching.
Researchers should distinguish fidelity measurement from implementation support.
If intensive supervision is provided only because the intervention is being studied, the resulting delivery conditions may differ from routine practice.
If supervision is an essential part of the treatment package, it should be included in the intervention specification and resource requirements.
How Does Standardization Affect the Target Estimand?
The estimand specifies the treatment effect the trial intends to estimate.
A pragmatic trial may seek the effect of offering a treatment strategy that permits ordinary provider discretion and participant adherence patterns.
Another trial may seek the effect of a more tightly specified intervention package.
These are different treatment strategies and may produce different effects.
Researchers should define the intended treatment conditions before deciding which delivery elements may vary.
Analytical methods should follow the estimand rather than attempt to redefine the intervention retrospectively according to observed implementation.
How Does Standardization Relate to Usual Care?
Pragmatic trials often evaluate an intervention alongside or against existing services.
Those services may vary across institutions and participants.
Researchers should specify whether the intervention is added to, replaces, or modifies usual care.
They should also distinguish intervention-specific requirements from background services that may continue according to ordinary practice.
For example, a digital writing program may have standardized core activities while universities retain different tutoring arrangements.
Such a design may be appropriate when the decision concerns introducing the program into existing institutional systems.
Can Different Sites Use Different Delivery Formats?
Potentially. Delivery variation may be appropriate when the intervention is intended to operate across settings with different resources or organizational arrangements.
For example, one university may deliver guided revision during scheduled classes while another uses supervised online sessions.
However, the research team should establish whether the alternative formats preserve essential functions and satisfy any necessary dose or timing requirements.
Differences should be documented when they may influence interpretation.
Researchers should not assume that all formats are equally effective without relevant evidence.
Should Researchers Analyze Outcomes According to Delivery Format?
Analyses comparing delivery formats may provide useful information about implementation variation.
However, formats may not have been randomly assigned.
Sites using different formats may also differ in staffing, resources, participant characteristics, and other outcome-related factors.
Consequently, observed differences between formats do not automatically identify causal effects.
If comparing delivery strategies is a central objective, researchers should consider a design that can evaluate those strategies appropriately.
How Should Adaptations Be Documented?
Researchers should record consequential modifications to intervention content, delivery, timing, or dose.
Documentation may include what changed, why, when, by whom, and whether the modification was planned or reactive.
The FRAME framework provides a structured approach to reporting intervention adaptations.
However, documentation should remain proportionate to the intervention and study resources.
The objective is to preserve information needed to understand the treatment actually delivered, not to create an administrative record of every minor difference.
How Should Intervention Standardization Be Reported?
Reports should identify the intervention's essential components, delivery requirements, permitted adaptations, and actual implementation.
They should explain which procedures were standardized and why those requirements were appropriate for the intended pragmatic question.
The TIDieR checklist provides guidance for describing intervention materials, procedures, providers, tailoring, modifications, and fidelity.
PRECIS-2 can help explain how flexibility of delivery fits the trial's overall design orientation.
Researchers should avoid claiming that a trial is pragmatic solely because providers were allowed to adapt the intervention.
Pragmatism should be established through the alignment of multiple design choices with the intended decision.
