03 · What You Need to Know
Understanding Usual-Care Variation in Multi-Site Pragmatic Trials
What Is Usual Care?
Usual care refers to the treatments, services, or practices ordinarily available to participants outside the experimental intervention.
Its precise meaning depends on the setting and research question.
In clinical research, usual care may include routine consultations, medications, referrals, and other services provided according to established practice.
In educational research, analogous services may include conventional instruction, tutoring, academic advising, learning resources, and instructor feedback.
Usual care is not necessarily a single standardized treatment. It may represent a set of practices that vary across providers, institutions, and participants.
Researchers should therefore define what usual care includes rather than assume the term describes an identical experience for everyone.
Why Does Usual Care Vary Across Sites?
Institutions differ in resources, staffing, organizational policies, professional practices, and participant populations.
For example, universities may have different tutoring arrangements, class sizes, digital infrastructure, and expectations concerning instructor consultation.
These differences may influence the services participants receive even when they are assigned to the same comparison condition.
Variation may also occur within institutions because individual participants have different needs.
Such differences are not automatically protocol deviations. They may be ordinary characteristics of the treatment environment.
Why Might Pragmatic Trials Preserve Usual-Care Variation?
Pragmatic trials are designed to inform decisions under conditions relevant to ordinary practice.
If a new intervention will be introduced into institutions with different existing services, preserving that variation may help evaluate the strategy that decision-makers are considering.
For example, a university may want to know whether introducing an AI-assisted writing program improves outcomes when students retain access to their existing academic support services.
Standardizing those services across universities could require resources or restrictions that would not exist during routine adoption.
However, preserving usual care should be a deliberate design choice rather than a consequence of failing to specify the comparator.
Does Varying Usual Care Mean Every Site Has a Different Control Group?
In a practical sense, control participants may receive different combinations of services across sites.
However, the trial can still define a common comparison strategy, such as continuing locally available usual care without introducing the experimental program.
The comparison is then a policy or treatment strategy that permits local variation.
Researchers should distinguish a common strategy with heterogeneous implementation from a collection of unrelated or insufficiently specified comparison conditions.
The former may be appropriate for a pragmatic question. The latter may make interpretation unnecessarily difficult.
How Can Usual-Care Variation Affect the Treatment Effect?
The effect of introducing an intervention may depend on what participants would otherwise receive.
For example, an AI-assisted writing program may provide substantial additional support in a university with limited writing services.
At an institution with extensive individualized tutoring, the program may add less to the existing support system.
However, the direction of such differences cannot be assumed. Existing services may complement the intervention rather than substitute for it.
Researchers should distinguish plausible effect modification from empirically demonstrated differences in treatment effects.
What Types of Usual-Care Variation Matter?
| Variation |
Illustrative example |
Research consideration |
| Service availability |
Some universities have writing centers. |
Document which services are available at each site. |
| Service intensity |
Tutoring frequency differs. |
Assess relevant treatment exposure where feasible. |
| Provider practices |
Instructors use different feedback methods. |
Describe important differences in ordinary delivery. |
| Participant access |
Services are available only to certain students. |
Distinguish availability from actual receipt. |
| Institutional resources |
Technology and staffing differ. |
Consider implementation context and applicability. |
| Changes over time |
A university introduces new tutoring services. |
Record consequential changes during follow-up. |
Not every difference requires extensive measurement. Researchers should prioritize features plausibly related to the intervention's intended effects.
When Should Usual Care Be Standardized?
Standardization may be appropriate when differences would undermine the intended treatment comparison or create unacceptable inconsistencies in participant care.
For example, researchers may establish common rules for access to a study-specific service or ensure that necessary safety procedures are available to participants.
They may also standardize outcome assessment even when treatment delivery varies.
However, requiring identical receipt of all background services may be unrealistic or inconsistent with the intended pragmatic strategy.
The decision should be informed by whether additional treatments should be standardized across study groups.
Can Usual Care Differ Between Intervention and Control Participants Within the Same Site?
Yes, although the implications depend on why the differences occur.
If the experimental intervention intentionally changes subsequent service use, those changes may belong to the treatment strategy.
For example, a program that helps students recognize writing difficulties may increase their use of existing consultations.
By contrast, if instructors provide additional unrelated support to experimental participants simply because they know their assignment, the difference may represent an unintended co-intervention.
Researchers should distinguish expected consequences of assignment from deviations inconsistent with the intended comparison.
How Does the Estimand Determine the Appropriate Comparator?
An estimand defines the treatment effect or quantity the study seeks to estimate.
For a pragmatic trial, the estimand may concern the effect of introducing an intervention into existing services across a specified population of institutions.
In that case, locally varying usual care may be part of the comparison strategy.
A different study might seek the effect of the intervention when background services are held constant.
These questions are not interchangeable.
Researchers should define the comparator and relevant intercurrent events before selecting the analytical approach.
Does Usual-Care Variation Threaten Internal Validity?
Not automatically. In a properly randomized trial, variation in usual care may be consistent with the intended treatment strategy.
However, the allocation structure matters.
When participants are randomized within sites, treatment groups may share the same institutional environment, although individual service receipt can still differ.
When entire sites are randomized, institutional characteristics may become imbalanced between treatment conditions, especially with few clusters.
Researchers should therefore distinguish ordinary treatment heterogeneity from design problems involving allocation, recruitment, measurement, or analysis.
Why Does the Randomization Unit Matter?
Individual randomization within institutions and cluster randomization across institutions create different comparison structures.
In an individually randomized multi-site trial, both intervention and control participants may be present at each university.
Site-specific usual care can then form part of the shared context for both assigned conditions.
In a cluster-randomized trial, entire universities may be assigned to one condition.
Differences between institutions may therefore be associated with treatment assignment, particularly when the number of randomized sites is small.
Sample size planning and analysis must account for the randomization unit and relevant clustering.
Should Researchers Stratify Randomization by Site?
Site-stratified randomization may be useful in individually randomized trials when researchers want treatment allocation balanced within institutions.
However, it is not automatically necessary or feasible in every design.
For cluster-randomized trials, investigators may consider stratification, restricted randomization, or other appropriate allocation procedures based on important site characteristics.
The suitability of these approaches depends on the number of sites and the trial's design.
Researchers should plan allocation procedures before randomization and ensure the analysis reflects the design actually used.
How Should Site-Level Variation Be Analyzed?
Analytical methods should correspond to the randomization structure, target estimand, and number of participating sites.
For individually randomized trials, site adjustment or stratified analyses may be appropriate in some circumstances.
For cluster-randomized trials, analyses must address correlation among participants within clusters.
Researchers may also investigate whether treatment effects vary across institutions.
However, site-specific effect estimates can be imprecise, especially when individual sites have small samples.
Researchers should avoid assuming that adding a site indicator to a regression model resolves every problem associated with contextual variation.
Can Researchers Estimate Whether the Intervention Works Better at Some Sites?
Potentially. Treatment-effect heterogeneity may be relevant when usual-care services differ substantially.
For example, researchers might hypothesize that the incremental benefit of a writing intervention depends on existing tutoring availability.
Such hypotheses can be investigated through prespecified interaction analyses when the design and sample size support them.
However, observed differences across sites may reflect sampling variability, participant characteristics, or other contextual factors.
Researchers should distinguish exploratory patterns from reliable evidence of effect modification.
Should Researchers Adjust for the Amount of Usual Care Received?
Not automatically. Actual service receipt may occur after randomization and may be influenced by treatment assignment or participant needs.
For example, students who struggle academically may seek additional tutoring.
Adjusting for tutoring receipt using ordinary regression may introduce bias or remove part of the intervention's intended effect.
Researchers should distinguish baseline characteristics of usual-care systems from post-randomization service use.
Analytical adjustment should follow the estimand and relevant causal assumptions rather than serve as an automatic correction for variation.
How Can Researchers Describe Usual Care Without Overburdening Sites?
Researchers should identify services and practices most relevant to the primary outcome and treatment contrast.
Possible sources include institutional policies, staff interviews, service records, and participant reports.
For example, an educational trial may document the availability of writing consultations, tutoring, and instructor feedback.
It may be unnecessary to record every unrelated institutional service.
Documentation should be proportionate, feasible, and sufficiently detailed to support interpretation.
Where possible, information about actual service use should be distinguished from information about service availability.
Should Usual Care Be Measured Before the Trial Begins?
Baseline characterization is often useful because it establishes the services available before the experimental intervention is introduced.
Researchers can document institutional policies, typical service arrangements, and relevant differences between sites.
This information may support trial design, contextual interpretation, and prespecified analyses.
However, usual care can change during follow-up.
A baseline description alone may therefore be insufficient when important services are introduced, discontinued, or modified during the study.
What Happens if Usual Care Changes During the Trial?
Changes may occur because of institutional policies, staffing, funding, or other circumstances unrelated to the study.
For example, a university may introduce a new writing support service halfway through the intervention period.
Researchers should document consequential changes and determine whether they affect the intended treatment comparison.
However, changes should not automatically be treated as protocol violations if the comparison strategy permits ordinary service evolution.
The analysis and interpretation should remain consistent with the prespecified estimand.
Can Usual-Care Variation Cause Contamination?
Usual-care variation and contamination are different concepts.
Contamination occurs when intervention-specific components reach participants assigned to another condition.
For example, a control instructor adopting a specialized feedback procedure restricted to the experimental condition may create contamination.
By contrast, a control university offering its established writing center does not necessarily represent contamination.
Researchers should distinguish ordinary services from co-interventions and unintended cross-group exposure.
Can Standardizing Usual Care Reduce Real-World Applicability?
Yes. Imposing identical background services may create institutional arrangements that would not exist during ordinary adoption.
For example, requiring every university to provide the same tutoring frequency could introduce substantial new resources into some institutions.
Alternatively, restricting existing support services could remove resources that students would ordinarily use.
Either approach may change the comparison strategy.
Researchers should consider when treatment standardization becomes unrealistic and whether the resulting trial still answers the intended practical question.
How Does Usual-Care Variation Affect Generalizability?
Including institutions with different usual-care systems may provide evidence relevant to a broader range of implementation contexts.
However, variation alone does not guarantee generalizability.
The participating sites may still differ from the institutions to which policymakers wish to apply the findings.
Researchers should describe the distribution of important contextual characteristics and consider whether the study population of sites reflects the intended target.
The average treatment effect should be interpreted in relation to the sites and implementation conditions represented by the trial.
Should Usual Care Be Described in the Protocol?
Yes. The protocol should define the intended comparator and specify which services may continue, which are restricted, and which require documentation.
For a pragmatic trial, the comparator may be locally delivered usual care rather than one fixed treatment package.
Researchers should explain why this arrangement is appropriate for the intended decision.
The protocol should also identify important site characteristics, monitoring procedures, and the analytical approach.
Leaving usual care undefined makes it difficult to determine what the intervention was compared against.
How Should Usual-Care Variation Be Reported?
Reports should describe the comparison condition as actually implemented, not merely label it usual care.
Relevant information may include service availability, provider practices, treatment intensity, site differences, and changes during follow-up.
The TIDieR checklist can support detailed descriptions of intervention and comparator procedures.
Researchers should explain how usual-care variation relates to the target estimand and the applicability of the findings.
When important information is unavailable, that limitation should be acknowledged rather than assuming the comparator was uniform.
