03 · What You Need to Know
How to Design a Credible Multisite Study of Rare Cases
Why are rare cases particularly difficult to recruit at one site?
The number of eligible cases depends on the frequency of the outcome, the size of the population served, the observation period, and the procedures used to identify cases.
A hospital may encounter only a few patients with a rare complication each year. A university may record only a small number of students experiencing a narrowly defined academic event.
Even when all eligible cases are recruited, the resulting sample may be too small for the planned comparisons or sufficiently precise estimates.
Extending recruitment at one institution may help, but prolonged accrual introduces its own challenges. Diagnostic standards, institutional policies, exposure patterns, and data systems may change over time.
Multisite recruitment can increase the number of cases observed during a common study period. Its value depends on whether the additional observations are sufficiently relevant and comparable.
What is a multisite research design?
A multisite study recruits participants or obtains data from two or more distinct research locations, institutions, or organizational settings under a coordinated protocol.
The sites might be hospitals, clinics, universities, schools, laboratories, or community-based organizations.
Multisite research is not a single statistical design. It can support cohort studies, case-control studies, randomized trials, cross-sectional studies, qualitative research, and other approaches.
For example, researchers investigating an uncommon outcome may identify cases across several hospitals and select controls according to a case-control sampling strategy.
Alternatively, several universities may contribute eligible participants to a longitudinal cohort.
The multisite arrangement addresses recruitment and data collection. The underlying research design still determines which comparisons and inferences are justified.
When does adding sites meaningfully improve feasibility?
Additional sites are most useful when they contribute eligible cases that would otherwise be unavailable and can provide the measurements required for the research question.
Researchers should estimate the expected number of eligible cases per site, the likely consent or participation rate, and the completeness of essential variables.
For example, ten institutions identifying five eligible cases each may collectively provide fifty cases during a defined period. However, the number ultimately usable for a particular analysis may be lower if exposure histories or primary outcomes are missing.
The expected gain should be assessed against the additional costs of coordination, approvals, training, data transfer, and quality assurance.
Adding a site that contributes only a few poorly documented cases may not improve the study as much as strengthening ascertainment at existing sites.
How should participating research sites be selected?
Site selection should follow the target population and scientific objective rather than convenience alone.
Researchers should consider the populations served, case volume, referral pathways, available records, measurement capacity, and willingness to follow a common protocol.
For example, recruiting exclusively from specialist hospitals may overrepresent severe or complicated cases because those institutions receive referrals from other facilities.
Similarly, recruiting only from highly resourced universities may exclude rare educational experiences occurring in less-resourced institutions.
These differences matter when researchers intend to generalize findings beyond participating sites.
Researchers should describe the site selection process and distinguish the population represented by the participating institutions from the broader population of interest.
Why must rare-case definitions be harmonized?
When an outcome is rare, even a small number of incorrectly classified cases can materially affect the analysis.
Suppose one hospital defines a complication using laboratory criteria while another relies on a clinician's recorded diagnosis. The two sites may identify different sets of patients despite using the same outcome label.
Likewise, universities may classify permanent withdrawal differently when students transfer, take temporary leave, or fail to reenroll.
Before recruitment, researchers should establish a common operational definition, eligibility period, inclusion and exclusion criteria, and verification procedure.
Where possible, standardized case-report forms, data dictionaries, and adjudication procedures can improve consistency.
However, harmonization should not conceal meaningful differences in the underlying events. If definitions cannot be reconciled, the researchers may need separate analyses or a narrower outcome definition.
How should case ascertainment be standardized?
Case ascertainment concerns how eligible cases are identified and verified.
Different sites may use electronic records, specialist referrals, registries, administrative databases, or direct screening.
Researchers should document which sources are searched, how frequently screening occurs, and how eligibility is verified.
For example, a site that searches all institutional records may identify cases that a site relying only on voluntary referrals misses.
Such differences can affect both the number and characteristics of recruited cases.
Where feasible, a common ascertainment protocol should be implemented. When procedures necessarily differ, their implications should be assessed and reported.
Can site differences introduce selection bias?
Yes. Sites may recruit participants through different pathways, and those pathways may depend on exposure, outcome severity, socioeconomic circumstances, or other characteristics.
For example, specialist hospitals may receive more severe cases, while community hospitals identify a broader range of presentations.
If severity also influences the exposure under investigation, combining cases without considering referral processes may distort the observed association.
Similarly, a university with an intensive student-support program may identify and retain students differently from an institution without comparable services.
Researchers should examine whether site participation or case identification conditions on variables related to both exposure and outcome.
Statistical adjustment for site does not automatically eliminate selection bias introduced by the recruitment process.
Should every site recruit the same number of cases?
Not necessarily. Equal recruitment quotas may be impractical when eligible cases are distributed unevenly.
A large referral center may identify many more cases than a smaller community institution.
Researchers can use site-specific recruitment targets based on expected case volume, the intended target population, and the information required for analysis.
However, allowing one site to dominate the sample may affect the population represented by the estimate.
If the research objective concerns a broader population, investigators should consider how unequal site contributions and selection probabilities affect interpretation.
Equal allocation, proportional recruitment, and deliberate oversampling serve different purposes. The choice should be justified rather than determined solely by convenience.
How should controls be recruited in a multisite case-control study?
Control selection must reflect the source population that produced the cases.
If cases are identified at several hospitals, researchers should determine whether controls should be sampled within each hospital's source population, across a common underlying population, or through another defensible sampling scheme.
For example, selecting controls from a single university to compare with cases drawn from several institutions may produce differences in exposure opportunity and baseline characteristics.
Site matching may be useful in some circumstances, but it can also restrict comparisons and introduce analytical requirements.
The appropriate strategy depends on the study base and intended effect measure.
Researchers should not assume that adding cases from multiple sites automatically permits controls from any convenient population.
These considerations are central to case-control research involving rare outcomes.
What does data harmonization require?
Data harmonization aims to make information collected across sites sufficiently consistent for the intended analysis.
It may involve common variable definitions, coding rules, assessment instruments, measurement windows, and procedures for resolving discrepancies.
For example, one institution may record examination performance as a percentage while another reports standardized scores. These values should not be combined without examining whether they represent comparable constructs and scales.
Similarly, medical sites may use different laboratory assays or diagnostic classifications.
Researchers should identify which differences can be addressed through defensible transformations and which represent genuine measurement incompatibility.
A shared data dictionary and documented harmonization decisions can support transparency and reproducibility.
Why does clustering matter in multisite studies?
Participants recruited from the same institution may resemble one another because they share policies, personnel, environmental conditions, or other contextual influences.
Consequently, observations may not be statistically independent.
Ignoring relevant within-site correlation can produce inappropriate standard errors and confidence intervals.
Depending on the design and number of sites, researchers may consider site indicators, mixed-effects models, generalized estimating equations, cluster-robust inference, or other appropriate methods.
These approaches address different statistical objectives and have different assumptions.
When only a few sites are available, conventional cluster-robust standard errors may perform poorly, and specialized small-sample methods or sensitivity analyses may be needed.
Site adjustment also does not automatically correct selection bias or unmeasured confounding.
Should site be treated as a fixed or random effect?
The choice depends on the estimand, sampling process, number of sites, and assumptions about site heterogeneity.
Fixed site indicators can account for differences in baseline outcome levels across observed sites, but they do not estimate effects of site-level characteristics that do not vary within sites.
Random-effects models represent site variation through a specified probability distribution and may support partial pooling.
However, random-effects assumptions may be difficult to assess when few sites contribute observations or when site selection is highly nonrandom.
Researchers should not choose a model solely because one option produces a smaller p-value.
The analytical strategy should follow the scientific question and the actual recruitment structure.
Can treatment or exposure effects differ between sites?
Yes. Differences in participant characteristics, intervention implementation, institutional resources, or environmental conditions may produce effect heterogeneity.
For example, an educational intervention may operate differently in universities with different digital infrastructure or instructional support.
Researchers should distinguish variation in baseline outcome levels from variation in the exposure effect itself.
However, rare-case studies may have too few events within individual sites to estimate site-specific effects precisely.
Prespecified heterogeneity analyses may be useful, but exploratory comparisons across many sites can produce unstable estimates.
Where site-level effects cannot be estimated reliably, researchers should acknowledge the limitation rather than interpret apparent differences as established findings.
Does multisite recruitment automatically improve generalizability?
No. Recruiting from several institutions may broaden the settings represented, but the sites may still be selected through convenience, referral networks, or institutional willingness to participate.
For example, a study involving ten specialist hospitals may not represent community healthcare settings.
Likewise, five highly selective universities do not necessarily represent the wider higher education sector.
Generalizability depends on the relationship between participating sites, recruited participants, and the target population.
Researchers should describe relevant site characteristics and explain which populations the findings plausibly represent.
A larger geographical footprint does not automatically establish population representativeness.
How should sample size and statistical power be planned?
Planning should consider the expected number of rare cases, the distribution of cases across sites, the intended comparison, and the relevant statistical model.
For case-control studies, exposure prevalence among controls and the expected effect size are important. For cohort studies, event incidence, follow-up, and the distribution of exposure may be central.
Clustering and unequal site contributions can affect precision.
Researchers should not simply multiply the expected cases per site by the number of sites and assume that the resulting count guarantees adequate power.
Simulation may be useful when case counts are small, site sizes differ substantially, or complex sampling and modeling procedures are planned.
What operational arrangements are necessary?
Multisite studies require coordination beyond ordinary single-site recruitment.
Researchers should establish responsibilities for screening, consent, measurement, data quality, secure transfer, and reporting.
Participating institutions may require separate ethics review, reliance arrangements, data-sharing agreements, or other approvals under applicable requirements.
Training and periodic quality checks can help ensure that sites interpret the protocol consistently.
A central coordinating team may maintain the data dictionary, monitor recruitment, and document protocol deviations.
These arrangements are not merely administrative. Inconsistent implementation can create systematic differences that undermine the scientific comparison.
When might multisite recruitment be the wrong solution?
Adding sites may be inappropriate when outcome definitions cannot be harmonized, exposure information is incompatible, or site selection creates substantial unresolved bias.
It may also be inefficient when the additional cases are too few to justify coordination costs.
Researchers should consider whether a population registry, existing cohort, extended recruitment period, or revised research question would provide stronger evidence.
Where the entire eligible population is small, comprehensive recruitment and careful estimation may be more defensible than attempting to satisfy an arbitrary conventional sample size target.
The decision should be based on the information required for the primary estimand, not simply the desire to report a larger sample.
