Is Having Many Participants but Very Few Clusters Enough in a Cluster-Randomized Trial?

Can Thousands of Participants Make Up for Randomizing Only a Few Groups?

Suppose you are evaluating a new educational technology program in six universities. Three universities are randomly assigned to implement the program, while three continue their existing practices. Each university contributes 1,000 students, giving the study an impressive total of 6,000 participants.

Is that enough to conduct a credible randomized trial?

The participant count looks substantial, but only six universities were independently randomized. Students within the same university share an assigned condition and may also share institutional characteristics that influence their outcomes. Consequently, the 6,000 observations do not provide the same experimental information as 6,000 independently randomized students.

The difficulty is not simply that the study has a small sample. It has a small number of independent allocation units, which creates particular limitations for estimating intervention effects and their uncertainty.

Many Participants Cannot Fully Replace Independent Randomized Clusters

Why the Number of Randomized Clusters Can Matter More Than the Total Participant Count

Participants and randomized clusters provide different kinds of information

In a cluster-randomized trial, intact groups such as schools, clinics, hospitals, or communities are assigned to intervention conditions. Individual participants provide outcome measurements, but they do not independently determine which treatment they receive.

If six schools are randomized, the allocation procedure operates on six schools, regardless of whether the study includes 300 or 30,000 students.

Additional students may help researchers estimate each school's mean outcome more precisely. However, they do not provide additional independent comparisons between intervention and control schools.

This distinction is particularly consequential when schools differ in leadership, resources, baseline performance, or other characteristics that influence outcomes.

Researchers must distinguish the unit of randomization from the unit of observation before calculating power or selecting an analytical model.

Intracluster correlation explains why additional participants provide diminishing returns

Participants within the same cluster may have correlated outcomes because they share institutional environments, personnel, practices, or other contextual influences.

This similarity is commonly summarized using the intracluster correlation coefficient (ICC).

For a simple parallel cluster-randomized design with approximately equal cluster sizes, the design effect is often approximated as follows:

The calculation illustrates the considerable redundancy that can arise when large numbers of participants share a common cluster.

It does not mean that 50.95 participants are identical, nor does it establish the exact statistical power of the trial. The approximation assumes a particular correlation structure and does not incorporate all small-sample considerations.

Why effective sample size can be misleading when interpreted literally

Researchers sometimes divide the total participant count by the design effect to obtain an approximate effective sample size.

This figure is useful for illustrating the loss of statistical efficiency. However, the study still has only six independently randomized clusters.

The effective sample size of approximately 118 does not imply that the trial is equivalent to an individually randomized experiment with 118 independent treatment assignments. It is a variance-based approximation, not a replacement for design-specific power analysis.

In particular, it does not resolve the difficulty of estimating treatment-effect uncertainty from only six randomized groups.

Increasing cluster size eventually produces diminishing statistical gains

For a fixed number of clusters and a positive ICC, the information gained by recruiting additional participants eventually approaches a limit under the simple exchangeable-correlation model.

Consider six clusters with an ICC of 0.05. Increasing enrollment from 100 to 1,000 participants per cluster multiplies the total participant count by ten, but the approximate effective sample size rises far less dramatically.

These hypothetical calculations assume equal cluster sizes and a constant ICC. They are not power estimates or empirical findings.

Under these assumptions, increasing enrollment from 600 to 6,000 participants raises the approximate effective sample size from about 101 to 118. The additional 5,400 observations contribute relatively little additional variance-equivalent information.

As cluster size becomes very large, the approximation approaches K / ICC, where K is the number of clusters. This mathematical limit illustrates diminishing returns; it does not establish that a trial with very few clusters can achieve valid inference merely by approaching that limit.

Few clusters make baseline imbalance more consequential

Randomization does not guarantee that intervention and comparison clusters will have identical baseline characteristics.

With only three universities per arm, one unusually well-resourced university could substantially influence the estimated difference between conditions. Differences in institutional leadership, student composition, infrastructure, or prior achievement may be unevenly distributed by chance.

Randomization remains a legitimate allocation procedure, but the small number of units limits how effectively chance imbalances can average out.

Prespecified adjustment for important baseline characteristics may improve precision. Matching or restricted randomization can also improve balance when appropriate.

However, these procedures cannot guarantee comparability on unmeasured characteristics, and extensive adjustment is difficult when the number of clusters is extremely small.

Conventional statistical tests may be unreliable with very few clusters

Many statistical methods used for cluster-randomized trials rely on approximations that perform better as the number of independent clusters increases.

Mixed-effects models, generalized estimating equations, and cluster-robust standard errors can be appropriate, but their conventional large-sample implementations may perform poorly with very few clusters.

Potential consequences include inaccurate standard errors, confidence intervals with incorrect coverage, and inflated false-positive rates.

Researchers may consider cluster-level analyses, appropriate finite-sample corrections, or randomization-based procedures. Each method has assumptions and limitations, particularly when the number of possible allocations is small.

There is no statistical adjustment that automatically makes six clusters equivalent to sixty clusters.

Randomization-based inference also has limits when allocations are few

Randomization-based inference evaluates treatment effects using the allocation mechanism rather than relying entirely on conventional distributional approximations.

It can be attractive in small-cluster trials because the randomization procedure is explicitly incorporated into inference.

However, the number of possible allocations may be very limited.

For six clusters allocated equally between two arms, the number of distinct allocations is:

The exact attainable significance levels depend on the randomization scheme, test statistic, and definition of the test. The example demonstrates that exact randomization-based inference is not a universal solution to having very few clusters.

Restricted randomization or matching may further change the set of allowable allocations, making careful inferential planning essential.

Why pseudoreplication is a serious risk

Pseudoreplication occurs when observations are treated as independent experimental replicates even though they do not represent independent units of treatment assignment or relevant experimental variation.

In a trial randomizing six universities, analyzing 6,000 student outcomes with a conventional independent-samples test that ignores university clustering can substantially underestimate uncertainty.

The problem is not that student-level analysis is inherently inappropriate. Student-level models can be used, provided they correctly account for the randomization and dependence structure.

The error arises when the analysis attributes independent experimental information to observations that share the same assigned treatment.

More clusters may improve precision more efficiently

When additional clusters are available, distributing participants across more randomized units may improve statistical efficiency.

For example, a design with 30 schools and 40 students per school may provide more information than one with six schools and 200 students per school, even though both include 1,200 students.

The actual advantage depends on the ICC, cluster-size distribution, outcome variability, and planned analysis.

Recruiting additional clusters may also be more expensive because each institution requires coordination, training, and implementation support.

The relevant planning question is therefore how many clusters are needed for the intended statistical objective, rather than how many participants can be recruited from the available institutions.

When very few clusters may still be defensible

A small number of clusters does not automatically make a study scientifically worthless.

For a pilot or feasibility trial, the primary objectives may concern institutional recruitment, intervention acceptability, implementation fidelity, data collection procedures, or preliminary estimates of outcome variability.

Such studies may provide useful information without being powered to establish intervention effectiveness.

In some unusual settings, a small-cluster randomized evaluation may also be informative when the intervention effect is large, baseline information is strong, or the analysis exploits a well-defined randomization structure. Nevertheless, the attainable precision and inferential limitations must be demonstrated rather than assumed.

The distinction is between conducting a limited but transparent study and presenting an inadequately powered trial as definitive causal evidence.

What to do when additional clusters are unavailable

Researchers should first determine whether the intervention genuinely requires allocation at the proposed organizational level. Sometimes a smaller unit of randomization can be justified without unacceptable contamination.

If the allocation level cannot change, possible responses include expanding recruitment to additional institutions, incorporating strong baseline measurements, using appropriate repeated-measures designs, or revising the study objective.

These options require separate methodological evaluation. Repeated measurements, for example, may improve precision but do not create new independently randomized clusters.

Researchers should also consider the operational costs and limitations of cluster allocation before committing to an infeasible effectiveness trial.

Six Universities and 6,000 Students: Is the Trial Adequately Powered?

Interpretation: The study's large participant count does not establish adequate statistical power. Six randomized universities provide a limited basis for estimating the intervention effect independently of university-level differences.

A formal power calculation, appropriate small-sample analysis, and explicit discussion of attainable precision are necessary. If those assessments show that the intended effectiveness claim cannot be supported, increasing student enrollment alone may offer little remedy.

Misconceptions About Large Participant Samples and Few Clusters

Evaluate Independent Allocation Information Before Recruiting More Participants

If your study has very few randomized clusters, do not begin by increasing the number of participants. First assess whether the available cluster count can support the intended estimand, effect size, precision, and statistical analysis.

Then evaluate whether additional clusters can be recruited or whether a smaller allocation unit is methodologically defensible. If neither is possible, reconsider the objectives and reporting claims.

The methodological justification should distinguish what the study can estimate from what researchers would ideally like to establish. A small-cluster trial may yield valuable information, but its interpretation must remain proportionate to the design.

Before Relying on Many Participants From Few Randomized Clusters

Frequently Asked Questions About Trials With Very Few Clusters

Large Participant Numbers Do Not Create More Randomized Units

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.