When Should Clustering Be Accounted for in Research Design?

When Do Observations Become Too Related to Analyze as Independent Cases?

Suppose you collect academic performance data from 1,000 students attending 25 classrooms. You have a substantial number of student observations, but students within each classroom share teachers, instructional practices, assessment conditions, and institutional resources.

Or consider a clinical study involving patients recruited from several hospitals. Patients attending the same hospital may experience similar treatment protocols, staffing arrangements, and referral practices.

These shared circumstances can make observations within groups more alike than observations from different groups. If your research design ignores that possibility, the analysis may overstate precision or fail to distinguish individual and contextual relationships.

At what point should clustering influence the way you sample, recruit, measure, randomize, and analyze your data?

Account for Clustering Whenever Shared Units Can Affect the Design or Statistical Inference

How Clustering Changes Research Design, Sample Size, and Analysis

What Is Clustering in Research?

Clustering occurs when observations are organized into groups that may share characteristics, exposures, environments, or other influences.

Examples include students within classrooms, patients within hospitals, employees within organizations, households within communities, and repeated measurements within participants.

Clustering may arise naturally from social or biological organization, from the sampling procedure, or from the way an intervention is delivered.

The existence of groups does not guarantee substantial correlation for every outcome. However, grouping creates a plausible dependence structure that should be examined rather than ignored.

Researchers should distinguish the substantive presence of groups from the magnitude of statistical dependence under the proposed model.

Why Does Clustering Matter?

Many familiar statistical procedures assume independent observations or independent errors at a specified level.

When participants within the same group share unmeasured influences, their outcomes may be correlated.

For example, students in one classroom may respond similarly to an instructional intervention because they share a teacher. Patients within one hospital may experience similar outcomes because of institutional protocols.

Ignoring positive intracluster correlation commonly leads to underestimated standard errors in certain group-based comparisons.

However, the consequences are not universally identical. They depend on the estimator, sampling design, correlation structure, and level at which predictors vary.

Clustering may also affect the substantive relationship being estimated, particularly when individual-level and group-level associations differ.

When Does Clustering Arise?

Clustering can occur at more than one level. Students may be nested within classrooms, classrooms within schools, and schools within districts. Some designs also involve crossed classifications, such as students attending courses taught by several instructors.

Does Cluster Sampling Automatically Require Cluster-Aware Analysis?

Cluster sampling selects groups of units rather than drawing all individuals directly and independently from a population.

For example, researchers may randomly select schools and then recruit students within those schools.

This sampling process can create dependence and unequal selection probabilities that affect estimation and uncertainty.

Researchers should therefore account for the actual sampling design, including clustering, stratification, weights, and selection stages where relevant.

However, cluster sampling and cluster-randomized intervention studies are different designs. Sampling clusters determines how observations enter the study, while cluster randomization determines how interventions are assigned.

The appropriate analysis depends on which design features are present and what population quantity is being estimated.

What Is Intracluster Correlation?

The intracluster correlation coefficient, commonly abbreviated ICC, summarizes the similarity of observations within the same cluster under a specified statistical model.

In a simple random-intercept model for a continuous outcome, the ICC can be expressed as:

An ICC near zero indicates relatively little clustering under this model, while a larger positive ICC indicates stronger similarity among observations from the same group.

For binary outcomes or more complex models, ICC definitions and interpretations require additional care.

ICC values are outcome-specific and may differ between populations, institutions, and measurement periods.

Researchers should not assume that an ICC estimated for one outcome automatically applies to another.

How Does Clustering Affect Sample Size?

When observations within clusters are positively correlated, additional participants from the same cluster may provide less information than the same number of participants drawn independently.

For a simple equal-cluster-size design, an illustrative design-effect approximation is:

The approximation assumes equal cluster sizes and a particular correlation structure. Unequal clusters, repeated measurements, stratified sampling, and more complex models may require other formulas or simulation-based planning.

Sample size should therefore be based on the intended analysis rather than simply multiplying the number of groups by the number of participants.

Is It Better to Recruit More Clusters or More Participants Within Existing Clusters?

For many cluster-level comparisons, recruiting additional independent clusters provides substantial benefits because it increases replication at the level where important group variation occurs.

Increasing the number of participants within existing clusters can also improve precision, but gains may diminish as within-cluster correlation increases.

For example, increasing enrollment from 20 to 40 students per classroom does not provide the same information as doubling the number of classrooms.

The optimal allocation depends on recruitment costs, cluster sizes, ICC, outcome variability, and the estimand.

In cluster-randomized trials, the number of independently randomized clusters is especially important for treatment-effect inference.

Researchers should distinguish sample size based on participants from sample size based on clusters.

When Does Clustering Affect Randomization?

Clustering is particularly consequential when interventions are assigned to entire groups.

Suppose 12 schools are randomized to receive either a new instructional program or conventional instruction. Student outcomes are then measured within those schools.

The schools, not the individual students, are independently randomized.

Students within a school share treatment assignment and institutional circumstances. An analysis that treats them as independently randomized can produce misleading precision and potentially pseudoreplication.

The distinction between randomization and measurement units should be incorporated into the protocol, sample size calculation, and analysis plan.

Cluster randomization may also introduce recruitment or identification bias if individuals are selected after allocation is known.

What If the Exposure Varies Between Individuals but Outcomes Are Clustered?

Clustering can matter even when the exposure is measured or assigned at the individual level.

For example, students within the same school may differ in study habits, but their achievement outcomes may be correlated because of shared school conditions.

A model may need to account for school-level dependence while estimating an individual-level association.

Depending on the estimand, cluster-robust standard errors, fixed effects, random effects, or another suitable approach may be considered.

These methods answer different questions. School fixed effects, for example, focus on within-school variation and do not estimate the effects of school characteristics that remain constant within each school.

The presence of clustering should therefore inform both the variance estimator and the interpretation of model coefficients.

When Should You Use Multilevel Models?

Multilevel models are useful when researchers want to represent variation at several levels or examine relationships involving individual and group characteristics.

For example, a model might investigate how student motivation relates to achievement while allowing average achievement to vary across classrooms.

A random-intercept model represents differences in baseline outcome levels between groups. More complex models may permit certain relationships to vary between groups.

Multilevel models require assumptions about random effects, outcome distributions, and the relationship between predictors and unobserved group characteristics.

They are not automatically preferable whenever observations are clustered, particularly when the number of groups is very small or the intended estimand does not require modeling between-group variation.

When Are Cluster-Robust Standard Errors Appropriate?

Cluster-robust standard errors can adjust estimated uncertainty for certain forms of within-cluster dependence without requiring a fully specified correlation model.

They are commonly used in regression analyses where observations are grouped within institutions, communities, or other clusters.

However, conventional cluster-robust inference relies on asymptotic approximations that may be unreliable with few independent clusters.

Small-sample corrections, alternative reference distributions, or suitable randomization-based methods may be needed in particular designs.

Cluster-robust standard errors do not automatically remove confounding or correct an incorrectly specified estimand.

They primarily address uncertainty estimation under the assumptions of the estimator.

What If There Are Very Few Clusters?

Studies with few clusters face particular challenges.

For example, 1,000 students distributed across four schools may provide extensive individual measurements but limited independent information about school-level effects.

Conventional cluster-robust standard errors and complex random-effects models may perform poorly when the number of clusters is small.

For cluster-randomized studies, treatment effects may be difficult to distinguish from cluster-specific differences when only a few groups are assigned to each condition.

Appropriate responses may include recruiting more clusters, simplifying the model, using carefully justified small-sample methods, or narrowing the conclusions.

No analytical method can create independent clusters that were never sampled or randomized.

Can Clustering Exist Across Multiple Levels?

Yes. Many research settings involve nested, crossed, or overlapping group structures.

Students may be nested within classrooms and schools. Patients may receive care from multiple physicians within several hospitals. Employees may belong simultaneously to departments and project teams.

Some observations may therefore share more than one source of dependence.

A model that accounts only for one grouping level may be insufficient when another level materially affects inference.

Depending on the design, researchers may consider multilevel random effects, multiway cluster-robust variance estimation, or other suitable approaches.

The choice should follow the actual organization of observations and the intended comparison.

What About Spatial and Temporal Clustering?

Dependence can arise through geographic proximity or shared time periods even when observations do not belong to a conventional institution.

Neighboring communities may experience similar environmental conditions, and schools within one district may respond to the same policy changes.

Likewise, repeated observations from the same institution across years may share persistent characteristics and common temporal shocks.

Spatial correlation, serial correlation, and institutional clustering can require different analytical approaches.

Researchers should not assume that ordinary clustering by one identifier accounts for every relevant dependence structure.

How Does Clustering Affect Generalizability?

Clustering also has implications for population representation.

Recruiting many individuals from a small number of institutions may provide precise information about those institutions while offering limited coverage of the broader institutional population.

For example, a study involving 2,000 students from two universities cannot automatically be generalized to all universities merely because its individual sample is large.

The number, selection, and diversity of clusters matter for external validity.

Researchers should distinguish statistical dependence from sampling coverage. Correcting standard errors does not make a narrowly selected group of institutions representative of a broader population.

How Should Clustering Be Reported?

Report the number of clusters, cluster sizes, sampling or assignment procedures, and the relevant nesting structure.

When possible, provide descriptive information about variation between clusters and the ICC used in sample size planning.

Methods sections should explain how clustering was addressed analytically and whether small-cluster limitations were considered.

For cluster-randomized trials, the CONSORT extension provides specific reporting recommendations. For observational studies, STROBE emphasizes transparent reporting of design, sampling, statistical methods, and potential bias.

These details help readers assess whether the analysis avoids pseudoreplication and inappropriate claims of independent replication.

Planning a Study of Student Engagement Across Classrooms

Suppose the ICC for academic engagement is 0.08. Under the simple equal-cluster-size design-effect approximation:

The calculation illustrates how clustering can reduce precision for some estimates. It does not determine the standard error of the self-regulated learning regression coefficient, which also depends on how the predictor varies within and between classrooms.

If self-regulated learning varies substantially within classrooms, a within-classroom association may be estimated differently from an association based primarily on classroom averages.

The researcher should therefore select a model corresponding to the intended relationship rather than using the design effect as a universal correction.

Common Misconceptions About Clustering

How to Decide Whether Clustering Requires Design or Analytical Adjustments

Begin by identifying all meaningful grouping structures and determining whether observations share sampling, assignment, or outcome-generating influences.

Then assess which structures matter for the intended estimand and the proposed analysis.

Clustering should be treated as a design characteristic rather than an inconvenience discovered after statistical testing. Early recognition allows researchers to recruit enough independent units and choose an analysis that answers the intended question.

What to Verify About Clustering Before Conducting Research

Frequently Asked Questions About Clustering in Research

Account for Shared Sources of Dependence Before They Distort Your Inference

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.