How Does Clustering Affect Sample Size?

If You Recruit 400 Students from 10 Classrooms, Do You Really Have 400 Independent Observations?

You are designing a study to evaluate whether AI-generated formative feedback improves university students' writing performance. Your initial calculation suggests that 400 students would provide sufficient statistical power under an individually randomized design.

However, the intervention will be implemented at the classroom level. All students in a particular classroom will receive the same feedback condition, and different classrooms will be assigned to the intervention or comparison group.

You plan to recruit 40 students from each of 10 classrooms. Although the total is still 400, students within the same classroom share an instructor, instructional environment, assessment procedures, and potentially similar learning experiences.

Can you retain the original sample size calculation? Would recruiting more students from the same classrooms compensate for having relatively few classrooms?

The difficulty is that observations collected within groups may be statistically dependent. Clustering changes how much independent information a sample provides, which can affect precision, statistical power, and the number of participants or groups required.

Clustering Often Increases Sample Size Because Observations Within Groups Are Correlated

Why Clustered Observations Change Statistical Information and Recruitment Requirements

What Is Clustering in Research?

Clustering occurs when observations belong to shared groups or units that may influence their outcomes.

In educational research, students may be nested within classrooms, classrooms within schools, and schools within districts. Teachers may be nested within departments or universities.

Students within the same classroom may experience similar instructional methods, resources, institutional conditions, or peer influences.

Consequently, their outcomes may be more similar to one another than outcomes from students in different classrooms.

This dependence matters because many conventional statistical procedures and sample size formulas assume independent observations.

Clustering should be considered when the sampling process, intervention assignment, or data structure creates meaningful dependence among observations.

Is Cluster Sampling the Same as Cluster Randomization?

No. The two concepts concern different aspects of research design.

A study may use cluster sampling without being experimental, or cluster randomization without selecting clusters through a probability sampling design.

Both can create statistical dependence, but their sample size requirements differ according to the intended inference.

For further discussion of the sampling strategy, see when cluster sampling is appropriate.

Why Are Observations Within Clusters Often Correlated?

Members of a cluster may share characteristics or exposures that influence the outcome.

For example, students in the same classroom may share an instructor's feedback practices, assessment rubric, course schedule, and expectations about AI use.

These shared influences can produce similarity among student outcomes.

When outcomes are positively correlated, additional students from the same classroom provide some overlapping information.

This does not mean that their observations are identical or useless.

It means that the information gained from each additional participant may be less than it would be under independent sampling.

What Is the Intraclass Correlation Coefficient?

The intraclass correlation coefficient, commonly abbreviated ICC and represented by ρ, measures the degree of similarity among observations within the same cluster under a specified statistical model.

For a simple random-intercept model, the ICC can be expressed as the proportion of total outcome variance attributable to between-cluster variation.

An ICC near zero indicates little positive within-cluster similarity under this model.

A larger positive ICC indicates that cluster membership explains more of the observed variation.

The relevant ICC depends on the outcome, population, cluster definition, and statistical model. It should not be treated as a universal property of schools or classrooms.

Why Can a Small ICC Still Have a Large Effect?

The consequences of clustering depend on both the ICC and the number of observations within each cluster.

For example, an ICC of 0.05 may appear small, but its influence can become substantial when each classroom contains 40 students.

As cluster size increases, more pairs of observations share the same cluster-level influences.

The result is that even modest within-cluster correlation may materially increase the variance of a sample mean or treatment comparison.

Researchers should therefore avoid dismissing clustering merely because the estimated ICC is numerically small.

What Is the Design Effect for Clustering?

The design effect describes the ratio of an estimator's variance under a particular sampling design to its variance under a specified reference design.

For equal-sized clusters and a simple model with positive intraclass correlation, a commonly used approximation is:

Under the assumptions of this approximation, the variance is approximately 2.95 times that of the relevant independent-observation reference design.

This does not mean that 2.95 is the correct adjustment for every estimator, cluster-randomized trial, or multilevel analysis.

The distinction is explored further in what the design effect means for sample size.

How Does Clustering Affect Effective Sample Size?

Effective sample size is a way of expressing the precision of a complex sample relative to a reference design.

For a simple clustering approximation:

This calculation provides an approximate variance-equivalent sample size for the specified comparison.

It does not mean that only 136 students participated, nor that the dataset can literally be analyzed as 136 independent observations.

Effective sample size depends on the estimator and may differ across outcomes or statistical analyses.

Why Does the Number of Clusters Matter So Much?

In a cluster-randomized experiment, treatment assignment occurs at the cluster level.

For example, if ten classrooms are randomized, the study has ten independently assigned classroom units, even if those classrooms contain hundreds of students.

The number of clusters is particularly important for estimating between-condition effects and their uncertainty.

Recruiting additional students within the same classrooms may improve precision to some extent, but it does not create additional independently randomized classrooms.

With very few clusters, standard large-sample approximations may perform poorly, and specialized small-sample corrections or alternative analyses may be necessary.

Researchers should therefore plan both the number of clusters and the number of participants within each cluster.

Is Recruiting More Clusters Better Than Recruiting More Participants per Cluster?

Often, recruiting additional clusters can be more statistically valuable than enlarging existing clusters when within-cluster correlation is positive.

For example, a study involving 20 classrooms with 20 students each may provide different, and often more useful, information for a classroom-level treatment comparison than a study involving ten classrooms with 40 students each.

Both designs contain 400 students.

However, the first includes twice as many independently randomized classroom units.

The relative advantage depends on the ICC, cluster sizes, costs, and statistical analysis.

Researchers should not assume that increasing the number of students within existing clusters can indefinitely compensate for a small number of clusters.

What Happens When Cluster Size Increases?

Under the equal-cluster approximation, the design effect increases with cluster size when the ICC is positive.

For example, with ICC = 0.05, a cluster size of ten produces DEFF = 1.45.

A cluster size of 40 produces DEFF = 2.95.

A cluster size of 100 produces DEFF = 5.95.

These values illustrate diminishing statistical returns from recruiting more individuals within the same cluster.

However, larger clusters may still be practical or cost-efficient when recruiting additional clusters is expensive.

The optimal balance depends on both statistical information and recruitment costs.

How Do Unequal Cluster Sizes Affect Sample Size?

Real classrooms and institutions rarely contain identical numbers of participants.

For example, one classroom may contain 18 students while another contains 45.

Unequal cluster sizes can affect statistical efficiency, particularly when large clusters contribute disproportionately to the sample.

Under certain approximations, the coefficient of variation of cluster size can be incorporated into the design effect.

This approximation is useful in some cluster-sampling and cluster-trial planning settings but does not replace a design-specific calculation.

Researchers should consider whether cluster size is associated with outcomes or treatment effects, because such relationships may introduce additional complications.

How Does Clustering Affect Statistical Power?

Statistical power depends on the distribution of the planned test statistic under a specified alternative.

Clustering changes that distribution by introducing dependence among observations.

If the analysis incorrectly assumes independence, standard errors may be underestimated and statistical significance overstated.

When clustering is incorporated appropriately, the sample needed to maintain a specified power may increase.

However, the increase depends on the estimand, cluster allocation, number of clusters, ICC, and analysis method.

See statistical power in study design for the broader principles.

Does Clustering Always Increase Sample Size?

No universal increase applies to every design.

Positive intraclass correlation commonly reduces precision relative to independent observations when estimating an overall mean or comparing cluster-assigned treatments.

However, stratification, blocking, covariate adjustment, and different estimands may change the relevant variance comparison.

Some analyses focus on within-cluster contrasts, which can benefit from controlling shared cluster characteristics.

Researchers should therefore specify the estimator and reference design before claiming that clustering produces a particular inflation factor.

How Does Cluster Sampling Differ from Multistage Sampling?

Cluster sampling selects groups as sampling units. Multistage sampling selects units through successive stages, potentially including schools, classrooms, and students.

For example, researchers may first sample universities, then departments within universities, and finally instructors within departments.

Such designs may create multiple levels of clustering and unequal selection probabilities.

The sample size and analysis should reflect the actual selection structure.

See when multistage sampling is necessary.

How Does Clustering Interact with Repeated Measures?

Educational studies may involve repeated measurements nested within students and students nested within classrooms.

For example, writing scores may be measured at baseline, post-intervention, and follow-up for students assigned by classroom.

The design contains both within-person dependence and classroom-level dependence.

Accounting for only one level may produce inaccurate sample size requirements.

Mixed-effects models or other appropriate longitudinal methods may represent multiple levels of correlation.

The sample size calculation should match the planned analysis and the structure of the data.

See how repeated measures affect sample size.

Can Baseline Adjustment Improve Efficiency in Clustered Studies?

Potentially. Baseline measurements and cluster-level covariates may explain variation relevant to the treatment comparison.

For example, adjusting for classroom-average baseline writing proficiency may improve precision when that characteristic predicts follow-up performance.

However, the gain depends on the relationship between baseline and follow-up outcomes, the number of clusters, and the statistical model.

Researchers should use credible assumptions about the expected improvement rather than apply an arbitrary reduction to the calculated sample size.

Covariate adjustment does not remove the need to account for cluster randomization or within-cluster correlation.

How Should the ICC Be Estimated for Planning?

Relevant previous studies, pilot data, and institutional datasets may provide estimates of intraclass correlation.

However, ICC values can differ substantially across outcomes, populations, educational settings, and measurement occasions.

For example, an ICC for mathematics achievement may not apply directly to student perceptions of AI feedback.

Small pilot studies with few clusters may produce imprecise ICC estimates.

Researchers should therefore evaluate plausible ranges and consider sensitivity calculations.

Using an unrealistically small ICC merely to reduce the recruitment requirement is not defensible.

How Can Sensitivity Analysis Help with Uncertain ICC Values?

Sensitivity analysis examines sample size or precision under alternative ICC assumptions.

For example, researchers may calculate requirements under ICC values of 0.01, 0.05, and 0.10.

The resulting design effects can differ substantially when cluster sizes are large.

This analysis helps identify whether the proposed design remains adequate under plausible correlation structures.

If the study is adequately powered only under an unusually optimistic ICC, the research team should reconsider its recruitment strategy.

Reporting a range of plausible requirements is preferable to presenting one uncertain estimate as definitive.

What If the Study Has Very Few Clusters?

Very small numbers of clusters create difficulties beyond ordinary individual-level sample size inflation.

For example, a trial involving four schools per treatment condition may have limited degrees of freedom for estimating between-cluster uncertainty.

Cluster-robust standard errors and conventional mixed-model tests may require small-sample corrections or specialized inference.

Even a large number of students per school may not overcome the limitations created by very few independently assigned units.

Researchers should evaluate the number of clusters directly rather than rely exclusively on the total number of students.

Where the cluster count is severely constrained, the research question or design may need reconsideration.

Does Clustering Affect Observational Studies?

Yes. Clustering matters whenever the data structure creates relevant dependence, even without random assignment.

For example, a survey of teachers from several schools may contain correlated responses within each institution.

The analysis may require survey design methods, multilevel modeling, generalized estimating equations, or appropriate cluster-robust inference.

However, the correct approach depends on whether the objective concerns population estimation, regression coefficients, or contextual differences.

Clustering should not be ignored simply because the study is descriptive or observational.

How Does Clustering Affect Population Representativeness?

Clustering and representativeness are related but distinct concerns.

A probability sample of schools may support population inference when selection probabilities, coverage, nonresponse, and estimation procedures are appropriate.

However, recruiting many students from a few conveniently selected schools does not establish representativeness merely because the total sample is large.

The design effect concerns sampling variance relative to a reference design. It does not automatically correct selection bias.

Researchers should distinguish statistical efficiency from the validity of population generalization.

How Should Clustering Be Reported in the Methodology?

The methodology should identify the cluster unit, number of clusters, expected or achieved cluster sizes, and the basis for the ICC assumption.

For cluster-randomized studies, report how clusters were assigned to conditions and how many were allocated to each group.

The sample size calculation should describe the design effect or other method used, desired power, significance level, effect of interest, and relevant small-sample adjustments.

Where cluster sizes vary, the planning assumptions and analytical treatment of that variation should be explained.

The results should report both cluster counts and participant counts.

Transparent reporting helps readers evaluate whether the study had sufficient independent units for its intended conclusions.

Comparing Two Classroom Recruitment Plans with the Same Number of Students

The design effects are calculated using DEFF = 1 + (m − 1)ρ.

For ten classrooms, DEFF = 1 + 39 × 0.05 = 2.95.

For twenty classrooms, DEFF = 1 + 19 × 0.05 = 1.95.

The approximate effective sample sizes are 400 / 2.95 ≈ 136 and 400 / 1.95 ≈ 205.

Under this simplified variance-equivalence calculation, the twenty-classroom arrangement provides greater precision for an appropriate overall mean estimator.

If classrooms are randomized to feedback conditions, the larger number of independently assigned classrooms may also improve the treatment comparison.

However, the table is not a complete cluster-randomized trial power calculation. It does not specify treatment allocation, outcome effect size, cluster-level degrees of freedom, or the exact analysis.

The researcher should use those additional assumptions to determine whether either design has adequate power.

The example demonstrates why the distribution of participants across clusters matters, even when the total participant count remains unchanged.

Common Misunderstandings About Clustering and Sample Size

How to Plan Sample Size When Participants Are Grouped

Begin by identifying the level at which observations are sampled, assigned to conditions, and analyzed.

In the methodology section, report both the cluster count and the participant count, along with the assumed ICC and calculation method.

Do not interpret a large number of individual observations as proof that the study contains sufficient independent information.

The most useful recruitment decision may be to increase the number of clusters rather than simply enlarge existing ones.

Before Finalizing a Clustered Sample, Check the Independent Units

Frequently Asked Questions About Clustering and Sample Size

Clustered Samples Need Enough Independent Groups, Not Just Enough Individuals

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.