What Happens When Treatment Is Assigned to Groups but Outcomes Are Measured on Individuals?

Can You Evaluate an Intervention Using Individuals When Entire Groups Received the Treatment?

Suppose a researcher randomly assigns 12 classrooms to a new instructional strategy and another 12 classrooms to conventional instruction. Academic performance is then measured for 720 students across those classrooms.

The intervention was assigned to classrooms, but the outcomes belong to individual students. Should the researcher analyze 720 independent observations, 24 classrooms, or both levels simultaneously?

The same problem arises when hospitals adopt different care protocols, organizations implement workplace training, or communities receive public health programs while outcomes are measured among their members.

Group-level treatment assignment can be scientifically appropriate, but it changes the structure of independent replication, the assumptions required for causal inference, and the interpretation of treatment effects.

Analyze Individual Outcomes While Respecting Group-Level Treatment Assignment

How Group-Level Treatment Assignment Changes Intervention Research

What Is Group-Level Treatment Assignment?

Group-level treatment assignment occurs when an intervention or exposure condition is assigned to an entire collective entity rather than independently to each member.

Examples include assigning instructional programs to classrooms, infection prevention protocols to hospitals, or workplace policies to organizations.

When groups are randomly allocated, the design is commonly called a cluster-randomized trial.

When groups receive interventions without random assignment, the design may instead be quasi-experimental or observational, depending on the allocation mechanism and comparison strategy.

The distinction is essential. Group-level treatment does not automatically establish randomization or causal identification.

What Is the Unit of Treatment Assignment?

The unit of treatment assignment is the entity to which an intervention condition is allocated.

For example, if a school is assigned to implement a digital learning policy, the school is the assignment unit even though student outcomes are measured individually.

In a randomized design, the independently randomized groups provide treatment assignment replication.

Researchers should distinguish the unit of randomization from the unit of measurement.

The level of treatment assignment affects both the causal contrast and the statistical information available for estimating its effect.

What Is the Unit of Outcome Measurement?

The outcome measurement unit is the entity for which the outcome is defined and recorded.

For example, student achievement may be measured individually even when the classroom receives the intervention.

Patient recovery may be measured individually even when a hospital implements a common treatment protocol.

These individual observations can contain substantial variation and allow adjustment for relevant baseline characteristics.

However, observations within a group may share treatment exposure and contextual influences.

The number of outcome observations should therefore be distinguished from the number of independently assigned groups.

How Do the Assignment and Outcome Units Differ Across Settings?

These arrangements are not inherently problematic. The methodological challenge is to align the design, analysis, and interpretation with the actual assignment structure.

Why Are Individual Outcomes Within Treated Groups Correlated?

Individuals within the same group may share characteristics that influence their outcomes independently of the intervention.

Students in one classroom share teachers, instructional resources, peer environments, and assessment conditions.

Patients within a hospital may share clinical protocols, staffing patterns, and institutional resources.

Such common influences can produce intracluster correlation.

Because group members also share treatment assignment, ignoring within-group dependence can lead to misleading uncertainty estimates for the treatment comparison.

Researchers should assess the implications of clustering for research design before determining sample size and analysis.

Why Doesn't the Number of Individuals Equal the Number of Independent Treatment Replications?

Independent treatment replication arises from the assignment mechanism.

If 20 classrooms are independently randomized, the study has 20 randomized classroom units, regardless of whether each classroom contains 10 or 100 students.

Student outcomes provide information about classroom-level intervention effects, but students within the same classroom do not independently receive randomized assignments.

Consequently, treating all student outcomes as independently randomized can exaggerate the evidence supporting a treatment contrast.

This can create pseudoreplication, particularly when the analysis ignores the level at which treatment was assigned.

Correcting the model requires more than changing the reported sample size. The statistical analysis must reflect the independent assignment structure.

What Happens If You Use an Ordinary Individual-Level t-Test?

An ordinary independent-samples t-test generally assumes independent observations within and between treatment groups.

When treatment is assigned to clusters, that assumption may be inappropriate because outcomes within the same cluster are correlated.

With positive intracluster correlation, ignoring clustering commonly underestimates uncertainty for group-level treatment comparisons.

The resulting confidence intervals may be too narrow and p-values too small.

However, the magnitude of the problem depends on the number and size of clusters, intracluster correlation, and treatment allocation.

The appropriate response is to use an analysis suited to the actual design, not merely to apply a different significance threshold.

How Does Group Assignment Affect Sample Size Planning?

Sample size planning should consider the number of groups, number of individuals per group, outcome variability, and intracluster correlation.

For a simple equal-cluster-size setting, an illustrative design-effect approximation is:

The formula assumes equal group sizes and a particular correlation structure. Actual treatment-effect precision also depends on allocation balance, covariate adjustment, between-cluster variability, and the proposed estimator.

Researchers should consider the relative importance of participants and clusters in sample size planning.

Is It Better to Recruit More Groups or More Individuals Within Groups?

For many cluster-randomized comparisons, additional independently randomized groups are especially valuable because they increase treatment assignment replication.

Increasing the number of individuals within existing groups can improve outcome precision, but gains may diminish when intracluster correlation is positive.

For example, doubling the number of students in four randomized classrooms does not create eight randomized classrooms.

With very few groups, the treatment effect may be difficult to distinguish from group-specific characteristics.

The optimal balance depends on costs, expected effect size, ICC, group-size variation, and the analytical method.

Sample size planning should occur before the number of groups is fixed whenever feasible.

What Is the Difference Between a Cluster-Level and Individual-Level Analysis?

A cluster-level analysis summarizes outcomes within each group and compares those summaries across treatment conditions.

For example, the researcher might calculate the mean achievement score for each classroom and then compare the classroom means between intervention and comparison groups.

An individual-level analysis retains student observations while accounting for classroom assignment and dependence.

Both approaches may be appropriate, but they can differ in weighting, covariate adjustment, precision, and estimand.

Cluster-level analysis can be comparatively transparent, particularly with limited group counts, but it may discard some individual-level information.

Individual-level models may preserve more information but require appropriate treatment of clustering and model assumptions.

When Are Mixed-Effects Models Appropriate?

Mixed-effects models can represent group-level heterogeneity through random effects.

For example, a model of student achievement may include a classroom-level random intercept and student-level baseline covariates.

This can account for shared classroom influences while estimating a specified intervention association.

However, mixed-effects models rely on assumptions about random effects, outcome distributions, and model specification.

With very few randomized groups, estimation and inference may be sensitive to those assumptions.

Researchers should consider the intended conditional or marginal estimand and the number of independent groups before choosing a model.

When Are Generalized Estimating Equations Appropriate?

Generalized estimating equations, or GEE, are often used to estimate marginal or population-averaged relationships for correlated outcomes.

For example, a GEE model may estimate an intervention-associated difference in student achievement while recognizing classroom-level clustering.

GEE specifies a working correlation structure and may use robust variance estimation under suitable conditions.

However, conventional robust standard errors can be unreliable with small numbers of independent clusters.

Small-sample corrections or other approaches may be needed, and the choice should reflect the design.

GEE and mixed-effects models may estimate different parameters, especially for nonlinear outcomes, and should not be treated as interchangeable.

Can Cluster-Robust Standard Errors Solve the Problem?

Cluster-robust standard errors can adjust uncertainty estimates for certain forms of within-cluster dependence.

They may be useful when an individual-level regression model otherwise corresponds to the intended estimand.

However, they do not automatically remove confounding, correct selection bias, or create independent treatment assignments.

Conventional cluster-robust inference also relies on assumptions that can be problematic when only a small number of groups is available.

Researchers should not assume that selecting a clustered standard-error option in statistical software resolves every design limitation.

What If Only One Group Receives Each Treatment?

This is a particularly serious design limitation.

Suppose one school receives an intervention and another serves as the comparison. Hundreds of students are measured within each school.

Because treatment condition is completely aligned with school identity, the observed difference may reflect the intervention, preexisting school characteristics, or other school-specific influences.

The study does not contain independent school-level replication within treatment conditions.

No statistical adjustment can manufacture additional independently randomized schools from the student measurements.

Researchers may still describe the observed difference, but strong causal claims require assumptions or additional design information that the simple two-school comparison does not provide.

How Can Recruitment After Assignment Introduce Bias?

Cluster-randomized studies sometimes assign groups before identifying or recruiting individual participants.

When recruiters know the assigned condition, recruitment procedures or participation decisions may differ between intervention and comparison groups.

For example, teachers may encourage highly motivated students to complete an intervention evaluation, or students may be more willing to participate when they know they will receive a new learning technology.

These differences can undermine comparability even when the original group-level randomization was valid.

Potential safeguards include identifying eligible participants before randomization, standardized recruitment procedures, and masking recruiters to allocation where feasible.

The broader implications of recruiting individuals within selected groups are therefore relevant to intervention validity.

What If Treatment Assignment Is Not Randomized?

When groups receive interventions without random assignment, group-level confounding becomes an additional concern.

For example, universities that voluntarily adopt an AI tutoring system may differ systematically from institutions that do not adopt it.

Differences in institutional resources, student characteristics, leadership, and teaching practices may affect outcomes independently of the intervention.

Accounting for clustering addresses statistical dependence but does not establish causal exchangeability.

Quasi-experimental approaches such as difference-in-differences, interrupted time series, or other suitable designs may sometimes strengthen causal inference when their assumptions are defensible.

The analytical strategy must address both group-level assignment and the process determining treatment exposure.

What Is the Difference Between Individual-Average and Cluster-Average Effects?

A treatment effect can be defined for an average individual or for an average cluster.

These estimands may differ when clusters vary in size or when treatment effects differ across groups.

For example, a school with 1,000 students contributes more individuals than a school with 100 students.

An individual-weighted analysis may give the larger school more influence, while an equally weighted cluster analysis gives both schools the same weight.

Neither approach is universally correct.

The appropriate estimand depends on whether the scientific or policy question concerns the average student, average school, or another target population quantity.

Researchers should define the weighting and estimand before examining which approach produces more favorable results.

What About Spillovers and Interference?

Group-level interventions may influence individuals through interactions with other members of the group.

For example, an instructional strategy may change peer collaboration, classroom participation, or teacher behavior.

These within-group spillovers may be part of the intervention's intended effect.

However, spillovers between randomized groups can complicate causal interpretation. Students from intervention and comparison classrooms may share learning materials or interact outside class.

Researchers should specify the intervention contrast and consider whether treatment assignment in one group can affect outcomes in another.

Randomizing intact groups may reduce some contamination but does not guarantee that interference is absent.

How Should Group-Level Treatment Studies Be Reported?

Report the number of groups eligible, randomized or assigned, and analyzed, alongside the number of individuals measured within those groups.

Describe the assignment mechanism, recruitment timing, group sizes, outcome measurement, and relevant clustering.

Explain whether the primary analysis targets an individual-average or cluster-average effect and how unequal group sizes were handled.

For randomized studies, report allocation procedures, participant flow, and analysis methods consistent with the cluster-randomized CONSORT extension.

For nonrandomized studies, describe the assignment process and the assumptions supporting causal interpretation.

Transparent reporting should make clear that individual outcome counts and independent treatment assignment counts are different quantities.

Evaluating a Classroom-Level AI Tutoring Intervention

Suppose intervention classrooms have an average achievement score of 81, while comparison classrooms average 76. The observed unadjusted difference is five points.

That difference alone does not establish statistical significance or the precision of the estimated treatment effect.

The uncertainty depends on between-classroom variation, within-classroom correlation, allocation procedures, and the selected model.

If a naive student-level analysis reports p = 0.01 while ignoring classroom assignment, that hypothetical result should not be treated as definitive evidence of intervention effectiveness.

A cluster-aware reanalysis could produce different uncertainty estimates. The direction and magnitude of those changes cannot be determined from the group means alone.

The study may still provide useful causal evidence because classrooms were randomized, but the credibility of its conclusions depends on appropriate implementation, recruitment, outcome ascertainment, and analysis.

Common Mistakes With Group-Level Treatment and Individual Outcomes

How to Design and Analyze Group-Assigned Interventions

Identify the intervention assignment level before calculating sample size or selecting an analytical method.

Then define the outcome level, intended treatment effect, and relevant dependence structure.

The correct approach preserves individual outcome information without confusing the number of measurements with the number of independently assigned treatment units.

What to Verify Before Analyzing Group-Level Treatments

Frequently Asked Questions About Group-Level Treatment Assignment

Individual Outcomes Are Valuable, but Independent Treatment Assignment Still Matters

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.