03 · What You Need to Know
How to Recognize and Prevent Artificial Sample Size Inflation
What Is Sample Size Inflation From Nonindependent Observations?
Sample size inflation occurs when researchers count related measurements or subsamples as though they represented more independent entities than were actually observed or assigned.
For example, a study involving 50 patients who each contribute four lesions contains 200 lesion observations but only 50 patients.
Reporting 200 lesions is correct. Reporting 200 independent patients would be incorrect.
The distinction becomes especially important when the exposure or intervention is assigned at the patient level, because the 200 lesions do not represent 200 independent treatment assignments.
More generally, the relevant information count depends on the unit of analysis, the level of independent replication, and the intended estimand.
Why Is the Number of Dataset Rows Not Necessarily the Sample Size?
A dataset row is a storage convention. It may represent an individual, a measurement occasion, an event, a lesion, a classroom, or another recorded unit.
In long-format longitudinal data, one participant may occupy many rows. In hospital data, one patient may have several admissions. In educational data, one student may complete multiple assessments.
These rows can contain distinct information, but they do not necessarily represent independent participants.
Researchers should report the number of distinct entities and observations separately.
For example, a study might describe 120 participants contributing 480 repeated assessments, rather than reporting an unexplained sample size of 480.
This distinction is fundamental to understanding the difference between observation and analysis units.
Where Does Artificial Sample Size Inflation Commonly Occur?
| Research situation |
Recorded observations |
Important independent or grouping unit |
| Repeated student assessments |
Several scores per student |
Student |
| Students within classrooms |
Individual student outcomes |
Classroom for group-level assignment; student for individual outcomes |
| Both eyes measured |
Two eye measurements per patient |
Patient as a clustering unit |
| Multiple teeth or lesions |
Several anatomical observations per patient |
Patient, with possible additional anatomical levels |
| Repeated hospital admissions |
Several events per patient |
Patient and recurrent-event process |
| Several employees per organization |
Individual employee responses |
Organization for organization-level questions |
| Technical replicates |
Repeated laboratory readings of the same specimen |
Specimen or independently sampled biological unit |
These examples do not imply that only one observation per grouping unit may be analyzed. They indicate that the relevant dependence and replication levels must be represented correctly.
What Is the Difference Between Biological and Technical Replicates?
Biological replicates are independently sampled biological units relevant to the scientific question, such as independently recruited participants or independently prepared experimental specimens.
Technical replicates are repeated measurements of the same underlying material or experimental unit.
For example, measuring the same blood specimen three times may help assess measurement precision, but it does not create three independent patients.
Similarly, administering the same questionnaire repeatedly to one participant does not create additional independently recruited individuals.
The distinction matters because technical replication can reduce measurement error without increasing biological replication at the relevant level.
Whether observations qualify as independent biological replicates depends on how units were sampled, prepared, or assigned, not merely on whether measurements were conducted separately.
What Is Pseudoreplication?
Pseudoreplication occurs when observations that are not independent replicates at the relevant level are treated as though they provide independent experimental replication.
Suppose one classroom receives a new instructional program and another receives conventional instruction. Researchers assess 40 students in each classroom.
The study contains 80 student outcomes but only one classroom assigned to each condition.
Because classroom and intervention are completely confounded, the observed difference cannot be separated from classroom-specific influences using that assignment structure alone.
Analyzing the 80 students as independently assigned treatment units does not resolve the problem.
This is an example of pseudoreplication, a concept discussed extensively in experimental design.
Correctly counting experimental units is therefore as important as correctly modeling correlated outcomes.
How Does Intracluster Correlation Reduce Independent Information?
Observations within a cluster may share characteristics that produce statistical correlation.
The intracluster correlation coefficient, or ICC, summarizes within-cluster similarity under a specified model.
When ICC is positive, additional observations within the same cluster may contribute less new information about some population quantities than observations from independent clusters.
For example, students within one classroom may share instructional experiences that make their outcomes more similar than those of students in different classrooms.
However, the effect of ICC depends on the estimand. Precision for a classroom-level treatment effect differs from precision for a within-classroom student-level association.
Researchers should not interpret ICC as a universal percentage by which every statistical estimate loses information.
What Is Effective Sample Size?
Effective sample size is a way of expressing the information or precision of a particular estimator relative to a specified reference design.
It is not necessarily the number of distinct participants, clusters, or measurements.
For simple clustered estimates, a common approximation divides the nominal number of observations by a design effect.
The effective sample size depends on the quantity being estimated and the assumptions used in the calculation.
It should not replace reporting the actual number of participants or clusters.
For complex longitudinal, multilevel, or treatment-effect analyses, a single effective sample size may be insufficient to describe the available information.
How Do You Calculate a Simple Cluster Design Effect?
For equal-size clusters and an exchangeable correlation structure, a commonly used approximation is:
This formula does not automatically apply to unequal cluster sizes, complex sampling, within-cluster contrasts, or repeated-measures treatment effects.
For more complex designs, the sample size calculation should correspond directly to the proposed statistical model and estimand.
What Happens When Cluster Sizes Are Unequal?
Unequal cluster sizes can change statistical precision and the relative contribution of groups.
For example, a study may recruit 200 students from one school and 20 students from another. Treating both schools identically or weighting observations solely by student count can target different population quantities.
Cluster-size variation may also affect the design effect and the performance of statistical estimators.
Researchers should examine the distribution of cluster sizes rather than relying only on their average.
When cluster size is associated with the outcome, informative cluster size may require additional methodological consideration.
The appropriate approach depends on whether the research targets an average individual, an average cluster, or another estimand.
How Do You Prevent Inflation in Repeated-Measures Studies?
Repeated-measures studies should report distinct participants and total measurement occasions separately.
For example, 80 participants measured at four time points produce 320 observations, not 320 independent participants.
Appropriate analyses may include paired methods, repeated-measures ANOVA, mixed-effects models, or generalized estimating equations.
The choice depends on the research question, outcome distribution, measurement schedule, and dependence structure.
Researchers should distinguish repeated measurements from independent observations and avoid interpreting every additional measurement as an independent replicate.
Additional measurements can improve precision, especially for estimating individual change, but their value depends on the covariance structure.
How Do You Prevent Inflation With Multiple Organs or Lesions?
Studies involving multiple eyes, teeth, organs, or lesions should preserve participant identifiers and distinguish patient counts from anatomical observation counts.
For example, 300 lesions from 100 patients may provide 300 lesion-level observations but only 100 distinct patients.
If lesions are correlated within patients, suitable methods may include GEE, mixed-effects models, or appropriate patient-level summaries.
The analytical method should reflect whether the estimand concerns an average lesion or an average patient.
Researchers should also evaluate whether participants contributing more lesions are systematically different from those contributing fewer.
The methodological issues involved in multiple anatomical observations from one participant extend beyond simple sample size counting.
What If the Study Has Multiple Levels of Clustering?
Some studies involve several nested or crossed dependence structures.
For example, students may contribute repeated measurements while also being nested within classrooms and schools.
In this situation, observations may share participant-level, classroom-level, and school-level influences.
A model accounting only for repeated measurements within students may not fully address dependence associated with shared classrooms or schools.
Depending on the research question, multilevel models, multiway cluster-robust inference, or other approaches may be appropriate.
Researchers should map the actual grouping structure before choosing a model.
Simply counting the total number of observations cannot reveal how much independent information exists at each level.
Does Correcting Standard Errors Solve Sample Size Inflation?
Not always. Correcting standard errors can address certain inferential problems arising from dependence, but it does not automatically resolve an incorrect estimand or inadequate experimental replication.
For example, cluster-robust standard errors may be appropriate when many independent clusters are available and the model otherwise matches the research question.
However, a study with only one treated classroom and one comparison classroom cannot obtain independent classroom-level treatment replication by adjusting standard errors.
Likewise, a model cannot recover individual-level relationships that were lost when only aggregate data were retained.
The appropriate remedy depends on whether the problem concerns uncertainty estimation, analytical level, sampling, or treatment assignment.
Should You Aggregate Observations to One Record per Participant?
Participant-level aggregation may be appropriate when the research question concerns a meaningful participant-level summary.
For example, researchers may calculate each student's average engagement score across a semester.
However, aggregation can discard information about trajectories, temporal changes, or variation between anatomical units.
It may also change the estimand.
For instance, analyzing average lesion response per patient answers a different question from analyzing the response of an average lesion.
Researchers should therefore select aggregation only when the resulting summary corresponds to the intended scientific question.
How Should Sample Size Be Planned Before Data Collection?
Sample size planning should identify the relevant independent units, expected cluster sizes, correlation structure, outcome variability, and intended effect.
For cluster-randomized studies, the number of randomized clusters and participants per cluster should be considered jointly.
For repeated-measures studies, planning should account for the number and timing of measurements, within-person correlation, and expected missingness.
For studies with several organs or lesions per patient, calculations should reflect patient-level clustering and variation in anatomical observation counts.
Simulation-based calculations may be useful when the design is too complex for simple approximations.
The distinction between participants and clusters in sample size planning is particularly important when treatment assignment or inference operates at the group level.
How Should Sample Size Be Reported?
Reports should distinguish the number of recruited entities, independent assignment units, groups, and total observations.
For example, a clustered educational study might report 600 students from 20 classrooms across five schools.
A repeated-measures study might report 120 participants contributing 460 observed assessments out of 480 scheduled assessments.
An ophthalmological study might report 150 patients contributing 290 eligible eyes.
These descriptions allow readers to understand the dataset's structure without confusing measurement counts with independent replication.
Methods sections should also explain how dependence was handled and whether the sample size calculation reflected the actual design.
What If the Sample Size Was Already Reported Incorrectly?
Researchers should first determine whether the problem is merely inaccurate reporting or whether the statistical analysis also assumes inappropriate independence.
If the analysis correctly accounts for dependence but the manuscript reports only the number of observations, the description can be revised to distinguish participants and measurements.
If the analysis incorrectly treats related observations as independent, reanalysis may be necessary.
Where identifiers and sufficient independent units are available, appropriate clustered or longitudinal methods may address the problem.
When independent treatment replication is inadequate, additional data or narrower conclusions may be required.
Researchers should evaluate the consequences of an incorrectly identified analytical unit rather than assuming that changing the reported sample size alone resolves the issue.
