03 · What You Need to Know
How to Choose Defensible Boundaries for Your Target Population
What Does Population Scope Mean?
Population scope refers to the breadth of entities and characteristics included in the population about which a study intends to make claims.
It may be determined by educational or professional status, age, clinical characteristics, institutional context, geography, time period, experience, or other substantively relevant dimensions.
For example, all university students represent a broader population than undergraduate nursing students enrolled in clinical placement courses.
However, population breadth is multidimensional. A study may be geographically broad but substantively narrow, such as a national study of doctoral students specializing in educational technology.
Researchers should therefore describe the actual population boundaries rather than classifying a population as simply broad or narrow.
Why Is the Broadest Possible Population Not Always the Best?
A broad population may include individuals or settings for whom the research question has different meanings.
For example, an intervention designed to improve undergraduate academic writing may not operate similarly among postgraduate researchers, primary school pupils, and professional writers.
Including all these groups under a general category of learners could produce an estimand that combines substantially different educational processes.
Such heterogeneity is not necessarily undesirable, but it requires a clear scientific rationale and an appropriate analytical strategy.
A population should not be expanded merely to make the findings appear more widely applicable.
Broad wording does not create broad evidence.
Why Is the Narrowest Convenient Population Not Always Appropriate?
Researchers sometimes define populations around the institutions, participants, or records they can access most easily.
For example, a study concerning undergraduate AI literacy may restrict its target population to students from one university because the researcher works there.
That university may provide a legitimate study setting, but institutional access alone does not establish that the scientific question concerns only its students.
The researcher should distinguish the intended population from the population reachable through available recruitment arrangements.
This distinction is explained in the relationship between target population, accessible population, and study sample.
Unnecessarily narrow definitions may exclude groups whose experiences are relevant to the phenomenon or its practical application.
What Should Determine the Breadth of the Target Population?
The principal consideration is the intended research claim.
A study estimating the prevalence of AI use among all undergraduate students requires a population definition that includes users and nonusers across the intended educational settings.
A study examining the experiences of doctoral students who use AI for systematic reviews may reasonably target a much narrower group.
The following considerations can help establish appropriate boundaries.
| Consideration |
Question to ask |
Implication for scope |
| Research objective |
What population quantity or relationship is being investigated? |
Include the entities necessary to answer that question. |
| Conceptual relevance |
Does the phenomenon have a meaningful interpretation across included groups? |
Restrict or differentiate groups when the construct changes substantially. |
| Expected heterogeneity |
Could important characteristics modify the outcome or relationship? |
Consider broader inclusion with appropriate subgroup or interaction analysis. |
| Intended application |
To whom will the findings or recommendations be applied? |
Align population boundaries with the intended beneficiaries or decision context. |
| Measurement validity |
Can the relevant constructs be measured comparably? |
Assess whether measurement adaptation or invariance evaluation is needed. |
| Sampling feasibility |
Can the intended population be adequately represented? |
Identify coverage limitations without automatically redefining scientific intent. |
These considerations should be evaluated together. A population can be scientifically appropriate yet difficult to sample, while an easily sampled population may be poorly aligned with the research objective.
How Does the Estimand Influence Population Scope?
An estimand is the specific quantity the researcher intends to estimate.
For descriptive research, this might be the prevalence of a characteristic or the population mean of an outcome.
For causal research, it may be the average effect of an intervention in a defined population.
The estimand determines whose outcomes or potential outcomes contribute to the target quantity.
For example, the average effect of an AI tutoring intervention among first-year university students is not necessarily the same estimand as its average effect among all university students.
Even when the intervention is identical, the target populations may differ in prior knowledge, course requirements, and opportunities to use the technology.
Researchers should define the estimand and population together rather than treating population scope as an unrelated administrative decision.
Does a Broader Population Automatically Improve Generalizability?
No. Generalizability depends on the relationship between the population, the study design, and the evidence collected.
A researcher may describe the target population as all university students, but recruiting respondents from one conveniently selected institution does not establish that the findings apply nationally.
Conversely, a study with a carefully defined narrower population may support strong conclusions about that population.
The appropriate question is not simply how broad the population sounds, but whether the sampling and analytical procedures support inference to the population as defined.
Generalizability should therefore be treated as an evidential property of the study rather than a benefit obtained through broad terminology.
Can a Narrow Population Improve Internal Validity?
A narrower population can sometimes reduce heterogeneity or simplify the conditions under which a research question is investigated.
For example, restricting an intervention evaluation to students enrolled in the same course may reduce variation in curriculum, assessment requirements, and instructional exposure.
However, restriction does not automatically eliminate confounding, measurement error, or selection bias.
Nor does a broader population necessarily undermine internal validity.
A randomized experiment can support credible causal inference across a heterogeneous population when the design is appropriate.
The relationship between population restriction, internal validity, and generalizability depends on the causal structure and research design.
Researchers should avoid assuming that greater homogeneity always produces better evidence.
When Is Population Heterogeneity Valuable?
Population heterogeneity can be scientifically valuable when researchers want to understand how outcomes or effects vary across contexts.
For example, an AI tutoring intervention may operate differently among students with high and low prior achievement, varying digital access, or different disciplinary backgrounds.
A study restricted to one highly homogeneous subgroup may be unable to examine these differences.
Broader inclusion may support effect-heterogeneity analysis, provided the sample contains sufficient information across relevant subgroups.
However, merely including diverse participants does not guarantee adequate precision for subgroup comparisons.
Researchers should identify important heterogeneity in advance and plan recruitment and analysis accordingly.
Should the Population Be Restricted to Reduce Confounding?
Restriction can sometimes reduce variation in a confounding variable.
For example, limiting a study to a particular educational level may reduce differences associated with curriculum stage.
However, restricting the target population solely to simplify statistical adjustment can change the research question and reduce applicability.
It may also introduce selection problems if eligibility depends on variables affected by the exposure or related to the outcome.
Researchers should compare restriction with alternatives such as measurement and adjustment of confounders, stratification, or appropriate research design.
Not every potential confounder should become an eligibility restriction.
Should Prior Exposure or Experience Define the Population?
Prior experience should be included as a population requirement only when it is substantively necessary.
For example, a qualitative study examining how researchers negotiate AI-assisted manuscript preparation may reasonably recruit individuals with direct experience of that practice.
However, a prevalence study of AI-assisted manuscript preparation must include eligible researchers who do not use AI.
Restricting the population to users would remove nonusers from the denominator and make the prevalence question impossible to answer as originally defined.
Researchers should therefore determine whether experience is part of the phenomenon being studied or a variable to be measured within a broader population.
How Should Geography Influence Population Scope?
Geographic boundaries may be appropriate when location is substantively relevant to the research question.
Educational policies, infrastructure, healthcare systems, and institutional regulations can differ across jurisdictions.
For example, a study evaluating implementation of a national education policy may reasonably target institutions operating within the relevant country.
However, a researcher should not automatically define the target population as one city simply because data collection is conducted there.
The geographic scope should follow the intended inference and relevant contextual mechanisms.
Further distinctions are important when considering whether geography should be part of the target population definition.
How Should Time Affect the Population Definition?
Population membership and relevant characteristics can change over time.
For example, students' experiences with generative AI may differ between academic years as institutional policies, available tools, and educational practices change.
A study may therefore define its population as students enrolled during a specified academic year.
However, the population reference period should not be confused with the dates on which questionnaires are distributed.
Researchers should distinguish the period defining population membership from the observation period and the temporal scope of the intended conclusions.
Time boundaries should be explicit when they materially affect eligibility or interpretation.
Should Institutional Type Be Part of the Population Definition?
Institutional type may be relevant when the research concerns policies, organizational characteristics, or experiences associated with particular institutional settings.
For example, a study of public university governance may appropriately restrict its population to public universities.
However, a study seeking to describe undergraduate AI use nationally should not exclude private institutions without a defensible reason.
Institutional restrictions may affect the composition of students, available resources, and implementation conditions.
Researchers should determine whether institutional type defines the phenomenon or represents an important characteristic to be measured across the population.
Can a Broad Population Be Divided Into Subgroups?
Yes. Researchers can define a broad target population while planning analyses for relevant subgroups.
For example, a national study of undergraduate AI literacy might examine variation by discipline, institutional type, year level, or digital access.
This approach can preserve the broader population objective while allowing more specific questions.
However, subgroup analyses require sufficient data, appropriate measurement, and attention to multiple comparisons or exploratory interpretation.
Subgroup definitions should be conceptually justified rather than selected solely because they produce statistically significant results.
Stratified sampling or deliberate oversampling may be useful when important subgroups are small.
What If the Population Is Too Heterogeneous for One Analysis?
Heterogeneity becomes problematic when the population combines entities for which the outcome, exposure, or intended relationship is not meaningfully comparable.
For example, the meaning of academic writing performance may differ substantially between first-year undergraduate essays and doctoral dissertation chapters.
Researchers should consider whether a common outcome measure is conceptually valid across these groups.
Possible responses include narrowing the population, defining separate estimands, using appropriate measurement models, or analyzing substantively justified subgroups.
However, heterogeneity should not be removed merely because it complicates analysis.
Sometimes variation across groups is itself an important research finding.
How Does Population Scope Affect Sample Size?
A broader population may require a more complex sampling design, particularly when important subgroups or institutional contexts must be represented.
However, the numerical size of the target population does not, by itself, determine the required sample size.
For estimating a proportion under simple random sampling assumptions, precision depends strongly on the anticipated proportion, confidence level, and desired margin of error.
Population size may affect the calculation through finite population correction when the sampling fraction is substantial.
For clustered, stratified, or model-based studies, additional design features influence the required sample.
Researchers should avoid assuming that a broader population always requires a proportionally larger sample or that a narrowly defined population automatically needs very few participants.
What If the Population Is Scientifically Appropriate but Difficult to Access?
A scientifically appropriate target population may be difficult to recruit because of geography, institutional permissions, privacy restrictions, or limited resources.
For example, a researcher may want to investigate digital learning experiences across public and private universities but have access only to selected institutions.
The researcher has several possible responses.
One is to expand recruitment through additional institutions or sampling channels. Another is to retain the broader target population while explicitly acknowledging the limited accessible population and qualifying generalization.
A third is to revise the research question so that the narrower population becomes substantively appropriate.
The choice should be made transparently and preferably before data collection.
Researchers should not silently redefine the population after recruitment merely to make the final sample appear representative.
Can Population Scope Change During Research?
Population definitions may occasionally require revision because of new evidence, feasibility constraints, or changes in the scientific objective.
However, such revisions can alter the estimand and the interpretation of findings.
For example, changing a target population from all undergraduate students to students who regularly use generative AI changes the denominator and potentially the research question.
Researchers should document substantive changes and explain their implications for eligibility, sampling, and analysis.
Changes made after examining outcomes require particular transparency because they may introduce concerns about selective interpretation.
How Can You Justify the Population Scope in a Research Proposal?
A defensible justification should connect the population boundaries to the phenomenon and intended inference.
For example, a study might explain that first-year undergraduate students were selected because the intervention specifically targets transition-to-university academic writing.
That is a substantive rationale.
By contrast, explaining that first-year students were selected only because their classes were easiest to access is a recruitment rationale rather than a scientific justification for the target population.
Recruitment feasibility can legitimately influence study design, but it should be distinguished from conceptual relevance.
Methods sections should explain who belongs to the population, why those boundaries matter, and whether the sampling strategy supports the intended claims.
