03 · What You Need to Know
The Scientific and Practical Factors That Should Shape Inclusion Criteria
Why Should the Research Question Come First?
The research question establishes what the study seeks to estimate, explain, compare, or understand. Inclusion criteria should identify the entities whose information is necessary for that purpose.
For example, a survey estimating the prevalence of generative AI use among undergraduate students must include eligible students who use AI and those who do not.
Requiring previous AI use would make the intended prevalence estimate impossible because nonusers would be excluded from the denominator.
Conversely, a qualitative study examining experiences of AI-assisted manuscript writing may reasonably require firsthand involvement in that activity.
The topic is similar, but the eligibility requirements differ because the questions require different evidence.
Researchers should therefore begin by identifying the intended claim, not by drafting a list of preferred participant characteristics.
How Does the Target Population Determine Inclusion Criteria?
The target population specifies the individuals or other entities about which the researcher intends to make claims.
Inclusion criteria operationalize the characteristics necessary for membership in the eligible study population.
For example, if the target population comprises undergraduate students enrolled during a particular academic year, current undergraduate enrollment and the relevant reference period may be necessary criteria.
However, a requirement to own a personal laptop would not automatically follow from that definition.
It could exclude students whose access circumstances are important to the research.
Researchers should establish the intended target population before introducing operational restrictions.
How Does the Intended Estimand Affect Eligibility?
An estimand is the particular quantity a study seeks to estimate.
For descriptive research, it might be a population proportion or mean. For causal research, it might be an intervention effect in a specified population.
The estimand determines whose observations or potential outcomes are relevant.
For example, an intervention study estimating the effect of offering AI tutoring to first-year programming students should generally define eligibility before treatment assignment.
Requiring participants to complete the entire intervention before being considered eligible would change the population and potentially introduce selection bias.
Eligibility should therefore correspond to the intended effect of assignment, exposure, or another explicitly defined contrast.
Which Factors Commonly Determine Inclusion Criteria?
| Factor |
Question to ask |
Illustrative implication |
| Research question |
Whose information is necessary? |
Include users and nonusers for an adoption prevalence study. |
| Target population |
What characteristics define membership? |
Require enrollment in the intended educational population. |
| Study design |
Which units and comparisons are required? |
Establish eligibility before randomization where appropriate. |
| Intervention applicability |
For whom is the intervention meaningful? |
Require enrollment in a course where the intervention can be delivered. |
| Measurement validity |
Can the intended construct be assessed appropriately? |
Use comparable outcome tasks or validated adaptations. |
| Clinical or safety requirements |
Who can appropriately receive the intervention? |
Use justified diagnostic or treatment-indication criteria. |
| Time and setting |
When and where must membership apply? |
Specify a semester, diagnosis interval, or relevant jurisdiction. |
| Ethics and equity |
Would a restriction unnecessarily exclude relevant groups? |
Provide accessible participation arrangements where feasible. |
| Recruitment feasibility |
Can eligibility be assessed and participants recruited? |
Plan suitable verification without redefining the question for convenience. |
These factors interact. A criterion may be scientifically relevant but difficult to verify, or operationally convenient but unnecessary for the intended inference.
How Should the Study Design Influence Inclusion Criteria?
Different designs require different kinds of participants and observations.
A cross-sectional prevalence study needs an eligible population that includes the relevant range of outcomes.
A longitudinal cohort requires clear entry criteria and a defensible baseline or time zero.
A randomized trial requires participants for whom the assigned interventions are appropriate and whose eligibility can be established consistently.
A qualitative study may prioritize participants with particular experiences or roles that illuminate the phenomenon.
Researchers should not transfer eligibility criteria from one design to another without considering the different inferential objectives.
When Should Intervention Requirements Determine Eligibility?
Intervention applicability can legitimately determine inclusion criteria.
For example, an AI-assisted programming intervention may require participants to be enrolled in a course involving programming tasks.
Similarly, a clinical intervention may be appropriate only for patients with a specified diagnosis or treatment indication.
However, researchers should distinguish necessary intervention requirements from conditions that merely simplify delivery.
If a digital intervention can be provided using institutional devices, requiring participants to own personal devices may be unnecessary.
Eligibility should reflect what is essential for meaningful intervention delivery rather than the easiest possible implementation arrangement.
Should Diagnosis, Disease Stage, or Severity Determine Inclusion?
These characteristics may be necessary when the research concerns a defined clinical condition, treatment indication, or disease subgroup.
For example, an intervention intended for a specific disease stage may require that stage to be established using an appropriate classification system.
However, a study examining outcomes across the full disease spectrum should not restrict participation to one severity category without justification.
Researchers should distinguish stage and severity, specify relevant assessment methods, and identify when classification occurs.
The decision should follow the role of disease stage, severity, or experience in defining the population.
When Should Prior Experience Be Required?
Experience should determine eligibility when firsthand involvement is necessary to answer the question.
For example, researchers investigating the experiences of teachers implementing AI assessment policies may require participants who have actually participated in implementation.
However, a study examining factors associated with policy adoption may need both adopters and nonadopters.
Experience should be defined through relevant activities rather than vague descriptions such as knowledgeable, experienced, or expert.
Duration, frequency, and competence are different constructs and should not automatically be treated as interchangeable.
How Should Age and Demographic Characteristics Be Considered?
Age or other demographic characteristics may be relevant when they define the population, influence intervention suitability, or establish ethical and regulatory requirements.
For example, a study specifically concerning adolescent learners may appropriately use an age-based population definition.
However, a general educational technology study should not automatically exclude older students merely to obtain a more homogeneous sample.
Researchers should distinguish scientifically necessary eligibility conditions from demographic characteristics that can be measured as covariates or examined as effect modifiers.
Unnecessary demographic restrictions may reduce population coverage and exclude groups relevant to the intended application.
Should Language Proficiency Be an Inclusion Criterion?
Language proficiency may be relevant when the research question concerns language-specific competence or when valid participation requires understanding particular materials.
For example, an intervention evaluating English academic writing may legitimately require participants to engage in English-language writing tasks.
However, requiring English proficiency in an unrelated survey may unnecessarily exclude eligible participants.
Researchers should consider translated materials, appropriate interpretation, accessible communication, and measurement equivalence where feasible.
Language restrictions should be justified by the construct or research procedures rather than introduced solely for administrative convenience.
How Should Measurement Requirements Affect Eligibility?
Eligibility may require participants to have characteristics that make the intended outcome meaningful and measurable.
For example, an assessment of academic essay writing requires participants for whom the writing task is relevant.
However, difficulty measuring an outcome in a subgroup does not automatically justify excluding that subgroup.
Researchers should first consider whether instruments can be adapted or alternative measurement procedures used without compromising validity.
The decision should distinguish genuine construct incompatibility from limitations of the available instrument.
Should Safety and Ethical Considerations Determine Inclusion?
Yes, when participation or intervention exposure raises relevant safety or ethical concerns.
Clinical research may require eligibility conditions based on treatment indications, contraindications, or the ability to undergo specified procedures safely.
However, researchers should not assume that every vulnerable population must be excluded.
Ethical inclusion may require additional safeguards, consent procedures, accessible materials, or appropriate monitoring rather than automatic exclusion.
The Belmont Report emphasizes respect for persons, beneficence, and justice, while applicable institutional and national requirements govern specific studies.
Eligibility decisions should balance participant protection with fair access to research participation and its potential benefits.
Is Capacity to Consent Always an Inclusion Requirement?
Not universally. Consent arrangements depend on the population, study procedures, applicable regulations, and ethics approval.
Some research involving children or adults with impaired decision-making capacity may proceed through legally authorized permission, assent, or other approved safeguards.
Researchers should not automatically exclude such populations merely because standard adult consent procedures are unsuitable.
At the same time, participation must comply with applicable ethical and legal requirements.
The eligibility criterion should reflect the approved consent framework rather than an unsupported assumption that only independently consenting adults can participate in research.
Should Recruitment Feasibility Determine Inclusion Criteria?
Feasibility matters, but it should not silently determine the scientific population.
For example, recruiting only students who can complete an online questionnaire may simplify data collection while excluding those with limited connectivity.
If the research concerns digital inequality, this restriction could undermine the question itself.
Researchers should consider alternative recruitment arrangements before narrowing eligibility.
When restrictions are unavoidable, distinguish the target population from the accessible population and qualify conclusions accordingly.
Can Inclusion Criteria Be Too Restrictive?
Yes. Accumulating individually plausible restrictions can produce an eligible population much narrower than intended.
For example, requiring students to be full-time, enrolled in one course, experienced with AI, fluent in English, and owners of personal laptops may exclude substantial portions of a broader undergraduate population.
Some restrictions may be justified, but their combined consequences should be examined.
Researchers should assess whether each criterion remains necessary and whether important subgroups are disproportionately excluded.
A narrowly defined population may improve focus without necessarily improving internal validity.
Can Inclusion Criteria Be Too Broad?
Yes. Criteria that fail to establish meaningful population membership can include participants for whom the research question or intervention is inappropriate.
For example, an intervention developed for introductory programming students may not be meaningfully evaluated among participants who are not studying programming.
However, heterogeneity alone is not a reason to restrict eligibility.
Broader populations may be appropriate when the study seeks to examine variation across relevant groups.
The question is whether the resulting population supports a coherent estimand and valid measurement.
How Do Inclusion Criteria Affect Statistical Power?
Eligibility restrictions can influence the number of available participants, outcome variability, expected event rates, and treatment-effect assumptions.
A more homogeneous population may sometimes reduce outcome variation, but restriction can also make recruitment substantially harder.
For example, a study limited to a rare clinical subgroup may require recruitment across multiple hospitals.
Researchers should base sample size calculations on realistic assumptions for the eligible population.
Criteria should not be narrowed merely to obtain a favorable power estimate.
How Do Inclusion Criteria Affect Generalizability?
Eligibility rules determine which population members can contribute evidence.
Restricting eligibility may limit the direct applicability of findings to excluded groups, especially when relevant outcomes or intervention effects differ across those groups.
However, broader eligibility does not automatically establish representativeness.
Sampling, recruitment, participation, and analytical procedures also influence population inference.
Researchers should examine how eligibility criteria affect generalizability when defining the study population.
Should Every Important Characteristic Become an Inclusion Criterion?
No. Some characteristics should define eligibility, while others should be measured as exposures, outcomes, confounders, moderators, or descriptive variables.
For example, prior academic achievement may be relevant to an AI tutoring intervention but need not determine eligibility.
Measuring baseline achievement may allow researchers to examine whether intervention effects vary across achievement levels.
Restricting eligibility to high-achieving students would remove that variation and change the population of inference.
Researchers should decide whether a characteristic is essential for membership or analytically important within a broader population.
How Should Eligibility Criteria Be Operationalized?
Each criterion should specify a characteristic that can be assessed consistently.
For example, "currently enrolled undergraduate student" should be linked to an appropriate institutional enrollment definition and reference period.
Similarly, "previous experience with AI-assisted writing" should identify what activity qualifies and how experience will be verified.
Assessment may use self-report, administrative records, testing, or professional judgment depending on the characteristic.
Researchers should specify how uncertain or borderline cases will be handled and document eligibility decisions.
How Can Researchers Justify Each Inclusion Criterion?
A practical approach is to create an eligibility justification matrix before recruitment.
For every proposed criterion, document the characteristic, scientific or ethical rationale, assessment method, and consequences of excluding individuals who do not satisfy it.
Ask whether removing the criterion would make the research question unanswerable, the intervention inappropriate, or the measurement invalid.
If removing it would simply make recruitment less convenient, the restriction may require reconsideration.
Researchers should also evaluate the combined effects of all criteria rather than considering each in isolation.
When Should Inclusion Criteria Be Finalized?
Eligibility criteria should generally be specified during protocol development, before participant recruitment and outcome examination.
Changes may sometimes be justified by new scientific information, safety concerns, or operational difficulties.
However, modifications can change the study population and estimand.
Researchers should document the timing, rationale, and consequences of substantive revisions and follow applicable ethics, registration, or protocol amendment requirements.
Criteria should not be selectively revised after examining outcomes to obtain more favorable findings.
