How Broad Should Eligibility Criteria Be in a Pragmatic Trial?

Should a Pragmatic Trial Include Nearly Everyone Who Would Receive the Intervention in Ordinary Practice?

You are planning a pragmatic randomized trial of an AI-assisted learning program across several universities. The program is intended for students with different academic backgrounds, levels of digital literacy, learning needs, and prior experience with artificial intelligence.

One research team member proposes restricting participation to students with similar academic preparation, reliable internet access, and strong attendance records. The argument is that a more homogeneous sample would make the intervention easier to evaluate.

Another argues that these restrictions would exclude many students who would ordinarily use the program, making the findings less useful for institutional adoption decisions.

Should the trial include everyone? Are narrow eligibility criteria inconsistent with pragmatism? And how can researchers distinguish justified exclusions from restrictions introduced mainly for methodological convenience?

The appropriate answer begins with the population and decision the trial is intended to inform.

Eligibility Should Reflect the Intended Treatment Population, With Justified Restrictions

Designing Eligibility Criteria That Support Pragmatic Research

What Are Eligibility Criteria?

Eligibility criteria specify who may participate in a research study. They usually include inclusion criteria identifying the population of interest and exclusion criteria identifying circumstances under which participation is inappropriate or outside the study's scope.

For example, a university intervention trial might include students enrolled in a particular course and exclude individuals who cannot receive the intervention under the intended institutional arrangements.

Eligibility criteria affect the population represented by the treatment comparison and the circumstances under which the findings may apply.

They should therefore be determined by the research question and treatment strategy, not merely by a desire to reduce variation among participants.

Why Does Eligibility Matter in a Pragmatic Trial?

Pragmatic trials are intended to inform decisions under conditions relevant to ordinary practice.

If eligibility criteria exclude substantial portions of the population that would ordinarily receive an intervention, the study may provide limited evidence for the intended decision.

For example, a digital learning program intended for all first-year university students may be evaluated only among students with high prior achievement and reliable personal computers.

The trial might estimate an effect credibly for that selected group, but its applicability to students with different academic preparation or technological access would remain uncertain.

Researchers should consider whether each restriction is consistent with the population and implementation strategy the study is meant to represent.

How Do Pragmatic and Explanatory Trials Differ in Eligibility?

Explanatory trials may use narrower eligibility criteria to focus on participants meeting selected conditions or to clarify efficacy within a particular population.

Pragmatic trials often use criteria closer to those applied during ordinary treatment or service delivery.

However, the distinction is not absolute.

An explanatory trial may include a broad population, and a pragmatic trial may require restrictions when the intervention is appropriate only for a defined group.

The distinction between explanatory and pragmatic trials concerns the overall orientation of the design rather than one eligibility rule.

Does Pragmatic Eligibility Mean Including Everyone?

No. A pragmatic trial should not automatically include individuals for whom the intervention is inappropriate, unsafe, unavailable, or unrelated to the decision being studied.

For example, a program designed for students enrolled in research methods courses need not include students who are not taking such courses.

Likewise, a clinical intervention may have legitimate contraindications that should remain exclusion criteria.

The objective is not unrestricted participation. It is alignment between the eligible study population and the intended recipients of the intervention.

Researchers should distinguish justified boundaries of the target population from exclusions introduced mainly to simplify recruitment, improve adherence, or reduce outcome variability.

What Is the Difference Between the Target Population and the Eligible Population?

The target population is the population to which researchers intend the findings to apply.

The eligible population consists of individuals who meet the study's inclusion and exclusion criteria.

The recruited population is the subset of eligible individuals who actually enter the study.

These populations may differ substantially.

For example, broad eligibility does not guarantee that students with limited internet access will enroll if recruitment takes place entirely through an online portal.

Researchers should therefore evaluate eligibility and recruitment separately.

How Does PRECIS-2 Address Eligibility?

PRECIS-2 includes eligibility as one of nine domains used to assess how trial design choices align with explanatory or pragmatic objectives.

The eligibility domain considers the extent to which participants resemble those who would receive the intervention in usual care.

A more pragmatic eligibility approach generally avoids restrictions that would not apply during ordinary implementation.

However, PRECIS-2 is a design aid rather than a universal trial quality score.

A high pragmatic score is not inherently desirable when it conflicts with the intended research question or legitimate treatment restrictions.

Which Exclusion Criteria May Be Justified?

Exclusions may be appropriate when participation creates unacceptable risk, the intervention is not intended for the individual, or the study cannot ethically or meaningfully evaluate the treatment strategy for that person.

For example, a clinical intervention may have established contraindications. An educational intervention may be designed specifically for a particular course or stage of study.

However, researchers should distinguish genuine intervention requirements from limitations that could be addressed through reasonable accommodations or ordinary support.

For instance, excluding students with disabilities merely because the original digital materials are inaccessible may conceal a problem in intervention design rather than establish an appropriate eligibility boundary.

Eligibility decisions should be justified in relation to safety, ethics, feasibility, and the intended target population.

Which Exclusions Commonly Deserve Reconsideration?

None of these exclusions is universally inappropriate. Their justification depends on the intervention, target population, and intended decision.

Should Researchers Exclude Participants Likely to Have Low Adherence?

Not automatically. Restricting eligibility to participants expected to comply closely may produce an adherence pattern unlike ordinary implementation.

For example, requiring students to demonstrate high engagement during a preliminary screening period could exclude those who would ordinarily be offered the program.

If the study concerns routine effectiveness, such selection may limit applicability.

However, some interventions require minimum participation capabilities or commitments for safety or meaningful delivery.

Researchers should distinguish legitimate treatment requirements from efforts to select participants most likely to produce favorable results.

Should Participants With Additional Support Needs Be Included?

When they belong to the intended treatment population, researchers should consider how they can participate appropriately.

For example, students may require accessible materials, assistive technologies, flexible participation arrangements, or other reasonable accommodations.

Excluding these participants may narrow the evidence available for institutional adoption decisions.

However, inclusion should be supported by an intervention and assessment design that is appropriate for their needs.

Researchers should not assume that identical delivery is necessary for a valid treatment comparison.

Adaptation may be part of the intended strategy, provided essential functions and relevant measurement requirements are preserved.

Can Broad Eligibility Increase Outcome Heterogeneity?

Yes. Broader eligibility may include participants with different baseline characteristics, treatment needs, and responses.

This variation may increase outcome variability in some settings, potentially affecting statistical precision.

However, broader populations do not necessarily produce less precise estimates in every study.

Variation in treatment effects and baseline risk may also provide useful information about the intervention's performance across the intended population.

Researchers should address relevant variability through appropriate design and analysis rather than automatically exclude participants who differ from an idealized profile.

Does Broad Eligibility Reduce Internal Validity?

Not inherently. Internal validity depends on whether the study provides a credible estimate of the intended treatment effect.

Randomization can support comparability between treatment groups even when participants vary substantially in baseline characteristics.

However, broad eligibility may introduce practical challenges involving recruitment, intervention delivery, outcome measurement, and follow-up.

These challenges require appropriate safeguards.

Researchers should not confuse participant heterogeneity with confounding in a properly randomized trial.

How Can Eligibility Affect Treatment-Effect Heterogeneity?

Participants may respond differently to an intervention according to prior knowledge, baseline risk, access to resources, or other characteristics.

Broad eligibility can make such variation relevant to the study's average treatment effect.

However, detecting and interpreting subgroup effects requires adequate statistical information and appropriate analytical planning.

Researchers should prespecify important effect modifiers where possible and avoid treating exploratory subgroup findings as definitive.

A broad sample does not automatically provide enough participants within every subgroup for reliable inference.

Should Eligibility Criteria Match Routine Service Rules Exactly?

They should generally reflect the intended treatment population, but exact correspondence is not always necessary.

Research may require additional restrictions related to informed consent, safety, or the feasibility of collecting essential outcome information.

However, researchers should examine whether such restrictions meaningfully change the population being studied.

For example, requiring participants to have personal smartphones may be unnecessary if the institution ordinarily provides access to shared devices.

Eligibility criteria should reflect the actual implementation strategy rather than assumptions about how services are delivered.

How Do Eligibility and Recruitment Differ?

Eligibility determines who may enter the trial. Recruitment determines how eligible individuals are approached and enrolled.

A trial may have broad eligibility but recruit primarily from highly motivated volunteers.

For example, a university-wide intervention study may allow all eligible students to participate but advertise only through a research-interest mailing list.

The resulting sample may differ from the broader student population.

Researchers should consider recruitment pathways, participation rates, and differences between enrolled and nonenrolled individuals where information is available.

Pragmatic eligibility does not compensate automatically for selective recruitment.

Can Consent Procedures Affect Pragmatic Eligibility?

Yes. Consent and recruitment procedures may influence who participates.

For example, lengthy research appointments or complex enrollment requirements may disproportionately discourage participants with limited time or resources.

However, ethical requirements should not be weakened merely to make a study more pragmatic.

Researchers should use consent procedures appropriate to the intervention, study design, participant population, and applicable ethical requirements.

Where alternative consent arrangements are legally and ethically permissible, they require appropriate review and justification.

How Should Researchers Handle Multiple Sites?

Multi-site pragmatic trials may involve institutions serving different populations.

Researchers should establish common eligibility principles while determining whether site-specific differences are justified.

For example, universities may have different course structures but share a common definition of students eligible for a research writing intervention.

Unnecessary site-specific restrictions can complicate interpretation of the pooled study population.

Researchers should document important differences and consider whether the participating sites adequately represent the intended implementation settings.

How Should Eligibility Affect Sample Size Planning?

Eligibility influences the expected number of participants, outcome variability, and potentially the distribution of treatment effects.

Researchers should use plausible estimates of recruitment, baseline outcome variation, and follow-up completion when planning sample size.

For cluster-randomized trials, the number and size of clusters and intracluster correlation also matter.

Broadening eligibility may increase recruitment opportunities, but it does not guarantee adequate statistical power.

Sample size assumptions should correspond to the population and implementation conditions actually intended.

How Can Researchers Assess Representativeness?

Researchers should compare the enrolled sample with the intended target population using relevant characteristics where appropriate data are available.

For example, an educational trial may compare academic level, program, prior achievement, or access to learning resources.

However, representativeness is multidimensional and depends on which characteristics influence treatment effects or implementation.

A sample need not match every descriptive characteristic of the target population to support useful inference.

Researchers should focus on differences relevant to applicability and acknowledge limitations in available comparison data.

How Should Eligibility Criteria Be Reported?

Reports should specify all important inclusion and exclusion criteria and explain restrictions that may affect applicability.

Researchers should describe recruitment settings and participant characteristics rather than rely on the label pragmatic.

Where feasible, reporting should identify how many individuals were assessed for eligibility, excluded, enrolled, randomized, and followed up.

The CONSORT reporting framework provides guidance for transparent reporting of randomized trials.

Researchers should explain how the enrolled population relates to the population for whom the intervention decision is intended.

Setting Eligibility Criteria for a University-Wide AI Learning Trial

The resulting eligibility criteria may better represent the population for whom universities are considering adoption.

However, broad criteria do not guarantee that every subgroup will enroll or that the findings will apply to institutions with different resources.

The study must still address recruitment, intervention delivery, outcome measurement, and contextual differences.

Common Misconceptions About Pragmatic Trial Eligibility

Define Eligibility According to Who Would Actually Receive the Intervention

Begin with the target population and intended implementation strategy. Then evaluate every inclusion and exclusion criterion against that definition.

Retain restrictions that are necessary and justified, but reconsider those introduced mainly to make the study population easier to manage.

Researchers should also consider how eligibility contributes to the overall pragmatic–explanatory design profile rather than treating it as the sole determinant of trial orientation.

The most defensible eligibility criteria identify the population relevant to the research decision without introducing unnecessary exclusions.

Before Finalizing Eligibility Criteria for a Pragmatic Trial

Frequently Asked Questions About Pragmatic Trial Eligibility

Pragmatic Eligibility Should Reflect the Intended Recipients, Not an Idealized Participant

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.