01 · The Question
Who Might Be Missing From Your Research Before Recruitment Even Begins?
A study may use a carefully constructed questionnaire, an appropriate sampling strategy, and a defensible analytical method while still overlooking people whose experiences are central to the research problem. The exclusion may not be deliberate. It can arise from decisions about where participants are recruited, which languages are supported, what technologies are required, or who can realistically complete the study procedures.
The question is not simply whether your sample includes people from different backgrounds. It is whether the research design creates predictable barriers that prevent particular population groups from being represented in the evidence you collect.
Recognizing these barriers matters because systematic exclusion can affect whose experiences inform the findings, which conclusions are defensible, and whether the results apply to the population the study intends to describe.
03 · What You Need to Know
How Research Design Decisions Create Systematic Population Exclusion
What Makes Population Exclusion Systematic?
Systematic exclusion occurs when characteristics of a study's design consistently prevent or discourage participation by identifiable groups. Unlike random nonparticipation, the barriers are associated with particular characteristics, circumstances, or resources.
For example, an online survey that requires reliable internet access may be less accessible to people with limited connectivity. A study requiring repeated daytime visits may be difficult for people whose employment or caregiving responsibilities prevent attendance. A recruitment campaign conducted exclusively in one language may fail to reach otherwise eligible individuals who use another language.
These patterns do not establish selection bias automatically. Their implications depend on the target population, the research question, the characteristics associated with participation, and the outcomes being investigated.
Nevertheless, predictable differences in participation deserve attention before data collection begins.
Distinguish the Target Population From the Population You Can Actually Reach
The target population is the group to which the research intends to apply. The accessible population consists of people who can realistically be reached under the study's recruitment arrangements. The sampling frame identifies the units from which a sample can be selected, while the achieved sample contains those who ultimately participate and contribute data.
These populations may differ substantially.
Target population
All people or units about whom the research seeks to draw conclusions.
Accessible population
Those who can realistically be contacted or recruited through the study's arrangements.
Sampling frame
The operational list or source from which sampling units are selected, where the sampling approach uses a frame.
Achieved sample
The participants who enroll and provide usable information.
Suppose a researcher wants to understand the experiences of all adult learners in a regional education system but recruits only through university learning management systems. Adults studying through community programs or outside institutional platforms may have no opportunity to participate.
The resulting sample may accurately represent certain platform users without adequately representing the broader population originally described.
Which Population Groups Are Commonly at Risk of Exclusion?
The groups at risk depend on the research context. Researchers should avoid assuming that all members of a demographic category face identical barriers. Instead, examine how participant characteristics interact with specific study requirements.
| Population group or circumstance |
Potential design barrier |
Possible consequence |
| Older adults |
Unjustified upper age limits, technology-dependent procedures, or inaccessible recruitment |
Underrepresentation of older people whose experiences may differ from younger participants |
| People with disabilities |
Inaccessible materials, venues, interfaces, or communication procedures |
Exclusion associated with accessibility needs rather than scientific eligibility |
| Pregnant people |
Blanket pregnancy exclusions without an assessment of study-specific risks |
Limited evidence for a population potentially affected by the intervention or condition |
| People with limited internet access |
Online-only recruitment, consent, or data collection |
Coverage gaps associated with connectivity, affordability, or digital access |
| People with limited financial resources |
Travel expenses, unpaid time commitments, or required equipment |
Participation may disproportionately favor those with greater resources |
| People using other languages |
Single-language recruitment, consent forms, or instruments |
Exclusion based on language proficiency rather than relevance to the research |
| Rural or geographically isolated populations |
Recruitment concentrated in urban institutions or distant research sites |
Geographic differences may be insufficiently represented |
| People with demanding work or caregiving schedules |
Fixed appointments or repeated lengthy participation sessions |
Lower participation among people with limited scheduling flexibility |
| People outside formal institutions |
Recruitment restricted to schools, clinics, employers, or registered organizations |
Individuals outside those institutional systems remain outside the recruitment frame |
These are possible mechanisms, not claims that every study necessarily excludes every listed group. The appropriate assessment is study-specific.
How Do Eligibility Criteria Exclude Otherwise Relevant Participants?
Eligibility criteria determine who can enter a study. They may protect participants, define a scientifically coherent population, or ensure that a procedure can be administered and interpreted appropriately.
Problems arise when restrictions are broader than necessary or reflect convenience rather than the research question. For instance, excluding all older adults from a study of a service used across adulthood may narrow its applicability without providing a clear methodological benefit.
Researchers should distinguish necessary eligibility restrictions from avoidable barriers and examine whether eligibility criteria disproportionately exclude relevant population groups.
When a restriction is justified, the resulting limits on inference should still be documented.
How Can Recruitment Channels Create Coverage Gaps?
Recruitment channels determine who has an opportunity to learn about the study. Advertising through a university mailing list, professional association, social media platform, or healthcare facility may reach some segments of the target population more readily than others.
This becomes particularly consequential when the channel is related to the phenomenon being studied.
For example, recruiting participants for a study of digital exclusion exclusively through social media may systematically overlook people who rarely use digital platforms. Recruitment is then conditioned on a characteristic closely related to the research question.
Using several channels may improve coverage, although multiple channels do not automatically create a representative sample. Researchers must still consider overlapping audiences, inaccessible channels, and differences in participation probabilities.
Can Data Collection Methods Exclude People Who Meet the Eligibility Criteria?
Yes. Formal eligibility does not guarantee practical access to participation.
An individual may meet every inclusion criterion but be unable to complete an online questionnaire, attend a distant interview, read a complex consent form, or communicate through the required format.
Particular attention should be given to internet-dependent research procedures, travel requirements, and language restrictions.
Providing alternative participation modes can reduce some barriers, but mode changes may also influence measurement. For example, responses to sensitive questions may differ between interviewer-administered and self-administered surveys. Accessibility and measurement comparability therefore need to be considered together.
Why Do Accessibility Requirements Matter Beyond Disability Status?
Accessible research procedures can benefit participants with different sensory, cognitive, communication, literacy, or technological needs. Accessibility may involve screen-reader-compatible documents, readable formatting, alternative communication methods, accessible venues, or appropriately adapted consent procedures.
Researchers should assess whether research materials and participation procedures accommodate different accessibility needs rather than assuming that a single format is adequate for everyone.
Accommodations should preserve informed consent and the integrity of the measurement process. Where adaptations change an instrument's administration or meaning, their implications for comparability should be evaluated.
How Can Intersecting Characteristics Intensify Exclusion?
Participation barriers may overlap. A rural participant with limited internet access, restricted transportation, and caregiving responsibilities may encounter several obstacles simultaneously.
Considering these characteristics separately may underestimate the practical difficulty of participation. An alternative online interview may solve a travel problem but remain inaccessible to someone without reliable connectivity.
Researchers can examine combinations of relevant characteristics during feasibility assessment, consultation, and recruitment monitoring. This does not require treating every possible demographic combination as a separate analytical subgroup. The objective is to identify consequential barriers supported by the research context.
How Does Systematic Exclusion Affect Scientific Validity?
The consequences depend on the study's inferential goals.
For descriptive research, excluding population segments whose outcomes differ from those of included participants can distort estimates intended to describe the broader population.
For studies estimating associations or causal effects, selection mechanisms may introduce bias when participation is related to variables relevant to the estimand. Even when an effect estimate is internally valid for the studied population, applying it to another population may require additional assumptions about effect modification and population differences.
For qualitative research, exclusion may restrict the range of experiences and perspectives available for interpretation. The concern is not statistical representativeness in every qualitative design, but whether recruitment adequately supports the study's conceptual and interpretive aims.
Consequently, inclusive research design and scientific validity should be considered in relation to the research question rather than as interchangeable measures of sample diversity.
Does Every Study Need to Include Every Population Group?
No. A study focused on a clearly defined population may legitimately exclude people outside that population. A trial may also require restrictions because of intervention risks, contraindications, or specific scientific objectives.
The key distinction is between a justified restriction and an avoidable exclusion that undermines the intended inference.
Researchers should not expand eligibility merely to appear inclusive when doing so would create disproportionate risks or make the research question incoherent. Equally, methodological convenience alone should not be treated as sufficient justification for excluding relevant groups.
Watch Out
A sample can be diverse in visible demographic characteristics while still systematically excluding people who face less visible barriers. Conversely, a narrowly defined sample is not inherently biased if the target population and conclusions are appropriately restricted. Assess the selection mechanism and intended inference, not diversity counts alone.
04 · A Practical Example
How an Online Learning Study Can Exclude the Learners It Intends to Understand
Hypothetical Example
Evaluating the Accessibility of Online Learning Among Adult Students
A research team wants to investigate barriers to online learning among adult students enrolled across several educational institutions. The study uses an online questionnaire distributed through institutional learning platforms. Participants must log in, open a survey link, and complete the questionnaire in English within one week.
The research team initially assumes that this approach is appropriate because the participants are enrolled in programs offering online learning.
Step 1: Define the intended population.
The researchers intend to describe adult students enrolled in the participating programs, including those who experience difficulties accessing online learning.
Step 2: Identify participation requirements.
Students need functioning login credentials, sufficient internet access, awareness of the invitation, adequate English comprehension, and the ability to complete the questionnaire within the specified period.
Step 3: Examine who may be missed.
Students with intermittent connectivity, inaccessible platform interfaces, language barriers, or limited time may be less likely to encounter or complete the survey. Some of these barriers may be directly related to the difficulties under investigation.
Step 4: Revise the recruitment and participation arrangements.
The team considers additional invitation channels, accessible survey formats, an extended response period, and alternative participation methods. It also evaluates whether translated or interviewer-administered versions can maintain acceptable measurement equivalence.
Step 5: Monitor and report remaining limitations.
The researchers compare available characteristics of respondents with the intended population, examine differential participation where possible, and describe groups that remain difficult to reach.
The revised design does not guarantee representativeness. However, it addresses foreseeable coverage problems and makes the relationship between the achieved sample and the intended population more transparent.
If students with the greatest access difficulties remain underrepresented, the researchers should avoid interpreting the findings as an unbiased estimate of barriers across all enrolled adult students without further supporting evidence.
06 · What This Means for You
How to Evaluate Exclusion Before Finalizing Your Research Design
Begin with the population named in your research question. Then examine each requirement that separates that population from the people who can actually enter and complete the study.
For each potential barrier, ask whether it is necessary, whether it disproportionately affects relevant groups, whether an alternative is feasible, and whether the resulting exclusion could change the conclusions.
A simple decision framework
If a group is outside the scientifically defined target population
Document the population boundary and restrict the conclusions accordingly.
If a group is excluded because of a genuine safety or methodological requirement
Explain the justification, consider narrower restrictions where feasible, and acknowledge limits on applicability.
If a relevant group faces an avoidable participation barrier
Modify recruitment, accessibility, scheduling, or data collection procedures where scientifically and ethically appropriate.
If inclusion is feasible but the group is likely to remain underrepresented
If a consequential exclusion cannot be resolved
Record the limitation, evaluate its likely direction and magnitude where possible, and avoid unsupported population-wide claims.
Where appropriate, consult potential participants or community representatives before finalizing procedures. Their feedback may identify barriers that are not apparent to the research team.
Importantly, inclusion decisions should be revisited during recruitment. A design that appears accessible on paper may produce participation patterns suggesting otherwise.
