03 · What You Need to Know
How Recruitment Constraints Affect Research Design and Evidence Quality
Distinguish Recruitment Difficulty From Sample Inadequacy
Recruitment difficulty concerns the practical ability to identify, contact, enroll, and retain eligible participants. Sample inadequacy concerns whether the participants ultimately included can provide sufficient evidence for the study's objectives.
These problems are related but not identical.
A researcher may recruit fewer participants than originally planned yet retain adequate precision for a narrower descriptive objective. Another researcher may recruit the intended number but obtain a sample that excludes important population groups.
Consequently, recruitment success should not be evaluated solely by comparing the achieved number with the original target.
Researchers must also consider who participated, who did not participate, how participants were selected, and whether the resulting data are appropriate for the intended conclusions.
Why Might the Required Sample Be Difficult to Recruit?
Recruitment challenges arise from different mechanisms, and each may require a different response.
-
Limited eligible population: The condition, experience, or characteristic under investigation may be uncommon.
-
Restrictive eligibility criteria: Inclusion and exclusion requirements may substantially reduce the available population.
-
Access barriers: Institutions, organizations, or gatekeepers may restrict contact with potential participants.
-
Participant burden: Travel, repeated assessments, lengthy instruments, or inconvenient schedules may discourage participation.
-
Trust and privacy concerns: Prospective participants may hesitate because of confidentiality, stigma, or uncertainty about how their information will be used.
-
Limited recruitment capacity: The team may lack sufficient time, personnel, funding, or recruitment channels.
-
Population mismatch: The intended target population may be broader than the population the research team can realistically access.
Before changing the methodology, identify which of these factors is responsible for the shortfall. Increasing advertising expenditure will not solve a problem caused primarily by an extremely small eligible population.
Does a Smaller Sample Automatically Make a Study Invalid?
No. Sample adequacy depends on the research question, design, outcome, analytical approach, and intended interpretation.
In quantitative research, a smaller sample may reduce precision, increase uncertainty, or decrease the probability of detecting a specified effect. The consequences depend on the expected effect size, variability, statistical model, and other design assumptions.
In qualitative research, sample adequacy is not determined by statistical power. Instead, researchers consider whether the collected material is sufficiently rich and relevant to address the analytical purpose.
Nevertheless, neither approach permits arbitrary sample reduction without methodological justification.
A small sample may be suitable for a narrowly focused exploratory study but inadequate for a complex analysis involving several subgroups or rare outcomes.
Reassess Quantitative Sample Size Using the Primary Research Objective
Quantitative sample size planning should begin with the primary estimand, hypothesis, or precision requirement rather than a preferred number of participants.
Relevant considerations may include:
- The outcome being estimated or compared.
- The smallest effect or difference considered scientifically meaningful.
- Expected outcome variability or event frequency.
- The desired confidence interval width or statistical power.
- The significance level for hypothesis-testing designs, where applicable.
- Clustering, repeated measurements, or other dependencies.
- Anticipated missing data, dropout, and nonadherence.
When the available sample differs from the planned sample, researchers should reassess the expected precision or the range of effects the design can meaningfully investigate.
For example, a descriptive study estimating a population proportion may remain useful with a smaller sample if the resulting uncertainty is acceptable for its purpose. A comparative study designed to detect a small effect may become substantially less informative when enrollment falls far below the planned target.
Researchers should avoid changing the assumed effect size merely to make the reduced sample appear adequate. Assumptions should remain scientifically justified.
How Does Sample Size Affect Statistical Precision?
For many simple estimation problems, uncertainty decreases as sample size increases, although the relationship is not linear.
For example, under a simple random sampling model with independent observations, the approximate standard error of a sample proportion is:
This example illustrates why increasing sample size can improve precision. It does not imply that every study requires 400 participants or that a sample of 100 is inherently inadequate.
The appropriate level of precision depends on the research objective and the consequences of uncertainty.
Why Recruitment Shortfalls Cannot Always Be Solved by Changing the Analysis
Researchers sometimes respond to insufficient enrollment by selecting a different statistical test, removing variables, or switching from parametric to nonparametric methods.
Such changes may be appropriate when justified by the data structure and research question. However, they do not automatically restore information that was never collected.
For example, replacing a planned regression model with a simpler analysis may reduce the number of estimated parameters but also remove important confounders or change the scientific question.
Similarly, a nonparametric test is not a general solution to low statistical power.
Post hoc changes motivated by observed results may introduce additional interpretive problems. Researchers should document substantive changes, distinguish prespecified from exploratory analyses, and avoid presenting revised analyses as though they had been planned from the beginning.
Watch Out
Do not interpret a statistically nonsignificant result from an underpowered study as proof that no meaningful effect exists. Examine the estimated effect and its uncertainty, and distinguish insufficient evidence from evidence supporting a sufficiently precise conclusion.
Can You Broaden Eligibility Criteria to Recruit More Participants?
Potentially, but eligibility criteria should be changed only after examining their scientific and ethical purposes.
Some criteria protect participants, define the target population, or ensure that the intervention and measurements are appropriate. Others may have been unnecessarily restrictive.
For example, excluding participants with a broad range of characteristics may simplify a study but substantially reduce recruitment and limit applicability. Revising unnecessary exclusions could improve both feasibility and relevance.
However, broadening eligibility may increase heterogeneity, introduce additional confounding, or require changes to the analysis and sample size plan.
Any substantive change should be justified and, where required, approved before implementation.
How Can You Improve Recruitment Without Weakening the Sample?
Recruitment strategies should address the barriers experienced by the intended population while preserving voluntary participation and appropriate selection procedures.
Potential improvements include clearer participant information, additional approved recruitment channels, flexible scheduling, accessible study materials, reasonable reimbursement, and reducing unnecessary participant burden.
Community engagement or collaboration with trusted organizations may improve understanding of the study and identify practical obstacles to participation.
However, these strategies must be assessed for possible selection effects. Online-only recruitment, for example, may exclude individuals with limited internet access or digital literacy.
Reimbursement and incentives should comply with applicable ethical requirements and should not create inappropriate pressure to participate.
Recruitment efficiency should never be achieved through misleading claims, inadequate consent, or concealment of participant burdens.
What If the Population Is Too Rare or Difficult to Access?
Some research questions concern populations that cannot realistically support large samples within a single institution or recruitment period.
In such cases, researchers may consider multisite collaboration, longer recruitment periods, appropriate registries, existing datasets, or alternative designs suited to the available population.
For certain rare conditions or specialized populations, carefully designed case series, descriptive investigations, or qualitative studies may provide useful evidence. However, these approaches answer different questions from adequately powered comparative studies.
The rarity of a population does not eliminate the need to justify the selected design. It may instead require more explicit recognition of what can and cannot be inferred from the available observations.
How Should Qualitative Researchers Assess Sample Adequacy?
Qualitative sample adequacy depends on the relationship between the research question, sampling strategy, methodological approach, and information obtained.
Malterud and colleagues proposed the concept of information power, suggesting that sample requirements may be influenced by the specificity of the study aim, participant characteristics, use of established theory, quality of dialogue, and analytical strategy.
Other qualitative traditions use concepts such as thematic or theoretical saturation, although their meaning and appropriateness vary across methodologies.
Researchers should avoid treating saturation as a universal numerical threshold or assuming that a small sample is sufficient simply because interviews appear repetitive.
A narrowly focused study involving participants with highly relevant experiences may require a different sampling strategy from an investigation seeking variation across several social groups.
If an important participant category cannot be recruited, the researcher must determine whether the study can still answer its question or whether the scope requires revision.
Can You Change the Research Question to Fit the Accessible Sample?
Yes, provided the revised question remains meaningful and the design can answer it appropriately.
For example, a study originally intended to compare outcomes across several demographic groups may be unable to recruit sufficient participants in each group. A narrower descriptive study of the accessible population may remain defensible.
Similarly, a planned effectiveness trial may be redesigned as a feasibility study focused on recruitment, intervention delivery, and measurement procedures.
These adaptations should be made transparently. A feasibility study does not become an effectiveness trial merely because it reports outcome differences.
Researchers should also avoid redefining the target population after observing favorable results simply to create an appearance of methodological adequacy.
When Is It Better to Postpone or Discontinue the Study?
Postponement or discontinuation may be appropriate when recruitment limitations prevent the study from producing sufficiently informative evidence for its intended purpose.
This decision may be especially important in intervention research where participants assume burdens or risks and the study has little realistic prospect of answering its primary question.
However, low recruitment does not automatically make all collected data worthless. Depending on the protocol and ethical permissions, the data may support a limited descriptive or feasibility analysis.
Researchers should distinguish between stopping recruitment, terminating an intervention, revising the objectives, and abandoning the study entirely. These decisions may involve different ethical, scientific, and administrative obligations.
Where applicable, consult the supervisor, research ethics committee, funder, trial oversight body, or other responsible authority before implementing substantial changes.