03 · What You Need to Know
How to Match Sampling Methods With Research Questions and Designs
Why Should Sampling Begin With the Research Question?
The research question determines what the sample must enable the researcher to observe, compare, estimate, or understand.
For example, estimating the prevalence of generative AI use among all undergraduate students requires evidence about users and nonusers across the intended population.
By contrast, investigating how experienced researchers incorporate AI into manuscript preparation may require participants with particular firsthand experiences.
A sampling strategy suitable for one question may be inappropriate for the other.
Researchers should identify the intended inference before deciding whether random selection, stratification, purposive selection, or another method is appropriate.
What Is the Difference Between Sampling and Recruitment?
Sampling concerns how units are selected from a defined population or sampling frame.
Recruitment concerns how selected or potentially eligible individuals are contacted, invited, and enrolled.
For example, a researcher may randomly select 500 students from an enrollment register and invite them by email.
The random selection is the sampling procedure. Email invitations and subsequent consent are recruitment procedures.
If only 100 students respond, the final participant group is affected by nonresponse even though the initial selection was random.
Researchers should distinguish the planned sample from the achieved sample and examine how recruitment affects the intended inference.
What Is the Difference Between Probability and Nonprobability Sampling?
Probability sampling uses a random selection mechanism through which population units have known, nonzero inclusion probabilities under the intended design.
These probabilities support design-based estimation and quantification of sampling uncertainty when the relevant assumptions and procedures are satisfied.
Nonprobability sampling does not establish known selection probabilities for all relevant population units.
Examples include convenience, purposive, quota, and conventional snowball sampling.
Nonprobability samples can provide valuable evidence, but population estimation generally requires additional assumptions or modeling approaches.
The distinction concerns the selection mechanism, not whether a study uses quantitative or qualitative data.
Which Sampling Methods Fit Different Research Objectives?
| Research objective |
Potentially suitable strategy |
Important consideration |
| Estimate a population prevalence or mean |
Probability sampling from a suitable frame |
Coverage, selection probabilities, nonresponse, and weighting |
| Compare important population subgroups |
Stratified sampling or justified subgroup oversampling |
Precision within subgroups and appropriate population weighting |
| Sample geographically dispersed populations |
Cluster or multistage probability sampling |
Clustering, design effects, and selection probabilities |
| Explore a particular lived experience |
Purposive sampling |
Experiential relevance and information richness |
| Develop or refine a qualitative explanation |
Purposive or theoretically guided sampling |
Conceptual development and relevant variation |
| Investigate difficult-to-reach populations |
Specialized network-based or venue-based strategies |
Coverage, network assumptions, and selection bias |
| Estimate an intervention effect |
Sampling appropriate to the target population, alongside a defensible causal design |
Distinguish participant selection from treatment assignment |
| Develop a prediction model |
Sampling aligned with the intended application population |
Case mix, outcome prevalence, and external validation |
These are possible starting points rather than universal prescriptions. The final choice depends on the population, available resources, and specific analytical requirements.
When Is Simple Random Sampling Appropriate?
Simple random sampling is appropriate when a suitable sampling frame exists and every eligible unit can be selected through the specified random procedure.
For example, a university may randomly select students from a complete enrollment register.
Under simple random sampling without replacement, every set of a given sample size has an equal probability of selection.
The method is conceptually straightforward and supports design-based inference.
However, it may be inefficient when important subgroups are small or when the population is geographically dispersed.
Random selection also does not eliminate nonresponse or frame coverage problems.
When Is Systematic Sampling Appropriate?
Systematic sampling selects units at a regular interval from an ordered frame, generally using a random start.
For example, a researcher may select every twentieth record from a suitably ordered enrollment list after establishing the starting position.
The method can simplify selection from large frames.
However, ordering matters. If the frame contains periodic patterns related to the selection interval, the sample may be distorted.
Researchers should examine how the list is organized and whether the resulting selection probabilities support the intended analysis.
When Is Stratified Sampling Appropriate?
Stratified sampling divides the population into defined, nonoverlapping strata and selects units within them using an appropriate probability mechanism.
For example, undergraduate students may be stratified by academic discipline or institutional type.
Stratification can improve precision for some estimates and ensure adequate representation of important subgroups.
However, disproportionate allocation may require weighting for population-wide estimation.
Researchers should choose strata that are relevant to the estimand and available in the sampling frame.
Stratification should not be confused with merely recruiting convenient participants until each category reaches a quota.
When Is Cluster Sampling Appropriate?
Cluster sampling selects groups of units, such as schools, classrooms, communities, or hospitals, rather than sampling every individual directly from one complete population list.
For example, a national educational survey may select schools and subsequently sample students within those schools.
This approach can reduce travel and administrative costs when the population is geographically dispersed.
However, observations within clusters may be correlated.
Cluster sampling therefore requires appropriate sample size planning, variance estimation, and consideration of the number and size of clusters.
A large number of students from a few selected schools does not automatically provide the same precision as an equal-sized simple random sample.
When Is Multistage Sampling Useful?
Multistage sampling selects units through successive stages.
For example, a national study might select regions, then schools, then students.
Each stage should have a defined selection mechanism appropriate to the intended inference.
Multistage designs can be practical when a complete individual-level frame is unavailable but suitable frames exist for higher-level units.
However, selection probabilities and clustering can become more complex.
Researchers should document every stage and use analytical methods that reflect the resulting design.
When Is Convenience Sampling Appropriate?
Convenience sampling recruits participants who are readily accessible.
For example, a researcher may invite students from classes they currently teach.
This approach can be practical for pilot studies, preliminary investigations, and certain focused research objectives.
However, convenience selection does not establish that participants represent a broader population.
Students in the researcher's classes may differ from others in academic discipline, motivation, technology access, or other characteristics.
Researchers should avoid treating a large convenience sample as equivalent to a probability sample.
When Is Purposive Sampling Appropriate?
Purposive sampling deliberately selects participants or cases because their characteristics or experiences are relevant to the research question.
For example, a qualitative study of AI policy implementation may recruit university administrators who have participated in developing or applying institutional AI policies.
Purposive selection may seek contrasting experiences, unusual cases, or variation across important contexts.
Its value depends on the quality and relevance of the information obtained rather than known statistical inclusion probabilities.
However, purposive sampling should not be presented as producing statistically representative population estimates without additional justification.
When Is Snowball Sampling Appropriate?
Conventional snowball sampling uses existing participants or contacts to identify additional potential participants.
It may be useful when the relevant population is difficult to identify through conventional lists or recruitment channels.
For example, researchers investigating informal professional networks may obtain referrals from initial participants.
However, referral networks can overrepresent socially connected individuals and exclude people outside the initial networks.
Researchers should distinguish conventional snowball sampling from specialized respondent-driven sampling methods, which use additional procedures and assumptions for certain forms of population estimation.
When Is Quota Sampling Appropriate?
Quota sampling recruits participants until specified numbers or proportions are reached within selected categories.
For example, a researcher may seek a predetermined number of respondents from each academic discipline.
Quotas can improve coverage of selected categories compared with unrestricted convenience recruitment.
However, quota sampling does not automatically establish known inclusion probabilities within those categories.
Equal numbers across disciplines also do not necessarily reflect their proportions in the target population.
Researchers should distinguish quota-based diversity from probability-based representativeness.
How Does the Sampling Frame Affect Your Choice?
A sampling frame identifies the units or mechanisms through which population members can be selected.
For example, an official enrollment register may support probability sampling of students if it adequately covers the intended population.
However, an institutional email list may omit eligible students or contain outdated records.
Researchers should assess whether the sampling frame matches the target population.
When no complete frame exists, alternative probability designs or carefully justified nonprobability approaches may be considered.
The absence of one complete individual-level list does not necessarily mean probability sampling is impossible.
Does Every Study Need a Representative Sample?
No. Representativeness matters particularly when the study intends to estimate characteristics of a defined population.
For example, estimating the proportion of university students using generative AI requires careful attention to population coverage and selection.
However, a qualitative study examining how experienced researchers negotiate authorship concerns when using AI may prioritize relevant experiences and conceptual variation.
The appropriate sampling strategy depends on whether the objective concerns population estimation, causal inference, prediction, or qualitative understanding.
Researchers should avoid treating representativeness as a universal requirement for all research designs.
How Does Sampling Affect Causal Inference?
Sampling and causal identification are related but distinct concerns.
Random sampling determines how participants are selected from a population. Random assignment determines how eligible participants are allocated to interventions.
A randomized experiment can support internally valid causal inference among its study participants even when participants were recruited through nonprobability methods, subject to the relevant assumptions.
However, extending the estimated effect to a broader population requires additional evidence or assumptions about population differences.
Conversely, a probability sample does not by itself eliminate confounding in an observational exposure comparison.
Researchers should avoid confusing representative selection with causal identification.
How Should Sampling Be Chosen for Prediction Research?
Prediction research requires data that support model development and evaluation for the intended application population.
For example, a model predicting student dropout should be evaluated among students whose characteristics and outcome processes resemble those expected in deployment.
Sampling affects outcome prevalence, predictor distributions, and the range of cases represented.
Case-control or enriched sampling may be useful in some development settings, but it can require attention to calibration and the intended deployment distribution.
External validation using appropriately selected data is often important.
Researchers should not assume that a model developed using a convenient sample will perform equally well across institutions or future cohorts.
How Should Sampling Be Chosen for Qualitative Research?
Qualitative sampling should support the depth and relevance of evidence needed for the research question.
For example, a study of teachers' experiences implementing generative AI policies may recruit participants from schools with different implementation conditions.
Purposive strategies may emphasize maximum variation, critical cases, or theoretically relevant experiences.
Sampling may also evolve as analysis identifies gaps or contrasting cases, depending on the methodological tradition.
Researchers should justify why selected participants are likely to contribute meaningful evidence and how the sample supports the intended analytical claims.
Statistical representativeness is not the default standard for evaluating qualitative sample adequacy.
Does Sample Size Determine the Sampling Method?
No. Sample size and sampling method address different questions.
Sample size concerns how much information is needed for the intended analysis, precision, or qualitative purpose.
Sampling method concerns how participants or other units are selected.
A sample of 1,000 convenience respondents may remain systematically selected, while a smaller probability sample may support more defensible population estimation.
However, probability sampling does not guarantee adequate precision when the sample is too small or the design is inefficient.
Researchers should plan sample size and selection procedures together without treating them as interchangeable.
How Does Nonresponse Affect Sampling Quality?
Nonresponse occurs when selected or invited units do not provide the required information.
For example, students selected through simple random sampling may decline participation or fail to complete a questionnaire.
If participation is related to the study variables, the achieved sample may differ systematically from the intended population.
Researchers should consider appropriate follow-up procedures, response propensity information, weighting, or other justified approaches.
However, a high response rate does not automatically eliminate nonresponse bias, and a lower response rate does not by itself quantify that bias.
The relationship between participation and the variables of interest matters.
Can Weighting Make a Nonprobability Sample Representative?
Weighting may improve some population estimates when relevant population information is available and the assumptions supporting adjustment are plausible.
For example, a nonprobability sample might be calibrated to known population distributions of academic discipline and year level.
However, matching those distributions does not guarantee that respondents resemble nonrespondents on unmeasured characteristics related to AI use.
Weighting cannot automatically eliminate selection bias or compensate for groups entirely absent from the data.
Researchers should distinguish probability-based design weights from model-based adjustments used with nonprobability samples.
What If Important Subgroups Are Very Small?
When important population subgroups are rare, ordinary sampling may produce too few observations for precise subgroup estimates.
For example, a national university survey may include only a small number of students from particular programs.
Stratification or deliberate oversampling may improve subgroup coverage.
However, disproportionate probability sampling requires appropriate treatment of selection probabilities when estimating overall population quantities.
Researchers should specify whether subgroup estimation is a primary objective rather than assuming every subgroup must contribute equal numbers.
What If the Population Is Hidden or Difficult to Reach?
Some populations lack complete lists or are difficult to contact because of privacy, mobility, stigma, or informal organizational arrangements.
Researchers may consider network-based, venue-based, time-location, or other specialized approaches.
However, these methods have different assumptions and limitations.
Conventional referral sampling, for example, may overrepresent well-connected individuals.
Researchers should identify which population members are likely to remain unreachable and how that affects the intended inference.
Difficulty accessing a population does not justify claiming representativeness without supporting evidence.
How Should Sampling Be Chosen When Resources Are Limited?
Budget, time, personnel, and access constraints are legitimate design considerations.
However, they should not silently determine the scientific population or the strength of claims.
For example, a researcher unable to conduct a national probability survey may choose a focused institutional study rather than present a convenience sample as nationally representative.
Alternatively, a multistage probability design may reduce fieldwork costs while preserving a defensible selection mechanism.
The appropriate response is to align the research question, design, and available resources.
Researchers should be willing to narrow the intended inference when the available sampling method cannot support broader claims.
How Can You Evaluate Whether a Sampling Method Fits the Study?
Begin by identifying the population and the quantity or phenomenon the study intends to investigate.
Then examine whether the sampling method can reach relevant units and support the intended analysis.
For population estimation, consider coverage, selection probabilities, weighting, clustering, and nonresponse.
For causal inference, consider how sampling relates to treatment assignment, confounding, and effect transportability.
For qualitative inquiry, consider participant relevance, experiential variation, and the methodological logic of case selection.
For prediction, consider the intended application population and whether model performance can be validated appropriately.
How Should Sampling Methods Be Reported?
Methods sections should identify the target population, sampling frame, sampling units, selection procedure, and recruitment arrangements.
For probability sampling, report relevant stages, selection probabilities, stratification, clustering, and weighting procedures.
For nonprobability sampling, describe how participants were identified and selected, including the rationale for purposive or convenience-based decisions.
Explain relevant coverage limitations and nonresponse.
Researchers should distinguish the intended sample from the achieved sample and avoid unsupported population-wide claims.
Transparent reporting allows readers to evaluate what the sampling strategy actually supports.
