03 · What You Need to Know
When Representativeness Is Necessary and When Other Sampling Priorities Matter
What Does It Mean for a Sample to Be Representative?
A representative sample provides an appropriate basis for inference about a defined target population for the characteristics being investigated.
In ordinary usage, researchers sometimes describe a sample as representative when its demographic composition resembles that of the population.
However, matching a population on age, gender, or academic discipline does not establish that the sample is representative for every variable.
For example, a university sample may match the distribution of academic programs while disproportionately including students who frequently use generative AI.
Such a sample may still overestimate AI adoption.
Representativeness should therefore be evaluated in relation to the intended population quantity and the processes through which participants were selected and recruited.
Why Does the Research Question Determine Whether Representativeness Matters?
Different research questions require different forms of evidence.
A prevalence question asks how common a characteristic is in a population. A causal question asks what effect an exposure or intervention produces. A qualitative question may investigate how participants experience or interpret a phenomenon.
These objectives are not interchangeable.
For example, estimating the percentage of teachers using AI requires a defensible relationship between the observed sample and the intended teacher population.
Understanding how experienced teachers negotiate AI-related assessment decisions may instead require selecting participants with relevant firsthand experiences.
The sampling strategy should follow the intended inference rather than a universal demand for demographic proportionality.
Which Research Designs Typically Require Representativeness?
| Research objective |
Importance of representativeness |
Primary consideration |
| Population prevalence survey |
Central |
Coverage, selection probabilities, nonresponse, and estimation |
| Population mean or total estimation |
Central |
Appropriate probability design and population inference |
| Randomized intervention study |
Important for broader application, not automatically required for internal causal validity |
Valid treatment assignment and clearly defined estimand |
| Observational causal study |
Relevant to population applicability |
Confounding control, selection mechanisms, and causal identification |
| Qualitative phenomenological study |
Usually not the principal requirement |
Relevant lived experience and analytical depth |
| Grounded theory study |
Not ordinarily the central objective |
Conceptually relevant cases and theoretical development |
| Prediction model development |
Relevant to deployment population and case mix |
Model performance, calibration, and external validation |
| Feasibility or pilot study |
Depends on the feasibility objective |
Information about procedures, recruitment, and implementation |
These distinctions concern methodological priorities. A particular study may have several objectives, each requiring different sampling considerations.
Why Is Representativeness Important for Population Surveys?
Population surveys frequently aim to estimate proportions, means, totals, or distributions for a defined population.
For example, a researcher may estimate the proportion of undergraduate students who use generative AI for academic writing.
If the sample disproportionately includes students with high technology engagement, the estimate may not reflect the broader population.
Probability sampling from a suitable frame can support design-based population estimation because selection probabilities are established through the sampling procedure.
However, the achieved sample may still be affected by coverage error, nonresponse, or measurement problems.
Representativeness is therefore an inferential concern rather than a guarantee produced simply by labeling a procedure random.
Does Probability Sampling Guarantee a Representative Sample?
No. Probability sampling establishes a known random selection mechanism, but a realized sample can differ from the population through sampling variation.
For example, a simple random sample may contain fewer students from a small academic program than expected.
That difference does not necessarily indicate a flawed selection procedure.
However, incomplete frames or systematic nonresponse can create additional distortions.
Researchers should distinguish random sampling uncertainty from coverage and participation problems.
The advantages of probability sampling depend on the actual selection design and its implementation.
Can Quantitative Research Use Nonrepresentative Samples?
Yes. Quantitative research includes objectives that do not require direct estimation of a broad population distribution.
For example, a randomized laboratory experiment may recruit eligible volunteers to investigate the effect of a specified instructional manipulation.
Under appropriate randomization and other design assumptions, the experiment may support a causal comparison within its study population.
However, the treatment effect may differ in populations with different characteristics or implementation conditions.
Researchers should distinguish the validity of the within-study estimate from its applicability beyond the recruited participants.
Using quantitative statistics does not automatically create a requirement for probability sampling.
Does Experimental Research Require Representative Participants?
Not necessarily. Experimental research primarily requires a design capable of supporting the intended causal comparison.
For example, 120 eligible volunteers may be randomly assigned to an AI tutoring intervention or a comparison condition.
Random assignment can reduce systematic baseline confounding between treatment groups in expectation.
However, it does not establish that volunteers represent all students.
If the objective is to estimate the effect among trial participants, representativeness of a broader population may not be the principal concern.
If the objective is to recommend the intervention nationally, population applicability becomes substantially more important.
What Is the Difference Between Internal Validity and Representativeness?
Internal validity concerns whether the study supports the intended conclusion about relationships or effects within the study context.
Representativeness concerns the relationship between the sample and a defined population.
These properties are distinct.
A randomized trial using volunteers may have strong internal causal validity but limited evidence about effects in other populations.
A probability survey may represent a population well for descriptive estimation but cannot establish causation merely because participants were randomly sampled.
Researchers should evaluate each property according to its relevant methodological requirements.
Does Qualitative Research Require a Representative Sample?
Many qualitative studies do not seek numerical estimates of population characteristics.
Instead, they investigate experiences, meanings, processes, or social relationships.
For example, a qualitative study examining how researchers negotiate responsibility for AI-generated text may purposively recruit academics who have encountered relevant authorship decisions.
The adequacy of that sample depends on its relationship to the phenomenon, the quality of the evidence, and the analytical approach.
Statistical representativeness is therefore not usually the primary criterion.
However, researchers should still explain participant selection and the boundaries of their interpretations.
What Does Purposive Sampling Accomplish?
Purposive sampling deliberately selects participants or cases because they possess characteristics relevant to the research question.
For example, a study of AI policy implementation may recruit administrators who participated in developing institutional guidelines.
Randomly selecting university employees could produce many participants without relevant experience.
Purposive selection may therefore improve the relevance and depth of evidence.
However, it does not automatically establish how common the identified experiences or opinions are across the broader population.
The method should be justified according to its analytical purpose.
Can Qualitative Findings Be Transferable Without Statistical Representativeness?
Yes, although transferability requires careful interpretation.
Qualitative findings may provide concepts, explanations, or contextual insights relevant to settings beyond the original participants.
For example, an investigation of how teachers respond to AI-related assessment uncertainty may identify processes recognizable in other institutions.
However, transferability does not mean that the frequency of those experiences can be estimated for all teachers.
Researchers should provide sufficient contextual information for readers to assess the relevance of findings to other circumstances.
Analytical or theoretical transfer should not be confused with statistical population generalization.
What Is Information Power in Qualitative Sampling?
Malterud and colleagues proposed information power as a way of considering qualitative sample adequacy in relation to the study's objectives and methodological characteristics.
A narrowly focused study involving participants with highly relevant experiences may obtain substantial information from a relatively small sample.
A broader investigation spanning many contexts may require greater variation and more participants.
Information power is not a universal numerical threshold.
It encourages researchers to consider the study aim, sample specificity, theoretical background, dialogue quality, and analytical strategy.
Representativeness is not a substitute for these considerations.
Can Case Studies Be Valuable Without Representative Cases?
Yes. Case studies may investigate particular organizations, events, interventions, or processes in depth.
For example, a researcher may examine how one university developed an institutional generative AI policy.
The university need not statistically represent every higher education institution for the investigation to provide useful evidence about policy development processes.
However, the researcher should justify the case selection and avoid treating context-specific findings as universal.
Case study inference depends on the research design, evidence, and analytical argument rather than demographic proportionality alone.
Do Pilot Studies Require Representative Samples?
Not always. Pilot and feasibility studies often examine whether procedures can be implemented, whether participants can be recruited, or whether instruments function as intended.
For example, a pilot test of an AI learning platform may recruit participants who can evaluate interface usability and identify procedural difficulties.
However, if feasibility depends strongly on characteristics of the intended implementation population, the pilot sample should include relevant variation.
A pilot involving only highly proficient technology users may provide insufficient evidence about usability among novices.
Sampling should therefore reflect the specific feasibility questions rather than an automatic requirement for population representativeness.
What About Methodological and Instrument Validation Studies?
Methodological studies may prioritize participants who provide information about the measurement properties or performance of an instrument.
For example, validating an AI literacy assessment may require adequate representation across the ability range and relevant educational contexts.
Strict population-proportional sampling may not always provide enough observations at the extremes of ability.
However, the resulting validity evidence should be interpreted in relation to the populations and uses for which the instrument is intended.
Researchers should distinguish sampling for measurement evaluation from sampling for population prevalence estimation.
When Does Representativeness Matter for Prediction Research?
Prediction research requires data relevant to the population and conditions in which the model will be applied.
For example, a model predicting university dropout should be evaluated among students whose characteristics and outcome processes resemble the intended deployment setting.
However, model development may deliberately include additional rare outcomes or difficult cases.
Such designs can be useful but may require adjustments and careful calibration.
External validation remains important because performance can change across institutions, time periods, and population distributions.
The relevant objective is predictive validity in the intended setting, not demographic balance for its own sake.
What If the Study Concerns a Rare Population?
Research involving rare populations may require specialized recruitment approaches because conventional probability sampling would yield too few relevant participants.
For example, researchers studying a small group of professionals with experience developing institutional AI governance policies may use purposive or network-based recruitment.
Such methods can provide valuable evidence about specialized experiences.
However, researchers should not infer population prevalence from a referral sample without suitable assumptions and methods.
Sampling feasibility and the intended inference should be evaluated together.
Can a Representative Sample Still Produce Invalid Findings?
Yes. Representativeness does not guarantee valid measurement, appropriate analysis, or causal identification.
For example, a probability sample of students may be asked a poorly worded question that systematically misclassifies AI use.
The sample selection may be defensible, but the resulting estimate may still be affected by measurement bias.
Similarly, a representative observational sample does not automatically eliminate confounding between AI use and academic performance.
Sampling is one component of research validity, not a substitute for the others.
Can a Nonrepresentative Sample Produce Valid Findings?
Yes, when the sample and design support the specific conclusion.
For example, a purposive interview sample may support a well-grounded analysis of how experienced researchers interpret authorship responsibility.
A randomized experiment using eligible volunteers may support a treatment-effect estimate within its study population.
However, the same evidence may not support estimates of population prevalence or broad average effects without additional assumptions.
The distinction is explored further in whether a nonrepresentative sample can answer a research question well.
How Do Sampling and Generalizability Interact?
Generalizability depends on the intended target population and the differences between that population and the study participants.
For descriptive population estimation, probability sampling and appropriate response adjustment can provide a defensible inferential basis.
For causal effects, differences in effect modifiers may determine whether results transport to another population.
For qualitative research, contextual and theoretical relevance may support transferability.
Researchers should therefore avoid using generalizability as though it referred to one identical process across all designs.
What Is the Difference Between Statistical and Analytical Generalization?
Statistical generalization uses a defensible inferential process to estimate characteristics or quantities for a defined population.
Analytical generalization concerns the relevance of findings to theoretical propositions, mechanisms, or conceptual explanations.
For example, a probability survey may estimate the proportion of students using generative AI.
A case study may instead contribute an explanation of how institutional policies shape AI adoption.
These are different forms of inference and require different evidence.
Researchers should state which form of generalization their design supports.
Should Every Study Attempt to Maximize Diversity?
Not necessarily. Diversity can be valuable when the research question concerns variation across participants or contexts.
However, a study investigating a narrowly defined experience may appropriately focus on individuals who share particular characteristics.
For example, research on the experiences of first-time users of an AI assessment tool may require participants without prior exposure.
Including experienced users solely to increase diversity could undermine the conceptual focus.
The appropriate range of variation depends on the population and phenomenon being investigated.
When Should Researchers Use Probability Sampling?
Probability sampling is particularly valuable when the research seeks population prevalence, means, totals, or other descriptive estimates.
For example, estimating national patterns of AI adoption among teachers may require a suitable sampling frame and probability-based selection.
However, the design must also address stratification, clustering, nonresponse, and weighting where relevant.
The distinction between probability and non-probability sampling concerns the selection mechanism rather than whether the research uses quantitative data.
When Should Researchers Use Purposive or Other Nonprobability Sampling?
Purposive sampling may be appropriate when participants are selected for particular experiences, roles, or conceptual relevance.
Convenience sampling may be suitable for certain preliminary or feasibility objectives when its limitations are recognized.
Network-based approaches may help reach populations that lack suitable conventional sampling frames.
However, these methods do not automatically support design-based population estimation.
Researchers should describe the selection process and avoid claims that exceed the evidence.
How Can Researchers Decide Whether Representativeness Is Necessary?
Begin by completing the statement: "The purpose of this study is to estimate, explain, predict, or understand..."
Then identify the population or cases to which the intended conclusion applies.
If the study estimates a population proportion or mean, representative population inference is central.
If it investigates a causal effect, prioritize causal identification while considering population applicability.
If it explores experiences or develops theory, prioritize relevant cases and analytical depth.
Sampling should be evaluated according to the intended inference rather than a universal requirement for proportional representation.
How Should Sampling Limitations Be Reported?
Researchers should describe the target population, sampling frame, participant selection, recruitment, and achieved sample.
For probability surveys, report relevant design and response limitations.
For experiments, distinguish the population supporting internal causal inference from the population of intended application.
For qualitative studies, explain the rationale for case selection and the contextual boundaries of interpretation.
Statements about representativeness should be supported by the actual design rather than inferred from sample size or demographic diversity.
