When Does Using a More Accessible Population Change the Scientific Question Too Much?

You can recruit another population more easily, but would you still be answering the same question?

Recruitment problems often produce an attractive solution: study someone else.

Perhaps experienced teachers are difficult to reach, but preservice teachers are readily available. Hospital administrators will not permit access to nurses, but nursing students can be recruited from your university. Small-business owners rarely respond to invitations, while business students can complete your survey during class.

The substitution may solve the logistical problem immediately. Scientifically, however, it can create a more serious one.

A population is not interchangeable merely because its members resemble the original participants in some respects. If the characteristics that distinguish the accessible population are also relevant to the phenomenon being investigated, changing populations can change the question itself.

The substitution goes too far when it changes what the evidence can answer

Judge population similarity in relation to the research question, not convenience

Start with why the original population mattered

Before comparing populations, identify why the original one appeared in the research question or design.

Sometimes population characteristics are central. A question about how novice teachers develop classroom-management strategies requires people who are actually experiencing novice teaching. A question about clinical decision-making among emergency nurses depends on professional experience and the clinical environment. A study of doctoral-student attrition concerns conditions that may not exist among undergraduate students.

In other studies, the original recruitment setting may be less important. If the scientific population is first-year computer science students and the question does not depend on one particular university, moving recruitment from University A to a sufficiently appropriate University B may preserve much of the original question.

This is why an inaccessible population should first prompt you to ask whether the research question can be preserved through another legitimate recruitment route or site before substituting a substantively different group.

Do not confuse the target population, study population, and sample

These concepts are related but not identical.

Your eventual sample is drawn through the study's sampling and recruitment processes. Whether findings can reasonably inform claims beyond that sample depends on the design, selection processes, context, and the particular inference being made.

The problem arises when the study population shifts substantially while the research question continues to imply the original target population.

Compare populations on variables that could matter to the phenomenon

Researchers can be distracted by superficial similarity. Two groups may have similar ages, educational levels, or job titles while differing substantially on the characteristic that gives the research question its meaning.

Suppose the original population is teachers who have used generative AI in actual classroom assessment. Preservice teachers may have similar disciplinary knowledge and may even have studied AI tools. Yet they may lack responsibility for assigning grades, implementing institutional policy, handling academic-integrity disputes, or making consequential assessment decisions.

Those differences matter if the study concerns actual professional decision-making. They might matter much less if the question concerns general attitudes toward hypothetical AI applications.

The scientific relevance of a population difference is therefore question-dependent.

Ask whether the phenomenon still exists in the substitute population

This is often the fastest diagnostic question.

If you want to investigate burnout among practicing intensive-care nurses, does the proposed substitute population actually experience the occupational conditions that constitute the phenomenon? If you want to understand researchers' experiences of journal peer review, have the substitute participants submitted manuscripts and received peer-review decisions? If the question concerns adoption of a technology in professional practice, have the substitute participants actually had the opportunity and responsibility to adopt it?

If the answer is no, the substitution may remove the phenomenon rather than merely change the recruitment source.

Check whether the exposure or intervention means the same thing

Population substitution can also change what an exposure or intervention represents.

An AI decision-support system used by practicing clinicians under real workload, accountability, and patient-care conditions may not represent the same exposure as a simulated system used by students in a classroom exercise. A professional-development program delivered to employed teachers may function differently when adapted for preservice teachers.

If population membership changes the meaning, intensity, consequences, or context of the exposure, the resulting study may answer a related but different question.

Context can be scientifically important even when the people look similar

Moving from one institution to another does not automatically change the scientific question, but neither should site differences be dismissed automatically.

Organizational policies, resources, curriculum, technology infrastructure, institutional culture, workload, incentives, or local practices may influence the phenomenon under investigation. Whether these differences threaten the intended inference depends on what the study asks.

If the question concerns the implementation of a particular institutional policy, changing institutions could be fundamental. If it concerns a psychological construct expected to operate across similar educational settings, the site change may be less consequential, although generalizability still deserves consideration.

Use the conclusion test

A practical way to evaluate a proposed substitution is to imagine that data collection is finished.

Complete this sentence:

“Based on this sample, our findings suggest that...”

Now ask whether you can honestly finish the sentence using the original population.

If you recruited preservice teachers, can the sentence defensibly begin “among experienced secondary-school teachers”? If you studied healthy university students, can it support a conclusion about patients receiving clinical care? If you surveyed managers from large corporations, can the resulting evidence answer a question specifically about microenterprise owners?

If you must change the population named in the conclusion, you have strong evidence that the population substitution has changed the scope of the study.

Distinguish a narrower inference from a different scientific question

Not every population difference requires an entirely new study. Sometimes the substitution mainly narrows what can be inferred.

For example, suppose your original question concerns university instructors broadly, but you can recruit only instructors from private universities. If institutional sector is not central to the mechanism being studied, you might retain much of the substantive question while narrowing the population and being appropriately cautious about inference beyond the sampled context.

By contrast, replacing university instructors with undergraduate students in a study of instructors' grading decisions changes the actors performing the behavior of interest. That is not simply narrower generalizability. It changes the object of inquiry.

Sampling limitations cannot repair a population-question mismatch

Researchers sometimes knowingly recruit an easier population and plan to acknowledge the difference later under “limitations.” That can be reasonable when the limitation concerns restricted generalizability. It is not sufficient when the participants cannot provide evidence about the phenomenon named in the question.

A limitations section can say that findings from one type of university may not transfer fully to another. It cannot transform student responses into observations of practicing professionals' behavior.

If the population change alters what is actually being studied, revise the question, objectives, title, interpretation, and intended claims accordingly.

When replacing teachers with students turns one study into another

Similarity between populations is not enough by itself

Test the substitution against the inference you want to make

When considering a more accessible population, do not begin by asking how similar the two groups look overall. Begin with the scientific logic connecting participants to the research question.

This is ultimately an alignment problem. Your research question, participants, measurements, analysis, and conclusions should refer to the same scientific object. Recruitment feasibility matters, but it should constrain the study transparently rather than quietly redefine it.

Before replacing the ideal population, test the scientific consequences

Questions about replacing an inaccessible population

Accessibility cannot substitute for scientific fit

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.