03 · What You Need to Know
What AI-Simulated Participants Can and Cannot Represent
What is an AI-simulated research participant?
An AI-simulated participant is a computationally generated representation of a hypothetical respondent or decision-maker.
Researchers may instruct a language model to answer questions according to a specified profile, scenario, or set of characteristics.
For example, an AI system might be prompted to respond as a student enrolled in an online course, a teacher implementing educational technology, or a consumer evaluating a hypothetical product.
The resulting answers may resemble those that a person could provide.
However, the system is not a recruited human participant. Its responses are generated through the model's computational processes.
Researchers should therefore distinguish simulated respondent records from observations obtained through genuine human participation.
How does generative AI simulate human responses?
Large language models generate outputs conditioned on input instructions, conversation context, and learned patterns from training.
A researcher may provide a hypothetical participant profile containing characteristics such as age category, educational background, or professional role.
The model then produces responses consistent with its interpretation of that profile and the requested task.
However, the process does not establish that the model possesses the experiences attributed to the fictional respondent.
Nor does it establish that actual people with similar characteristics would answer in the same way.
The model may generate plausible narratives because it has learned linguistic and conceptual patterns associated with those circumstances.
Simulation quality must therefore be evaluated rather than inferred from fluency.
What kinds of research participation can AI simulate?
| Research activity |
Possible AI simulation |
Principal limitation |
| Survey research |
Artificial questionnaire responses and rating-scale answers |
Generated distributions may not represent actual populations. |
| Qualitative interviews |
Fictional narratives and responses to interview questions |
Outputs do not document genuine lived experiences. |
| Behavioral experiments |
Simulated choices under hypothetical experimental conditions |
Model behavior may differ from human decisions. |
| Usability testing |
Hypothetical feedback about interfaces and workflows |
Generated feedback cannot establish actual user experience. |
| Research instrument piloting |
Possible interpretations and responses to questionnaire items |
Simulations cannot independently validate human response processes. |
| Qualitative coding tests |
Artificial excerpts with specified themes or characteristics |
Success does not establish validity on authentic participant narratives. |
These applications may be legitimate when the simulated nature of the outputs is explicit and the scientific claims remain appropriately limited.
Can AI-generated responses resemble genuine human survey data?
Yes, under some conditions.
Research by Argyle and colleagues examined whether language models could simulate selected patterns in human survey responses when conditioned on demographic and political information.
Such work suggests that language models may reproduce certain aggregate response patterns in particular research settings.
However, these findings do not establish that AI-generated responses reliably represent all human populations or research topics.
Simulation performance may depend on the model, prompting procedure, population, question wording, and evaluation criteria.
Researchers should avoid generalizing from successful reproduction of selected patterns to universal equivalence between AI-generated and human responses.
Can AI simulate qualitative interview participants?
Generative AI can produce interview-style dialogue and fictional accounts of experiences.
For example, a model might generate a narrative describing difficulties encountered during online learning.
The narrative may include details about internet access, time management, or interactions with instructors.
However, the output does not establish that any actual student experienced those circumstances.
Qualitative research often depends on the situated meanings, experiences, and contexts of genuine participants.
AI-generated narratives may reflect generalized descriptions or assumptions rather than lived experience.
The question of simulating qualitative interviews with generative AI should therefore be approached as a methodological or computational activity rather than equivalent human testimony.
Can AI simulate different demographic groups?
Researchers can condition AI responses on demographic or social profiles, but doing so introduces important validity concerns.
For example, a model instructed to respond as a student from a particular background may produce answers based on generalized associations in its training data.
These associations may reflect stereotypes, incomplete representation, or assumptions embedded in the prompt.
The resulting responses should not automatically be interpreted as representative of actual people sharing those characteristics.
Researchers should evaluate whether the simulated distributions and relationships correspond to appropriate empirical reference data.
They should also consider whether demographic prompting itself shapes the conclusions.
A model's portrayal of a group is evidence about the model's behavior, not necessarily about the group.
Can AI-generated participants represent a population statistically?
Not automatically.
In conventional survey research, population inference depends on the sampling design, response process, measurement properties, and other assumptions.
AI-generated respondents are not selected from a human sampling frame in the same way.
Generating 1,000 fictional respondents does not create a probability sample of 1,000 actual people.
The resulting distribution reflects the model, prompts, generation settings, and any conditioning information.
Researchers may investigate whether those outputs approximate selected empirical population characteristics, but representativeness must be established through validation.
The issue of whether generative AI can predict human survey responses is therefore an empirical question rather than an automatic property of language-model generation.
Why can prompt wording influence simulated participants?
Language-model outputs are sensitive to the instructions and context provided.
For example, asking a model to respond as a struggling online student may produce different answers from asking it to respond as a typical university student.
Similarly, including assumptions about motivation, technology access, or academic performance may influence the resulting responses.
These prompt choices can introduce researcher expectations into the generated dataset.
Researchers should document consequential prompts and examine whether alternative formulations produce materially different outcomes.
Otherwise, the simulated findings may partly reflect how the researcher framed the hypothetical participant rather than a stable characteristic of the model or target population.
Does generating many simulated participants create independent observations?
Not in the same sense as recruiting independent human respondents.
Repeated model outputs may share common training patterns, prompting assumptions, and model-specific biases.
Even when generated responses differ, their dependence on the same computational system can limit the interpretation of apparent sample size.
For example, generating 2,000 simulated student responses does not establish that 2,000 independent students expressed those views.
Researchers should distinguish the number of model-generated records from the number of actual human participants.
Statistical uncertainty should be evaluated according to the simulation design rather than automatically using assumptions intended for independent human samples.
Can AI-simulated participants reproduce behavioral experiments?
Researchers have investigated whether language models can reproduce selected outcomes from human-subject experiments.
For example, Aher, Arriaga, and Kalai explored the use of language models to simulate human responses in experimental settings.
Such studies can provide insight into the extent to which model behavior resembles particular empirical findings.
However, successful reproduction of an experimental pattern does not establish that the model accurately represents every underlying psychological or social mechanism.
Nor does it guarantee that the same result will hold under new conditions or with different populations.
Researchers should distinguish reproducing an observed pattern from explaining the process that generated it.
Can simulated participants be useful for pilot testing?
Yes, particularly for preliminary methodological preparation.
Researchers may use AI-generated responses to examine whether questionnaire instructions are understandable, whether a coding framework can process different types of text, or whether an analytical workflow handles expected response formats.
For example, simulated respondents may reveal that a questionnaire contains ambiguous wording or that a survey platform does not handle certain response combinations.
However, these observations reflect the model's interpretation of the instrument.
They do not replace cognitive interviewing, usability testing, or other empirical pilot procedures when those are necessary to establish how actual participants understand and respond.
The use of AI-generated participants for research pilot testing is best understood as supplementary methodological preparation.
Can simulated participants help test qualitative coding frameworks?
Potentially. Researchers can generate artificial excerpts containing known or specified themes and use them to test coding procedures.
For example, a coding framework intended to identify barriers to online learning might be tested using fictional excerpts about internet connectivity, workload, or instructor communication.
Such testing may reveal inconsistencies in coding rules or software implementation.
However, success on generated excerpts does not establish that the framework adequately captures the complexity of authentic participant narratives.
Real qualitative data may contain ambiguity, contradiction, contextual meaning, and unexpected themes not represented in the synthetic material.
Empirical validation remains necessary when the research claims concern actual human experiences.
Can AI-simulated participants be used to study rare scenarios?
They may be useful for methodological exploration when particular scenarios are difficult to observe or recruit.
For example, researchers might generate hypothetical responses to unusual educational disruptions to test whether an interview protocol addresses relevant possibilities.
However, a model's response to a rare scenario may reflect assumptions or stereotypes rather than reliable evidence about how actual people would behave.
Researchers should therefore frame such work as scenario simulation or methodological testing.
Claims about real-world responses require appropriate validation.
The distinction is particularly important when simulated scenarios involve vulnerable populations or consequential decisions.
What is the difference between studying AI behavior and studying human behavior?
When researchers collect responses from a language model, the directly observed phenomenon is the model's output under specified conditions.
Those observations can support scientific claims about the model's behavior.
For example, a study may investigate how demographic prompts influence a model's simulated survey responses.
That is a legitimate research question about the AI system.
However, concluding that actual demographic groups hold the same attitudes would require additional empirical evidence.
Study of AI behavior
Examines how a computational model responds to prompts, scenarios, or experimental conditions.
Study of human behavior
Examines observations obtained from actual people through an appropriate research design.
The same simulated responses may be relevant to the first question without providing sufficient evidence for the second.
Can AI-simulated participants replace real participants?
Only for activities where artificial responses are methodologically appropriate.
They may substitute for real records during software testing or certain simulation studies.
However, they generally cannot replace genuine participation when the research seeks to establish actual attitudes, experiences, or behavior.
The distinction between synthetic data and real participant evidence remains central.
Recruitment difficulty does not justify presenting generated responses as though actual participants were interviewed or surveyed.
If the research design changes from studying people to studying AI simulations, the research questions, methods, and claims should change accordingly.
What are the risks of treating AI-simulated participants as real?
Misrepresenting artificial responses as genuine participant data can distort the research record.
The United States Office of Research Integrity (ORI) defines fabrication as making up data or results and recording or reporting them.
For example, reporting 300 AI-generated survey responses as observations from 300 recruited students would falsely represent the data collection process.
Even when the artificial origin is disclosed, unsupported claims about real populations may remain methodologically invalid.
Researchers should distinguish research misconduct concerns from broader questions of inference quality.
A transparently reported simulation may be legitimate while still providing insufficient evidence for a particular conclusion about humans.
How should AI-simulated participants be validated?
Validation depends on the intended scientific claim.
If the objective is to examine whether a model reproduces selected human response patterns, researchers should compare its outputs with appropriate empirical reference data.
They may examine response distributions, subgroup differences, relationships among variables, and sensitivity to prompts or model settings.
Repeated generation may be necessary to assess response variability.
However, matching selected aggregate statistics does not establish complete behavioral equivalence.
Researchers should also consider whether the empirical reference data adequately represent the intended population.
Validation should therefore address both the limitations of the AI simulation and the quality of the human comparison.
Should researchers use multiple AI models?
Potentially, when model dependence is relevant to the research question.
Different language models may produce different simulated responses because of differences in training, alignment, architecture, and system behavior.
Comparing several models can help determine whether a finding is specific to one system or appears across the evaluated models.
However, agreement among models does not establish that their outputs accurately represent humans.
Models may share training sources, assumptions, or systematic biases.
Researchers should therefore distinguish cross-model consistency from empirical validity.
The appropriate number and selection of models should be justified by the study design.
How should simulated participant studies be reported?
Researchers should clearly identify the participants as simulated or AI-generated rather than human respondents.
The methods should describe the model, relevant version information, prompts, participant profiles, generation settings, repetitions, and evaluation procedures.
Researchers should also explain whether any genuine human data were used for conditioning or validation.
Artificial response counts should not be presented as human sample sizes.
Where models change over time, the testing date and available configuration information may be important for reproducibility.
The manuscript should distinguish findings about model behavior from claims independently supported by empirical human observations.
What scientific claims can researchers make from simulated participants?
Researchers can make claims supported by the actual simulation design and observations.
For example, they may report that a particular model produced certain response distributions under specified prompts.
They may also evaluate whether those outputs resemble selected human survey results or how sensitive the simulation is to changes in participant profiles.
However, they should not automatically conclude that the generated responses establish population attitudes, lived experiences, or causal effects among real people.
The question of scientific claims supported by AI-simulated participants requires careful separation of model observations from human evidence.
