03 · What You Need to Know
When AI-Generated Synthetic Data Are Scientifically Appropriate
What are synthetic research data?
Synthetic data are artificially generated records created to represent selected characteristics, structures, or relationships relevant to a research purpose.
They may be generated using statistical models, simulations, generative machine-learning systems, or other computational procedures.
Some synthetic datasets are designed to resemble an existing empirical dataset. Others are generated from theoretical assumptions without relying on a particular observed dataset.
For example, a researcher may create artificial survey records containing specified questionnaire variables to test an analysis program.
Another researcher may generate simulated measurements under a mathematical model to evaluate how a statistical estimator performs.
In both cases, the observations are artificial. Their legitimacy depends on whether that status is accurately represented and whether the method is appropriate for the research question.
How is synthetic data generation different from fabrication?
The distinction concerns the representation and use of the generated material.
The United States Office of Research Integrity (ORI) defines fabrication as making up data or results and recording or reporting them.
In the context of misconduct, the problem arises when invented observations or results are falsely presented as genuine research evidence.
Generating artificial observations for a clearly identified simulation does not, by itself, constitute fabrication.
Legitimate synthetic data generation
Artificial records are created for a defined research purpose, with their origin and methodological limitations accurately identified.
Research data fabrication
Invented observations or results are falsely recorded or reported as though they came from genuine participants, experiments, measurements, or other research activities.
A dataset containing 500 artificial questionnaire records may be appropriate for testing survey-analysis software. The same records cannot legitimately be reported as responses collected from 500 actual participants.
The distinction between synthetic and fabricated data is therefore not determined simply by whether the values were generated computationally.
What legitimate research purposes can synthetic data serve?
Synthetic data can support several research activities when their limitations are properly understood.
| Research purpose |
Possible synthetic-data use |
Important limitation |
| Software testing |
Testing data import, cleaning, analysis, and visualization functions |
Successful tests do not establish performance on all real datasets. |
| Statistical methods development |
Evaluating estimators under controlled data-generating assumptions |
Findings depend on the simulation scenarios examined. |
| Prediction model development |
Supplementing development or testing under specified conditions |
Artificial data may not reproduce real-world distributions or rare cases. |
| Privacy-related research |
Creating datasets that reduce direct exposure of identifiable records |
Synthetic data can still create disclosure or re-identification risks. |
| Educational demonstrations |
Providing practice datasets for teaching analytical methods |
Artificial observations should not be presented as genuine research evidence. |
| Methodological pilot testing |
Testing coding schemes, workflows, and computational procedures |
Simulation does not replace validation involving actual participants when required. |
These applications have different validation requirements. A dataset suitable for checking whether a script executes correctly may be unsuitable for training a model intended for real-world deployment.
Does legitimate synthetic data need to resemble real data?
Not always.
The degree of resemblance required depends on the intended purpose.
For software testing, researchers may deliberately generate unrealistic values, missing fields, or unusual combinations to determine whether a program handles unexpected inputs correctly.
For statistical simulation, the important requirement may be that the data follow a specified mathematical model rather than resemble an existing empirical sample.
For applications intended to approximate real-world data, however, distributional similarity and preservation of relevant relationships may be important.
Researchers should therefore avoid using realism as a universal criterion for synthetic-data quality.
A highly realistic dataset may be inappropriate for testing rare software errors, while a deliberately simplified simulation may be scientifically valuable for investigating an estimator's properties.
Can generative AI create synthetic survey data?
Yes. AI systems can produce artificial questionnaire responses, demographic variables, and open-ended answers.
Such records may be useful for testing survey platforms, evaluating data-processing scripts, or demonstrating analytical methods.
However, synthetic respondents do not constitute actual human participants.
Researchers should not use generated responses to claim that a target population holds particular attitudes or behaves in a particular way without appropriate empirical support.
For example, AI-generated responses from fictional teachers may help test a questionnaire's data structure. They cannot independently establish teachers' actual perceptions of educational technology.
The distinction becomes especially important when examining AI-generated survey responses that might otherwise be mistaken for genuine participant observations.
Can synthetic data help researchers develop statistical methods?
Yes. Simulation is a well-established approach to evaluating statistical procedures.
Researchers may generate artificial datasets under specified assumptions and examine how an estimator behaves across repeated simulated samples.
For example, a methodological study may investigate how missing data, unequal group sizes, or violations of model assumptions affect an analytical procedure.
The resulting findings concern the method's performance under the simulated conditions.
They do not automatically establish that the same performance will occur in every empirical setting.
Generative AI may assist in writing simulation code or constructing candidate scenarios, but the researcher must verify that the implemented data-generating process corresponds to the intended assumptions.
A simulation is only as informative as the conditions it represents and the correctness of its implementation.
Can synthetic data be used to train prediction models?
Potentially. Synthetic data may support prediction-model development, particularly when genuine observations are scarce, imbalanced, or difficult to access.
However, artificial data may fail to preserve important relationships, rare cases, or subgroup characteristics.
A prediction model trained on synthetic observations may perform well on similar artificial test data while performing poorly on genuine observations.
Researchers should therefore evaluate the model using appropriate real-world or otherwise independently justified reference data whenever the intended claim concerns real-world performance.
They should also examine whether synthetic-data generation introduces biases or reproduces limitations of the original training material.
The question of using synthetic data to develop prediction models requires validation beyond demonstrating that a model can learn patterns from generated records.
Can synthetic data protect participant privacy?
Potentially. Synthetic datasets may reduce the need to distribute original participant-level records.
For example, researchers may generate artificial records that approximate selected characteristics of a sensitive dataset while avoiding direct release of the original observations.
However, synthetic data are not automatically anonymous or free from disclosure risk.
Generative models may reproduce distinctive records or reveal information about individuals represented in their training data under some circumstances.
Privacy risk depends on the generation method, the source data, the output, and the potential information available to an adversary.
Researchers should evaluate whether synthetic data protect research participant privacy and whether they can still leak participant information rather than assuming that artificial generation guarantees anonymity.
Can synthetic data replace real participant data?
Only for purposes where artificial observations provide an appropriate evidentiary basis.
Synthetic data can replace real records in some software-testing or methodological-development activities.
They generally cannot be treated as equivalent substitutes when the research objective is to establish what actual participants experienced, believed, or did.
For example, a dataset generated to resemble university students' survey responses does not independently establish students' actual engagement with online learning.
Similarly, an AI-generated interview transcript does not document a human interview that never occurred.
The distinction between synthetic and real participant data depends on the research question and the type of inference being made.
How should researchers validate synthetic data?
Validation should examine whether the artificial dataset is suitable for its intended application.
For a dataset intended to approximate empirical observations, relevant assessments may include marginal distributions, relationships among variables, subgroup characteristics, rare events, and the performance of downstream analyses.
For software testing, validation may focus on whether the generated records exercise the intended program functions and edge cases.
For statistical simulation, researchers should verify that the implemented data-generating process matches the specified mathematical assumptions.
Where source data contain sensitive information, privacy evaluation may also be necessary.
No single validation metric establishes that a synthetic dataset is appropriate for every purpose.
The question of how researchers validate synthetic data should therefore be answered in relation to the intended scientific use.
What limitations arise when language models generate numerical datasets?
General-purpose language models can produce structured numerical outputs, but their responses should not automatically be treated as samples from a validated statistical distribution.
Generated values may contain arithmetic inconsistencies, inappropriate dependencies, or patterns that do not satisfy the intended assumptions.
When a researcher requests a dataset with specified characteristics, the model may produce plausible-looking values without implementing a rigorous statistical generation procedure.
For research requiring controlled statistical properties, verified simulation code or purpose-built data-generation methods may be more appropriate.
Generative AI can still assist with drafting code or explaining methodological options, provided that the implementation is independently checked.
Researchers should distinguish the convenience of generating a table from the scientific validity of the process that produced its values.
Should synthetic datasets be labeled?
Yes. Artificial records should be clearly identifiable as synthetic or simulated wherever confusion with genuine observations could occur.
Appropriate labeling may involve dataset documentation, variable-level provenance information, metadata, filenames, or explicit descriptions in the research methods.
When a project uses both empirical and artificial records, researchers should maintain a reliable distinction between them.
Labels should remain understandable to collaborators and downstream users, not merely to the person who generated the dataset.
However, labeling alone does not establish methodological legitimacy.
A synthetic dataset must still be appropriate for the research question, and the conclusions must remain consistent with its artificial origin.
The need for clear labeling of synthetic research data is therefore both a transparency and a data-management concern.
How should synthetic data be reported in a research paper?
Researchers should explain why synthetic data were used, how they were generated, and what role they played in the analysis.
The description should be sufficiently detailed for readers to evaluate the method and, where feasible, reproduce the generation procedure.
Relevant information may include the generation approach, source data where applicable, assumptions, model or software versions, validation procedures, and limitations.
Where AI systems were used, applicable journal and institutional AI disclosure requirements should also be followed.
For example, a manuscript should not simply state that 1,000 observations were analyzed if those observations were artificially generated and the distinction is necessary to interpret the results.
Instead, the methods should establish the synthetic nature of the observations and explain what conclusions the analysis can support.
The separate issue of reporting synthetic data in research papers requires attention to both methodological detail and evidentiary interpretation.
Can synthetic data generation itself create a research-integrity problem?
Yes, if the generation or reporting process misrepresents the research.
For example, a researcher might present artificial observations as genuine participant data, conceal consequential assumptions, or claim that a synthetic dataset demonstrates a real-world effect without appropriate evidence.
These situations differ in seriousness and may involve fabrication, misleading reporting, or methodological weaknesses depending on the circumstances.
However, the mere presence of synthetic data does not establish misconduct.
The broader question of when AI use becomes a research-integrity problem concerns the honesty and reliability of the research process rather than the use of a particular technology.
