03 · What You Need to Know
Why Synthetic-Data Labels Matter for Research Integrity and Reproducibility
Why should synthetic research data be labeled?
Labeling communicates the origin and evidentiary status of research information.
Synthetic records may resemble genuine survey responses, experimental measurements, interview transcripts, or other empirical observations.
Without clear identification, collaborators or downstream users may incorrectly assume that the artificial records were collected through actual research procedures.
This confusion can affect sample counts, statistical analyses, interpretations, and reporting.
For example, 200 synthetic survey records should not be counted as 200 additional human participants.
Clear labeling helps prevent such errors and supports accurate communication about the evidence underlying scientific claims.
What does labeling synthetic data actually mean?
Labeling involves identifying records or datasets as artificially generated and providing sufficient information to understand their origin and intended role.
Depending on the project, labels may appear in filenames, metadata, data dictionaries, repository descriptions, or individual dataset fields.
For example, a file named synthetic_student_survey.csv communicates its artificial origin at the file level.
However, filenames can change, and records may be extracted or merged into other files.
Researchers should therefore consider whether additional provenance information is necessary.
The goal is not merely to attach the word synthetic somewhere in the project. It is to ensure that the artificial status remains understandable wherever the data are used.
What levels of synthetic-data labeling are available?
| Labeling level |
Typical implementation |
When it is useful |
| Dataset level |
Dataset title, metadata, README, or repository description |
When the entire dataset is synthetic |
| File level |
Descriptive filenames and file-specific documentation |
When empirical and synthetic files are stored separately |
| Record level |
A field identifying whether each row is observed or synthetic |
When artificial and empirical records coexist |
| Variable level |
Indicators identifying generated, imputed, or transformed values |
When only selected fields contain estimated or generated information |
| Generation-run level |
Identifiers linking records to particular generation procedures |
When several models, scenarios, or generation runs are compared |
| Publication level |
Methods descriptions and data availability statements |
When synthetic data contribute to reported research procedures or results |
These approaches can be combined. A dataset may contain record-level labels while also being accompanied by detailed generation metadata.
The appropriate choice should reflect the risk of confusion and the information needed for reproducibility.
Is a synthetic filename sufficient?
Sometimes, but not always.
If an entire file contains artificial test records and remains within a controlled workflow, a descriptive filename supported by documentation may be adequate.
However, filenames are relatively fragile provenance mechanisms.
They can be renamed, shortened, removed during exports, or separated from the records they originally described.
A collaborator may copy several rows into another dataset without preserving the filename.
For reusable research datasets, metadata and accompanying documentation can provide more durable identification.
Researchers should not rely exclusively on a filename when losing that information would materially affect the interpretation of the data.
Should every synthetic record contain a label?
Not necessarily.
When every record in a dataset is synthetic and the dataset's provenance is clearly documented, adding the same indicator to every row may provide limited additional value.
However, record-level labeling becomes particularly useful when artificial and empirical observations are stored or analyzed together.
For example, a variable named record_origin might distinguish observed records from generated records.
The labels should be defined in the data dictionary and applied consistently.
Researchers should also ensure that the provenance indicator survives merging, filtering, exporting, and other transformations.
The important requirement is reliable traceability, not a universal rule that every row must contain an identical label.
How should mixed synthetic and empirical datasets be labeled?
Mixed datasets require particular care because records with different origins may appear identical in structure.
For example, a research project might contain genuine survey responses alongside synthetic records used for methodological comparison.
If the records are combined, researchers should maintain a reliable way to distinguish their origins.
Illustrative Dataset Structure
Preserving record provenance
| record_id |
engagement_score |
support_score |
record_origin |
| R001 |
4 |
5 |
observed |
| R002 |
3 |
4 |
observed |
| S001 |
4 |
4 |
synthetic |
| S002 |
2 |
3 |
synthetic |
All values in this illustration are hypothetical. The table demonstrates how a provenance field can distinguish the origin of records with otherwise similar structures.
The presence of a provenance field does not automatically justify combining synthetic and empirical observations in statistical inference.
Researchers must still establish that the analytical design appropriately accounts for their different origins.
How should imputed values be labeled?
Imputed values require a related but distinct form of documentation.
Imputation estimates missing information within an existing dataset under specified assumptions.
For example, a genuine participant may have completed most questionnaire items but left one response missing.
An imputation procedure may estimate that missing value for a particular analysis.
The participant remains genuine, but the estimated response was not directly observed.
Researchers should preserve information about missingness and document the imputation procedure.
Where appropriate, variable-level indicators or separate imputed datasets may help distinguish observed from estimated values.
Multiple imputation also requires appropriate treatment of uncertainty rather than simply replacing missing values with one apparently observed number.
Should AI-generated and statistically simulated data have different labels?
Potentially, when the distinction matters to the research.
Synthetic data may be generated through statistical simulation, probabilistic models, generative neural networks, or language-model prompting.
These methods can have different assumptions, limitations, and validation requirements.
A general synthetic label identifies the artificial status of the records, but it does not explain how they were produced.
Researchers may therefore document the generation method separately or include a generation-method identifier when comparing outputs from different approaches.
The distinction between synthetic and simulated data is not absolute, so labels should describe the actual procedure rather than impose an unsupported categorical boundary.
What information should synthetic-data metadata contain?
Metadata should provide sufficient context for understanding and evaluating the dataset.
Relevant information may include the purpose of generation, source-data characteristics, generation method, software or model version, relevant parameters, and date of creation.
Researchers may also document random seeds, validation procedures, known limitations, and intended use restrictions.
For example, a synthetic dataset generated for software testing should be identified as unsuitable for empirical claims about a target population unless additional evidence supports such use.
When sensitive source data were used, metadata should not unnecessarily disclose confidential information.
The appropriate level of detail should support reproducibility while respecting privacy, security, and contractual restrictions.
How does data provenance relate to synthetic-data labeling?
Provenance describes the origin and history of data, including the activities that produced or transformed them.
Labeling is one way to communicate provenance, but it is not the entire provenance record.
For example, identifying a dataset as synthetic does not reveal which generation model was used, whether empirical source records were involved, or how the outputs were validated.
The World Wide Web Consortium's PROV framework provides a general conceptual model for describing relationships among entities, activities, and agents.
Researchers do not necessarily need to adopt that framework formally, but its underlying principle is relevant.
A dataset should be traceable to the process that produced it.
The distinction between synthetic data and real research observations becomes more defensible when provenance is preserved beyond a simple label.
Can synthetic-data labels be lost during analysis?
Yes. Data-processing operations can remove or obscure provenance information.
For example, a script may select only variables required for a statistical model and inadvertently discard the field identifying record origin.
A merge operation may combine empirical and synthetic records without preserving their source identifiers.
Summary tables may also conceal the presence of artificial records if the analytical procedures do not distinguish them.
Researchers should therefore examine whether provenance survives consequential transformations.
Automated checks can help identify datasets that unexpectedly lack required origin information.
Version control and documented processing procedures may also support the reconstruction of how records entered an analysis.
Can labels prevent synthetic data from entering empirical analyses accidentally?
They can reduce the risk, particularly when combined with appropriate analytical safeguards.
For example, a research script intended to analyze genuine survey responses may explicitly select records identified as observed.
The script could also verify that the number of records matches the expected empirical sample.
However, labeling alone cannot guarantee correct analysis.
A programming error may ignore the provenance field, or a record may be incorrectly labeled.
Researchers should therefore combine clear identification with code testing, data validation, and appropriate review.
The use of synthetic data to test research analysis code can help verify that provenance-based selection rules behave as intended.
Does labeling synthetic data prevent research fabrication?
Labeling supports transparency, but it does not independently prevent misconduct.
The United States Office of Research Integrity (ORI) defines fabrication as making up data or results and recording or reporting them.
When invented observations are falsely represented as genuine empirical evidence, the research record is misleading regardless of whether the original generation process was documented elsewhere.
For example, a researcher cannot legitimately report 300 actual participants merely because an internal file identifies 100 of those records as synthetic.
The manuscript must accurately represent the empirical sample and the role of the artificial records.
Furthermore, a clearly labeled synthetic dataset may still be methodologically unsuitable for a particular research question.
The distinction between synthetic and fabricated data concerns truthful representation and scientific purpose, not labeling alone.
Should synthetic data be labeled when used only for internal testing?
Generally, yes, although the required documentation may be relatively simple.
Internal test datasets can be confused with empirical files, particularly when they share variable names and file formats.
Clear labels reduce the risk that artificial records are accidentally incorporated into substantive analyses.
They also help collaborators understand the purpose of files that may be reused months or years later.
For a small software test, a descriptive filename and brief README may be sufficient.
More complex projects may benefit from generation scripts, provenance fields, and automated validation.
The documentation effort should be proportionate to the consequences of confusion.
How should synthetic data be labeled in public repositories?
Public repositories should communicate the artificial origin of the dataset in a way that remains visible to downstream users.
The repository title or description may identify the dataset as synthetic, while a README or data dictionary explains how it was generated and what it can support.
Where appropriate, metadata should identify the generation procedure, relevant source information, validation results, and usage limitations.
Researchers should avoid descriptions that could imply genuine participant recruitment or experimental measurement when those activities did not occur.
For example, a dataset titled Student Engagement Survey Responses may be misleading if every record was generated artificially and the description does not clarify that fact.
A more precise title would identify the records as synthetic student engagement survey data.
Repository documentation should remain useful even when individual files are downloaded separately.
Can synthetic-data labeling support reproducibility?
Yes. Accurate provenance documentation helps researchers understand how artificial records were produced and used.
For example, a synthetic dataset accompanied by generation code, parameters, and random seeds may allow others to reproduce or evaluate the generation procedure.
However, not all generative systems produce identical outputs across repeated runs.
Model versions, computational environments, and nondeterministic behavior may affect the resulting records.
Researchers should therefore distinguish reproducing the generation procedure from reproducing an identical dataset.
Where exact regeneration is necessary, preserving the generated records themselves may be appropriate.
Labeling contributes to reproducibility by making the evidentiary role of the records explicit.
Should synthetic datasets carry warnings about their limitations?
Where relevant, yes.
A label identifying artificial origin does not explain whether the dataset is appropriate for particular analyses.
For example, a synthetic dataset may be suitable for testing data-processing software but unsuitable for estimating subgroup differences.
Another may preserve selected statistical properties while presenting residual privacy risks.
Researchers should document material limitations that affect foreseeable reuse.
However, warnings should be specific rather than generic statements that synthetic data may be imperfect.
Useful documentation explains which properties were evaluated, which applications are supported, and which conclusions would exceed the available evidence.
What if synthetic records are mistakenly reported as genuine observations?
Researchers should investigate the source of the error and determine which analyses or conclusions were affected.
Relevant files, provenance records, and analytical procedures should be preserved.
If the problem is identified before publication, the dataset and reporting should be corrected.
If results have already been submitted or published, appropriate editorial or institutional correction procedures may be necessary.
Accidental confusion does not automatically establish formal research misconduct, although it may seriously compromise the reliability of the findings.
Where invented observations are knowingly presented as genuine, the situation raises a fabrication concern under applicable definitions.
Clear labeling and traceable provenance can reduce the likelihood of these problems and support their investigation when they occur.
How should synthetic-data labeling be described in a research paper?
The methods should explain whether the research used synthetic data, empirical observations, or both.
Researchers should identify the role of each data source and explain how artificial records were distinguished during analysis.
For mixed datasets, the manuscript should clarify whether generated records contributed to model training, software testing, simulation, or another methodological activity.
Artificial record counts should not be presented as genuine participant numbers.
Where AI was used to generate the records, applicable disclosure requirements should also be followed.
The broader issue of reporting synthetic data in research papers
