03 · What You Need to Know
What Reviewers May Reasonably Ask About AI-Assisted Research
Why Was AI Necessary or Appropriate for the Research Task?
A reviewer may question why the research team selected AI rather than an established alternative.
The author should explain the intended contribution and why the selected system was considered suitable.
For example, AI-assisted classification of large volumes of unstructured text may be justified when the procedure is validated and offers a useful analytical capability.
By contrast, using a generative chatbot to perform straightforward arithmetic may be harder to justify when established statistical software provides a more direct and reproducible procedure.
The defense should focus on methodological suitability, not technological novelty.
Researchers need not demonstrate that AI was indispensable, but they should establish that its use was reasonable and did not introduce unjustified weaknesses.
Can You Identify Exactly What AI Contributed?
Reviewers may need to distinguish AI-generated material from the work of human researchers.
Authors should be able to identify whether AI generated code, proposed analytical categories, summarized sources, drafted text, transformed data, or influenced methodological decisions.
This distinction matters because different contributions require different forms of validation.
For example, AI-assisted language editing may primarily require checking that scientific meaning was preserved. AI-generated classifications used as primary research variables may require task-specific performance evaluation.
A generic statement that AI assisted the study does not establish which procedures are subject to scrutiny.
Can You Justify the Selected AI Model?
A reviewer may ask why a particular model or service was chosen.
Relevant considerations may include task suitability, available validation evidence, model documentation, data protection, reproducibility, and the availability of alternative methods.
Authors should identify the system and relevant configuration to the extent possible.
Where the model version is unavailable or the provider does not disclose certain technical details, the limitation should be acknowledged.
Researchers should not claim that a model is scientifically validated merely because it is commercially prominent or performs well on unrelated benchmarks.
Can You Demonstrate That the AI Outputs Were Verified?
Verification is often central to the reviewer's assessment.
A reviewer may ask how AI-generated citations were checked, how code was tested, or how qualitative classifications were evaluated against source material.
The author should describe the actual verification procedure and its findings.
For example, stating that statistical code was reviewed is less informative than explaining that its transformations were inspected, calculations were independently reproduced, and important assumptions were evaluated.
Similarly, qualitative coding validation should reflect the selected methodological approach rather than relying automatically on a single agreement statistic.
Authors should be prepared to provide appropriate supporting records when requested and permitted.
Can You Defend the Research Method Independently of AI?
AI may recommend a statistical model or qualitative analytical procedure, but the recommendation itself is not a methodological justification.
Reviewers may ask whether the selected method addresses the research question and whether its assumptions are satisfied.
For example, if AI recommends multilevel modeling, the researcher should explain the data structure, analytical purpose, model specification, and relevant assumptions.
The response should rely on methodological evidence and the study design.
Researchers should be able to defend the analysis even if the reviewer has no interest in the AI system's own explanation.
Can You Explain How AI Affected the Data?
AI-assisted data processing may alter observations, variables, categories, or analytical datasets.
A reviewer may reasonably ask whether the system excluded records, recoded values, imputed missing data, or transformed variables.
Researchers should preserve enough information to explain these operations and verify their effects.
For example, if AI-generated code removed incomplete records, the author should establish how many observations were affected and whether the exclusion procedure was methodologically justified.
Unexplained data transformations can undermine confidence in the findings even when the final numerical outputs appear plausible.
Can You Defend AI-Assisted Qualitative Interpretation?
When AI contributes to qualitative analysis, reviewers may question whether the generated categories reflect participant meaning and the selected analytical approach.
Authors should explain whether AI suggested preliminary codes, applied a predefined coding framework, or contributed to theme development.
They should also describe how researchers examined the outputs against original data and handled disagreements or alternative interpretations.
For interpretive qualitative approaches, methodological rigor may require attention to reflexivity, context, and the construction of meaning rather than only numerical agreement.
The defense should demonstrate that the final analysis is grounded in the research material and compatible with its epistemological assumptions.
Can You Show That AI Did Not Introduce Unsupported Claims?
AI-assisted manuscript drafting can introduce statements that exceed the evidence.
Reviewers may question whether literature claims are supported, whether interpretations are appropriately qualified, or whether the conclusions overstate causality.
Researchers should verify consequential claims against original publications and validated results.
A citation's existence does not establish that it supports the statement attributed to it.
Authors should also be prepared to correct misleading language, unsupported generalizations, or inaccurate descriptions introduced during AI-assisted drafting.
Can You Explain the Reproducibility of the AI-Assisted Procedure?
Reproducibility questions become important when AI outputs determine research measurements or analytical results.
Reviewers may ask whether the model version was fixed, whether prompts and settings were preserved, and whether another researcher could reconstruct the procedure.
Authors should distinguish preservation of the original analytical record from the ability to generate identical outputs in a future run.
Model updates and stochastic generation may limit exact repetition.
Where reproducibility is essential, researchers should explain how model variability was controlled or evaluated.
For some studies, archived outputs and independent validation may support methodological scrutiny even when the original commercial model is no longer available.
Can You Demonstrate That AI Use Was Authorized?
Reviewers may raise concerns about confidential participant information or other restricted research material.
Authors should be able to establish that the AI processing was consistent with relevant consent, ethics approval, institutional policy, contractual obligations, and applicable law.
However, confidential documentation should not be disclosed indiscriminately during peer review.
Where sensitive records are relevant to a reviewer's question, authors should coordinate with the journal and responsible institution to determine an appropriate protected verification arrangement.
Transparency does not eliminate confidentiality obligations.
What Documentation Should Be Available During Peer Review?
Documentation should reflect the significance of AI's role.
| Reviewer concern |
Potential supporting evidence |
What the evidence should establish |
| Model identification |
Provider information, model identifier, version records, dates |
Which system was used and when |
| Prompt-dependent procedure |
Relevant prompts, settings, and output records |
How the analytical operation was instructed |
| Statistical validity |
Code, data-processing records, diagnostics, independent calculations |
Whether the analysis was correctly implemented and appropriate |
| Qualitative rigor |
Coding framework, source-data review records, analytical memos |
How interpretations were developed and evaluated |
| Output accuracy |
Original sources, reference data, verification logs |
Whether consequential claims and results are supported |
| Reproducibility |
Archived outputs, configuration records, scripts, validation procedures |
Whether the procedure can be understood and adequately reconstructed |
| Ethical authorization |
Relevant approvals or institutional confirmation, subject to confidentiality |
Whether the proposed processing was permitted |
Not every reviewer will require every record. Authors should retain sufficient evidence to respond to reasonable methodological questions without unnecessarily exposing protected information.
How Should Researchers Respond to a Reviewer Who Questions AI Reliability?
A useful response identifies the specific concern and addresses it with evidence.
If a reviewer questions AI-generated classifications, the authors may describe the reference dataset, validation design, relevant performance measures, and observed limitations.
If the concern involves statistical code, the response may explain independent testing or provide corrected analytical outputs.
General statements that AI is widely used or that the model is highly capable do not resolve task-specific validity concerns.
Authors should distinguish evidence supporting the original analysis from additional checks performed in response to the review.
Where new verification changes the results, the manuscript and response should accurately describe those changes.
What if the Reviewer Is Incorrect About AI?
Reviewers may misunderstand the AI system's role, assume that all AI-assisted methods are inherently invalid, or request information irrelevant to the research question.
Authors may respectfully clarify the procedure and explain why a particular concern does not apply.
For example, a reviewer might assume that a generative model performed the statistical calculations when AI was used only to draft code that was subsequently executed and independently checked.
The author can distinguish code generation from computational execution and provide the relevant validation evidence.
Disagreement should be resolved through methodological reasoning rather than appeals to either technological enthusiasm or categorical rejection.
What if Peer Review Reveals an AI-Related Error?
A reviewer may identify an incorrect transformation, unsupported citation, or misleading interpretation that escaped the authors' original checks.
The research team should investigate the concern using the original records and appropriate verification procedures.
If an error is confirmed, the authors should correct the affected analysis or text and assess whether other findings depend on the same problem.
Where the error materially changes the conclusions, the response should explain the consequences transparently.
Authors should not defend an incorrect result merely because it was produced by a reputable AI system or previously accepted by the research team.
When Is Additional Validation Necessary?
Additional validation may be warranted when the reviewer identifies an important gap in the original evidence supporting AI use.
For example, a classifier evaluated only through overall accuracy may require further examination of class-specific errors when certain misclassifications are scientifically consequential.
A qualitative analysis may require clearer documentation of how generated categories were reviewed against original data.
However, additional validation should be methodologically justified. Researchers should avoid conducting arbitrary analyses solely to obtain a favorable result.
Any new procedures should be reported accurately, including their timing and whether they were conducted after the reviewer's request.
What if the Researcher Cannot Defend a Consequential AI Contribution?
When an AI-assisted procedure cannot be adequately explained or validated, the researcher should reconsider whether the resulting evidence can support the manuscript.
Possible responses include repeating the analysis using a documented procedure, obtaining qualified methodological assistance, revising the claims, or removing unsupported results.
If the limitation cannot be resolved, authors should not present the work as fully validated.
In some circumstances, the appropriate decision may be to withdraw or substantially revise the manuscript rather than continue defending an unsupported analysis.
The objective of peer review is to improve the reliability and clarity of the scientific record, not to preserve every original methodological choice.
How Should Researchers Prepare Before Submission?
Preparation should begin before reviewers raise questions.
Researchers can examine the manuscript as though a skeptical but technically competent reviewer were evaluating every consequential AI contribution.
For each contribution, they should establish the rationale, documentation, verification evidence, methodological limitations, and reporting requirements.
Where the procedure cannot be defended, the issue should be addressed before submission.
This preparation may reduce avoidable revisions and reveal weaknesses that would otherwise remain hidden until peer review.
