03 · What You Need to Know
How to Distinguish Reviewer Requests for Clarification, Reanalysis, and New Research
Reviewer Comments Often Mix Reporting and Analytical Concerns
A reviewer may identify a legitimate concern without specifying exactly how it should be addressed.
For example, "Please explain whether missing observations could have influenced the findings" may invite clarification of an existing procedure, a sensitivity analysis, or a more cautious interpretation.
The appropriate response depends on what the authors originally did, what information is available, and how consequential the missing data may be.
Generative AI can help separate the different possibilities, but it should not automatically select the most elaborate response.
The purpose is to address the scientific concern adequately, not to maximize the number of new analyses.
What Types of Reviewer Comments May Require New Analyses?
Reviewer requests can be organized according to the nature of the work they imply.
| Reviewer Request |
Possible Classification |
What Authors Must Determine |
| "Clarify how missing data were handled." |
Reporting clarification |
Whether the existing procedure was adequate and sufficiently described |
| "Recalculate the estimates using cluster-robust standard errors." |
Potential reanalysis |
Whether the dependence structure and proposed method justify the change |
| "Provide confidence intervals for the reported effects." |
Additional calculation or reporting |
Whether intervals already exist or can be computed from the original analysis |
| "Examine whether the findings differ by student year level." |
Potential subgroup analysis |
Whether the sample and design support meaningful subgroup inference |
| "Explain how the qualitative themes were developed." |
Analytical reporting clarification |
Whether the existing analysis is adequately documented |
| "Reconsider the coding framework in light of contradictory cases." |
Potential qualitative reanalysis |
Whether the concern requires revisiting the data and interpretations |
| "Collect additional data from another institution." |
New data collection |
Whether the request is feasible, ethical, and necessary for the study's claims |
These classifications are provisional. The same wording may imply different work depending on the manuscript's existing analyses and the editor's instructions.
Clarification Is Not the Same as Reanalysis
A reviewer may request information that the authors already possess but did not report clearly.
For example, the manuscript may state that regression analysis was performed without describing how categorical predictors were coded.
The authors may be able to address the concern by expanding the Methods section and reporting the existing coding procedure.
No new model is necessarily required.
However, if the clarification reveals that the original analysis was inappropriate, reanalysis may become necessary.
The decision should follow the evidence rather than an AI-generated label.
Recalculation Is Different From a New Analytical Question
Some reviewer requests involve recomputing existing results under corrected assumptions or improved specifications.
For example, adjusting standard errors for clustering may change uncertainty estimates without changing the underlying research question.
Other requests introduce new questions, such as whether an intervention works differently across demographic groups.
These tasks have different methodological implications.
Recalculation may be necessary to correct an inferential problem. A new subgroup analysis may be exploratory and require additional caution concerning sample size, multiplicity, and interpretation.
AI should not treat both as equivalent forms of routine revision.
Can AI Identify Requests for Sensitivity Analysis?
AI may recognize terms such as robustness, sensitivity, alternative specification, or missing-data assumptions.
However, these terms do not specify a universally appropriate procedure.
A sensitivity analysis should examine how conclusions change under defensible alternative assumptions or analytical choices.
For example, authors might assess whether results remain similar under different missing-data assumptions or alternative operational definitions.
The relevant alternatives must be scientifically justified rather than selected merely because they produce favorable results.
Authors should determine which uncertainty or assumption the reviewer is asking them to examine before choosing an analysis.
New Analyses May Change the Scientific Conclusions
Additional analyses are not simply administrative tasks. They may change estimates, uncertainty intervals, interpretations, or the strength of the study's conclusions.
For example, accounting for classroom-level clustering may widen confidence intervals and alter how precisely an intervention effect is estimated.
A subgroup analysis may reveal heterogeneity, but it may also produce unstable estimates when subgroup sizes are small.
Authors should therefore update the relevant results, discussion, limitations, and abstract when justified analyses materially change the findings.
AI may help identify these downstream revision dependencies, but it should not determine the interpretation without independent verification.
When Is a Reviewer-Requested Analysis Inappropriate?
A reviewer may recommend a procedure that does not fit the research question, data structure, or methodological framework.
For example, requesting statistical power calculations for an interpretive qualitative study may be inappropriate.
Similarly, demanding a subgroup analysis may be problematic when the study lacks sufficient observations to estimate meaningful differences.
A recommendation to control for additional variables may also be inappropriate when those variables lie on the causal pathway or introduce other forms of bias.
Authors should evaluate the methodological justification rather than assume that every requested analysis will improve the research.
Can Authors Respectfully Decline a New Analysis?
Yes, provided the decision is scientifically defensible and consistent with the editor's requirements.
A reviewer may request an analysis that cannot be conducted because the necessary data were not collected or the procedure would not answer the study's research question.
Authors should explain the limitation or methodological rationale clearly.
For example, they might explain that the proposed subgroup comparison was not prespecified and that the available sample would produce imprecise estimates.
They may also revise the discussion to acknowledge the unanswered question without presenting unsupported new results.
The goal is to address the underlying concern rather than reject the recommendation dismissively.
Why AI Should Not Automatically Recommend More Statistical Tests
Generative AI may interpret a major revision as a reason to expand the analysis extensively.
However, additional tests can introduce multiplicity concerns, unstable estimates, and opportunities for selective reporting.
Exploratory analyses conducted after reviewer feedback may be useful, but they should be identified accurately and interpreted with appropriate caution.
Authors should avoid repeatedly testing alternative specifications until a desired result appears.
The distinction between confirmatory and exploratory analysis remains relevant even when additional work is requested during peer review.
Qualitative Reviewer Comments May Also Require Reanalysis
New analysis is not limited to statistics.
A reviewer may question whether a qualitative theme is adequately supported, whether contradictory cases were considered, or whether the interpretation reflects the study's theoretical orientation.
Addressing such concerns may require returning to transcripts, revisiting coding decisions, examining negative cases, or reconsidering theme development.
However, the appropriate response depends on the qualitative methodology.
For example, reflexive thematic analysis should not automatically be evaluated using coding-reliability procedures that conflict with its analytical assumptions.
AI-generated recommendations must therefore be assessed in relation to the study's methodological tradition.
New Data Collection Is Not the Same as New Analysis
Some reviewer comments request additional participants, experiments, interviews, or observations.
These requests extend beyond reanalyzing existing data.
New data collection may require additional ethics approval, participant recruitment, funding, or substantial changes to the study design.
It may also be impossible within the journal's revision period.
Authors should distinguish a request to analyze existing information differently from a request to generate new evidence.
If the requested work is infeasible or would constitute a substantially different study, a reasoned explanation and appropriately qualified conclusions may be more defensible.
How Should Authors Decide Whether an Analysis Is Feasible?
Feasibility should be evaluated before committing to the analysis in the response letter.
Relevant considerations include whether the required variables exist, whether the data are sufficiently complete, whether the analytical assumptions are defensible, and whether the research team has the necessary expertise.
For qualitative work, feasibility may concern access to the original transcripts, the adequacy of the data for the proposed interpretation, and consistency with the methodological approach.
Authors should also consider the journal's revision deadline and any ethical or institutional restrictions.
AI may help organize these considerations, but it cannot establish feasibility without accurate information about the actual research record.
Why Authors Should Document Analytical Changes
When additional analyses are conducted, the authors should preserve the relevant code, data-processing decisions, model specifications, and resulting outputs.
They should distinguish original analyses from those added during revision and explain whether the new work was confirmatory, exploratory, or conducted as a robustness check.
Where the results change, the manuscript and response letter should accurately describe those differences.
This documentation supports reproducibility and reduces the risk that an AI-assisted response exaggerates the extent or implications of the additional work.
AI Can Help Connect Analysis Requests to the Revision Plan
Once the authors have evaluated the scientific appropriateness of the requested work, AI may help organize the tasks and their dependencies.
For example, a new analysis may require updating a results table, revising the discussion, reconsidering the abstract, and preparing a response to the reviewer.
These related tasks can be tracked in a structured reviewer-comment revision action plan.
However, the plan should distinguish work that has been approved, completed, or rejected with justification.
AI-generated task lists should not be treated as evidence that analyses have been conducted.
Confidentiality and Disclosure Still Apply
Reviewer reports and manuscript materials may contain unpublished research information or confidential editorial content.
Authors should consult the journal's AI-use policies and the provider's data-handling arrangements before submitting such material to an external AI service.
Where AI assistance is permitted, the authors should use only authorized information and processing arrangements.
Disclosure requirements depend on the journal's current policy and the nature of the assistance.
The authors remain responsible for the scientific decisions, completed analyses, and accuracy of the final response letter.
