Can You Include Exploratory Analyses in a Registered Report?

Can You Investigate Unexpected Patterns After a Registered Report Is Accepted?

You have received in-principle acceptance for a Registered Report evaluating whether an educational intervention improves student achievement. The approved analysis plan specifies the primary outcome, statistical model, and hypotheses. After completing the study, you discover an unexpected relationship between students' engagement patterns and their achievement scores.

Can you investigate that relationship and include the findings in your Stage 2 manuscript? Would doing so violate the approved protocol, or must every analysis appearing in a Registered Report have been specified before data collection?

The concern is understandable. Registered Reports emphasize prospective methodological commitments, but research also depends on the capacity to recognize unexpected findings. The important distinction is not whether additional analyses are permitted, but how their origins, evidential status, and relationship to the approved research plan are communicated.

Yes, Exploratory Analyses Can Be Included Alongside Approved Analyses

How Exploratory Analyses Fit Within the Registered Report Format

Why Do Registered Reports Permit Exploratory Analyses?

Registered Reports are designed to strengthen prospective evaluation of research questions and methods, particularly where undisclosed analytical flexibility could distort confirmatory inference.

They are not intended to prevent researchers from noticing unexpected patterns or developing new explanations.

Scientific investigations frequently generate questions that were not anticipated when the study was designed. A surprising association, an unusual distribution, or an unexpected difference across participant groups may justify additional examination.

Preventing researchers from investigating such findings would unnecessarily restrict scientific discovery. The Registered Report format instead seeks to preserve the distinction between analyses planned before relevant results were available and investigations developed subsequently.

Under the conventional model, the approved Stage 1 protocol establishes the prospective research commitments. Additional exploratory work may be reported at Stage 2 when appropriately identified and consistent with the journal's policies.

What Is the Difference Between Confirmatory and Exploratory Analyses?

Confirmatory analyses evaluate hypotheses or predictions specified before researchers have access to the relevant evidence. Exploratory analyses investigate patterns, relationships, or questions without the same degree of prospective specification.

The distinction concerns how the analysis was developed and what inferential claims it can support. It does not establish that one type of research is inherently more valuable than the other.

An exploratory analysis can be statistically sophisticated and methodologically careful. Conversely, a preregistered confirmatory analysis can still be poorly designed or based on invalid assumptions.

Prospective specification improves transparency about the origin of decisions. It does not independently establish their scientific validity.

Are All Analyses Not Included in Stage 1 Automatically Exploratory?

Not necessarily. Analyses omitted from the original protocol can have different purposes and implications.

An additional analysis might investigate an unexpected substantive question, correct a technical error, evaluate the robustness of an approved result, or address an assumption that could not be assessed prospectively.

These situations should not automatically receive identical labels.

For example, adding an exploratory subgroup analysis is different from replacing the approved primary regression model because another model produces a smaller p-value.

The latter changes the original confirmatory evaluation and requires disclosure as a departure from the approved plan.

The principles for distinguishing fixed commitments from changeable research decisions remain relevant.

Can Exploratory Analyses Be Planned Before Data Collection?

Yes. Exploratory does not necessarily mean unplanned.

A researcher may specify in advance that the study will include descriptive visualizations, exploratory associations among secondary variables, or investigations of potential patterns without defining confirmatory hypotheses for those analyses.

Such investigations remain exploratory because their purpose is discovery rather than a fully specified prospective hypothesis test.

For example, a Registered Report might include an approved primary comparison of two instructional interventions while also planning an exploratory examination of relationships among learning engagement, feedback frequency, and student confidence.

The Stage 1 manuscript can describe the exploratory objectives and intended methods without claiming that all possible patterns are known.

Researchers should distinguish planned exploratory analyses from analyses developed after the results are examined. Both may be legitimate, but they have different methodological histories.

What Types of Exploratory Analyses Might Be Appropriate?

The possibilities depend on the research question, available data, and methodological design.

Common examples include:

• Investigating unexpected relationships among measured variables.
• Examining potential subgroup differences that were not specified at Stage 1.
• Exploring nonlinear relationships or alternative functional forms.
• Investigating possible mechanisms suggested by the observed findings.
• Examining descriptive patterns in secondary or contextual variables.
• Developing new hypotheses for independent evaluation.

These investigations can contribute useful knowledge when the questions are scientifically meaningful and the analyses are appropriately reported.

However, the availability of many possible analyses creates opportunities for selective reporting. Researchers should avoid treating whichever pattern appears most statistically favorable as though it were the only question considered.

Can You Conduct Exploratory Subgroup Analyses?

Yes, but subgroup findings require particular caution.

Suppose an intervention has a modest overall effect, but the observed difference appears larger among students with low baseline achievement.

Researchers may investigate whether the intervention effect varies by baseline achievement, even if that question was not included in the approved Stage 1 protocol.

However, the analysis should be identified as exploratory. The researchers should explain how the subgroup was defined and whether alternative subgroup definitions or comparisons were examined.

When the question concerns differences between subgroup-specific effects, an appropriate interaction test or direct comparison is generally preferable to interpreting separate significance tests.

For instance, statistical significance in one subgroup but not another does not establish that the two effects differ.

The methodological issues surrounding planned and exploratory subgroup analyses include statistical power, multiplicity, and the interpretation of effect modification.

How Does Multiple Testing Affect Exploratory Findings?

Exploratory analyses may involve many statistical comparisons. Even when no genuine relationship exists, examining numerous variables, models, or subgroups can produce apparently noteworthy results by chance.

For illustration, suppose 20 independent tests are conducted under true null hypotheses, each using an unadjusted significance threshold of 0.05.

This calculation illustrates why an apparently significant result may be less convincing when selected from many comparisons. It does not mean that every exploratory analysis has a 64.2% probability of being wrong.

Real analyses often involve correlated tests, mixed true and false hypotheses, and more complicated selection processes. The calculation therefore cannot be applied mechanically to every study.

Researchers may use appropriate multiplicity adjustments, estimation-focused reporting, or other strategies depending on their objectives. Transparency about the breadth of exploration remains important even when formal adjustment is not the principal inferential approach.

Can Exploratory Analyses Change the Interpretation of the Primary Findings?

They may provide useful context, but they should not silently redefine the original confirmatory result.

Suppose the approved primary analysis finds no convincing evidence of an overall intervention effect. An exploratory analysis then suggests that students with limited prior knowledge may benefit more than other students.

The researchers may discuss this pattern as a possible explanation for heterogeneous outcomes. However, they should not conclude that the original primary hypothesis was confirmed simply because an unplanned subgroup comparison appears favorable.

Exploratory findings can complicate, qualify, or motivate further examination of the primary result. Their evidential status should remain clear.

This distinction is particularly important when reporting null or unexpected Registered Report findings.

Should Exploratory Analyses Be Included in the Main Results or a Supplement?

There is no universal requirement that exploratory findings appear only in supplementary materials.

Placement should reflect their substantive relevance, methodological importance, and the target journal's reporting instructions.

An exploratory analysis that provides meaningful context for the main findings may deserve discussion in the main manuscript. A large collection of secondary investigations may be more clearly presented in supplementary materials.

Regardless of location, readers should be able to distinguish the approved confirmatory analyses from additional investigations.

Section headings such as "Exploratory Analyses" or explicit descriptions of the analysis status can help establish that distinction.

Researchers should not use supplementary placement to conceal unfavorable exploratory results while highlighting only favorable ones in the main text.

How Much Detail Should Be Reported About Exploratory Analyses?

Enough for readers to understand the question investigated, the analytical decisions made, and the limitations of the resulting evidence.

Useful reporting information includes the motivation for the analysis, variables examined, model specification, subgroup definitions, relevant data exclusions, and whether the analysis was developed before or after inspecting the results.

When feasible, researchers should also disclose the broader analytical search that led to the reported finding.

For example, if an interesting interaction emerged after examining 15 candidate moderators, reporting that context is more informative than presenting the selected interaction as an isolated test.

Analysis code and sufficiently detailed methodological documentation may improve reproducibility, subject to legitimate data-access restrictions.

The objective is not to document every inconsequential exploratory action, but to make consequential analytical choices understandable.

Can You Change an Approved Analysis and Call the Replacement Exploratory?

You can conduct an alternative exploratory analysis, but that does not erase the obligation to account for the approved confirmatory procedure.

Suppose the Stage 1 protocol specifies a linear regression model, but the researchers later prefer a different model after observing the results.

They may report the alternative model as exploratory or as a justified sensitivity analysis, depending on its purpose. However, the approved analysis should generally still be reported when meaningful and feasible.

If the original procedure is invalid or cannot reasonably be implemented, the researchers should document the problem and follow the journal's procedures for amendments.

Consequential design and analysis changes after Stage 1 acceptance may require editorial consultation or approval.

Can Exploratory Findings Become Confirmatory Evidence?

An exploratory finding can motivate a new confirmatory investigation, but the same evidence should not automatically be treated as an independent test of a hypothesis developed from that evidence.

For example, researchers may discover that a digital intervention appears especially beneficial for students with low baseline achievement.

They could formulate a precise hypothesis concerning effect modification and preregister a subsequent study designed to evaluate it.

Independent data, appropriate sample-size planning, and a prospective analysis plan can strengthen the evidential basis for the new hypothesis.

In some circumstances, a genuinely held-out dataset may support independent evaluation within a larger research program. The validity of that approach depends on how the data were separated and whether the held-out observations influenced model or hypothesis development.

When independent confirmation is unavailable, researchers should acknowledge that the findings remain exploratory rather than claiming that replication has occurred.

How Do Exploratory Analyses Work in Qualitative and Mixed-Methods Registered Reports?

Not every Registered Report uses the same distinction between fixed statistical tests and exploratory analyses.

Qualitative studies may involve iterative sampling, evolving interview questions, or interpretive theme development as part of the approved methodology.

Such developments should not automatically be classified as unplanned exploratory deviations merely because they were not specified as final outcomes at Stage 1.

For example, reflexive thematic analysis ordinarily involves developing themes through engagement with qualitative material. Predicting the final themes before analysis would be inconsistent with that approach.

Mixed-methods research may also involve planned dependencies between phases, such as selecting interview participants based on earlier quantitative findings.

Researchers should explain which developments are anticipated within the approved methodological framework and which constitute substantive departures requiring additional documentation.

The considerations for qualitative and mixed-methods Registered Reports therefore require attention to methodological coherence rather than mechanical application of quantitative reporting conventions.

What Will Stage 2 Reviewers Expect?

Stage 2 reviewers ordinarily assess whether the approved study was implemented appropriately and whether the completed manuscript reports and interprets the evidence accurately.

Exploratory analyses may be evaluated for methodological appropriateness, transparency, and whether the conclusions exceed the available evidence.

Reviewers may request clarification of analytical choices, additional sensitivity analyses, or more cautious interpretation.

However, exploratory findings should not become a reason to disregard the approved primary analysis or to make publication depend on the attractiveness of the results.

Journal policies differ in the permitted scope and presentation of additional analyses. Authors should verify the relevant instructions before preparing the Stage 2 manuscript.

Adding an Exploratory Analysis to an Educational Registered Report

The exploratory analysis adds a potentially useful research question without changing the original confirmatory evaluation.

Importantly, the observed association between feature use and achievement does not establish that requesting alternative explanations caused higher achievement. Students who use the feature frequently may differ systematically from those who do not.

The Registered Report can therefore accommodate the discovery while preserving the distinction between the randomized intervention effect and the observational association identified through exploration.

Common Misunderstandings About Exploratory Analyses in Registered Reports

How to Include Exploratory Analyses Without Compromising the Registered Report

Begin by preserving the distinction between the analyses approved at Stage 1 and the additional investigations conducted afterward.

The approved analyses should remain identifiable in the completed manuscript. Exploratory findings can then be presented according to their substantive relevance and methodological strength.

When reporting exploratory analyses, focus on effect estimates, uncertainty, plausible alternative explanations, and the consequences of analytical selection.

Where possible, maintain an analysis log documenting the questions investigated and major modeling decisions. This can help readers understand the breadth of exploration without requiring the final article to reproduce every preliminary calculation.

Before submission, consult the journal's Stage 2 instructions. Some journals may specify how additional analyses should be labeled, separated, or presented in supplementary materials.

Before Adding Exploratory Analyses to a Registered Report

Frequently Asked Questions About Exploratory Registered Report Analyses

Exploration Is Compatible With Registered Reports When Its Status Is Transparent

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.