Can a Randomized Controlled Trial Answer More Than One Primary Research Question?

Can One Randomized Trial Have Several Primary Research Questions?

You are designing a randomized controlled trial to evaluate an AI-assisted learning intervention. You want to know whether it improves academic performance, reduces instructor workload, and increases student engagement. All three outcomes seem important. Can they all be primary research questions?

Alternatively, suppose your trial compares two experimental interventions with conventional instruction. You may want to establish whether each intervention is effective and whether one performs better than the other.

Both situations involve more than one question, but they create different statistical problems. The challenge is not simply whether multiple primary questions are permitted. It is whether the trial can produce sufficiently reliable evidence for the claims the researchers intend to make.

Yes, but Each Primary Question Must Be Explicitly Justified

How Multiple Primary Questions Change Trial Design and Interpretation

What Makes a Research Question Primary?

A primary research question identifies a central claim that the trial is specifically designed to answer. It normally corresponds to a prespecified treatment comparison, outcome, target population, and summary measure of the treatment effect.

For example, a trial might primarily ask whether an AI-assisted feedback system improves students' writing scores compared with conventional instructor feedback after one semester.

The associated primary analysis should be sufficiently important to influence the trial's design and sample-size justification.

A question does not become primary merely because the researchers find it interesting or because its outcome is measured. The designation has consequences for how evidence will be interpreted and which conclusions the trial can support.

The primary questions should therefore be established before researchers examine the outcome data, ideally within a registered protocol and statistical analysis plan.

What Is the Difference Between Multiple Primary Questions and Multiple Primary Outcomes?

These concepts overlap but are not interchangeable.

A trial may have several primary questions involving different outcomes. For instance, investigators might ask whether an intervention improves learning achievement and whether it reduces instructor workload.

Alternatively, several primary questions may concern the same outcome but involve different treatment comparisons, such as intervention A versus control and intervention B versus control.

Researchers should distinguish the outcome being measured from the hypothesis being tested.

The distinction matters because each structure may require a different strategy for controlling statistical error and defining trial success.

When Are Multiple Primary Questions Scientifically Justified?

Multiple primary questions may be appropriate when a trial's central objective cannot be adequately represented by one outcome or treatment comparison.

Consider an intervention intended to improve student learning without increasing instructor workload. If both objectives are essential to the intervention's practical value, a trial may reasonably incorporate both into its primary evaluation.

However, the investigators must clarify whether success requires both outcomes to meet their criteria or whether either outcome could independently support a positive conclusion.

Those two decision rules have substantially different statistical implications.

Another justification arises when a multi-arm trial evaluates several interventions against a shared control. Each comparison may address a distinct primary objective, provided the trial has been designed to support those claims.

The relevant question is not how many primary hypotheses are allowed. It is whether each is essential, interpretable, and supported by the design.

What Are Co-Primary Endpoints?

Co-primary endpoints are two or more endpoints jointly designated as necessary to establish a specified treatment claim. Under an all-must-succeed decision rule, the intervention is considered successful only if the prespecified criteria are met for every co-primary endpoint.

For example, a digital learning intervention might be required to improve academic performance and reduce assessment time. A favorable result on academic performance alone would not satisfy the joint success criterion.

This differs from a multiple-primary-endpoint arrangement in which success on any one of several endpoints could support a claim.

For a properly specified intersection-union testing framework in which every component null hypothesis must be rejected, testing each component at the chosen significance level can control the Type I error for the joint claim without a conventional multiplicity correction. However, achieving adequate power to demonstrate success on all components can require a larger sample.

Researchers should not assume that labeling outcomes co-primary automatically determines the statistical procedure. The joint decision rule and corresponding hypotheses must be explicitly defined.

Why Does Testing Several Primary Hypotheses Increase False-Positive Risk?

When researchers conduct several statistical tests and treat a positive result on any one of them as sufficient for a confirmatory claim, the probability of at least one false rejection may exceed the nominal significance level.

This probability is commonly described through the familywise error rate.

For m independent tests, each conducted at significance level α, the probability of at least one false-positive result when all null hypotheses are true is:

In real trials, outcomes and treatment comparisons are often correlated. The formula therefore should not be applied mechanically when the independence assumption is inappropriate.

Moreover, multiplicity requirements depend on the inferential claims. A joint all-must-succeed objective does not create the same error-control problem as an any-one-succeeds objective.

What Methods Can Address Multiplicity?

Several established procedures can control error rates across a defined family of primary hypotheses.

The appropriate method depends on the hypothesis structure, correlations, intended claims, and whether the trial is confirmatory or exploratory.

Not every secondary or exploratory analysis necessarily belongs to the same multiplicity family. Researchers should specify which claims require confirmatory error control rather than automatically applying one adjustment to every analysis conducted.

How Do Multiple Primary Questions Affect Statistical Power?

Statistical power should reflect the trial's actual success criteria.

If success requires detecting effects on both learning achievement and instructor workload, the study needs adequate joint power for both endpoints. A sample size that provides 80% power for each outcome separately does not necessarily provide 80% power for simultaneous success.

If two tests were independent and each had 80% power under a specified alternative, the probability of both succeeding would be 0.80 × 0.80 = 0.64, or 64%.

This calculation is illustrative. Actual joint power depends on the correlation between endpoints, treatment effects, testing procedures, and statistical assumptions.

When either of several endpoints can support a positive claim, multiplicity-adjusted thresholds may reduce the power of individual tests unless the sample size is increased.

Consequently, sample-size planning should evaluate the full primary hypothesis structure rather than calculating power for one outcome and assuming the result applies to every other question.

Can One Primary Outcome Support Several Primary Comparisons?

Yes. A multi-arm trial may use one primary outcome while comparing several experimental interventions with a shared control.

For example, a three-arm study might evaluate AI-only feedback, instructor-mediated AI feedback, and conventional feedback using the same writing-performance outcome.

The trial could specify two primary treatment contrasts: AI-only versus conventional feedback and instructor-mediated AI versus conventional feedback.

These contrasts share the same control observations and are statistically correlated. A suitable multiplicity strategy may be needed if the trial intends to make confirmatory claims about either intervention.

This is one reason the decision about using a two-arm or multi-arm trial should be made alongside the primary hypothesis plan.

How Are Primary, Secondary, and Exploratory Questions Different?

Primary questions determine the central evidential claims the trial is designed to support. Secondary questions address additional outcomes or effects that are important but not necessarily central to the principal trial objective.

Exploratory questions investigate patterns or hypotheses that may inform subsequent research, particularly where the available evidence is insufficient for a definitive claim.

These labels should reflect the design and analysis plan rather than the statistical significance of the observed results.

A secondary outcome does not become primary because its p-value is small. Similarly, an unfavorable primary result should not be replaced retrospectively by a favorable exploratory result and presented as though it were the original objective.

Transparent reporting should identify prespecified outcomes and disclose changes, including their timing and justification.

What Is the Role of the Estimand?

An estimand describes the treatment effect the trial intends to estimate. Under the ICH E9(R1) framework, relevant attributes include the treatment conditions, population, outcome variable, handling of intercurrent events, and population-level summary measure.

Multiple primary questions may correspond to different estimands, even when they concern the same intervention.

For example, a trial could ask about the effect of assignment to an AI feedback program regardless of subsequent adherence, while another question concerns a treatment effect under a specified strategy for handling discontinuation.

These questions require careful specification because they do not necessarily represent the same causal effect.

Researchers should distinguish multiple estimands from multiple outcomes and avoid adding primary questions without clarifying the treatment effects each is intended to represent.

When Should You Avoid Multiple Primary Questions?

Multiple primary questions may be inappropriate when the available sample cannot support their required precision, the hypotheses lack a coherent scientific rationale, or the trial's main decision becomes unclear.

They may also be problematic when investigators cannot specify how conflicting outcomes will be interpreted.

Suppose an intervention improves achievement but increases instructor workload. If both outcomes were designated primary, the protocol should explain whether that pattern represents partial success, failure of a joint objective, or separate conclusions.

These decisions should not be improvised after results are available.

Where one outcome clearly represents the principal objective, designating it primary and treating other outcomes as secondary may produce a more interpretable trial.

Testing Learning Achievement and Instructor Workload

Suppose the university would adopt the intervention only if it improved learning without an unacceptable increase in workload. That objective is not identical to requiring statistically significant improvement in both outcomes. A more appropriate design might test superiority for learning and a prespecified non-inferiority or acceptability criterion for workload.

This distinction illustrates why primary questions must be translated into precise statistical claims. The appropriate hypotheses depend on what constitutes meaningful success, not merely on how many outcomes researchers wish to report.

The resulting protocol should define the primary decision rule before data collection and use a sample-size calculation that reflects the selected strategy.

Common Mistakes With Multiple Primary Research Questions

How to Decide Whether Your Trial Needs Multiple Primary Questions

Begin by identifying the decision that the trial is intended to inform. Determine whether one treatment effect is sufficient or whether several conditions must be demonstrated before the intervention can be considered successful.

Then define each proposed primary question as a specific treatment contrast and outcome, with an explicit interpretation of success.

Consult the trial statistician before finalizing the protocol. The statistical strategy should follow the scientific claims, rather than requiring researchers to redefine the claims after discovering which analysis is easiest.

Finally, ensure that the trial registration, protocol, statistical analysis plan, and final report consistently identify the primary questions and any justified amendments.

Before Including Multiple Primary Questions

Questions About Multiple Primary Hypotheses in Randomized Trials

Multiple Primary Questions Are Possible, but Their Claims Must Be Planned

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.