Can a Research Study Have Multiple Primary Outcomes?

Can More Than One Outcome Be Equally Important to Your Study?

You are developing an intervention study intended to improve students' academic performance while preserving their well-being. Both outcomes matter to the decision you hope the research will inform.

Improved examination scores may be insufficient evidence of success if the intervention also creates unacceptable psychological strain. Conversely, an intervention that improves well-being without improving academic performance may not achieve its principal instructional objective.

Should academic performance and well-being both be designated primary outcomes? Would that make the study methodologically weaker than one with a single primary outcome?

The same problem arises in clinical research, where efficacy and safety may both be important, and in organizational studies, where an intervention may seek to improve productivity without worsening employee outcomes.

The difficulty is not simply deciding how many outcomes to measure. Researchers must establish what evidence is required to support the main conclusion and how results across several outcomes will be evaluated together.

Yes, Multiple Primary Outcomes Are Appropriate When the Scientific Objective Requires Them

Designing Research With More Than One Primary Outcome

What Does Having Multiple Primary Outcomes Mean?

Multiple primary outcomes are two or more prespecified results that contribute directly to evaluating the study's main objectives.

For example, a study of an educational intervention might designate both academic achievement and independent problem-solving performance as primary outcomes because both are necessary to establish the intended educational benefit.

Another study might evaluate academic performance and participant burden as jointly important dimensions of an intervention.

However, the designation does not automatically establish how these outcomes should be interpreted.

Researchers must define whether each outcome represents a separate confirmatory claim, whether the outcomes jointly define success, or whether a prespecified hierarchy determines how evidence is evaluated.

The primary outcome structure should reflect the intended scientific inference rather than the number of measurements available.

Multiple Primary Outcomes Are Not Necessarily Co-Primary Outcomes

The terms multiple primary outcomes and co-primary outcomes are sometimes used interchangeably, but greater precision is useful.

Co-primary outcomes commonly refer to a strategy in which success must be demonstrated on every designated outcome to establish the intended overall claim.

Other multiple-primary-outcome strategies may permit success when at least one specified outcome demonstrates an effect, subject to appropriate multiplicity control.

These approaches answer different questions and should not be treated as equivalent merely because they involve several measurements.

Start With the Scientific Success Criterion

The most important decision is what the study must demonstrate to support its principal conclusion.

Suppose researchers evaluate an instructional intervention intended to improve both conceptual understanding and practical competence.

If the intervention is considered successful only when it improves both, a co-primary strategy may be appropriate.

If either improvement would independently justify a distinct claim, a multiple-endpoint strategy allowing separate claims may be more suitable.

These decisions should be made before data collection and ideally before the sample size calculation.

Researchers should not select the success rule after seeing which outcome produces the most favorable result.

When Are Co-Primary Outcomes Justified?

Co-primary outcomes may be appropriate when the main scientific objective cannot be established without evidence across several essential dimensions.

For example, a study might investigate whether a new instructional approach improves both theoretical understanding and practical performance.

Alternatively, a noninferiority or benefit-burden framework may require demonstrating effectiveness while satisfying a separate prespecified criterion.

However, not every important outcome must be designated co-primary.

Some dimensions may be better represented as secondary outcomes, safety monitoring measures, or components of a broader decision framework.

The decision should follow the intended claim, measurement properties, and consequences of different result combinations.

Why the Joint Success Rule Matters

Consider a study with two co-primary outcomes, A and B.

If the protocol requires success on both, a favorable result for A alone does not satisfy the overall criterion.

This is an intersection-union testing structure when the global claim requires rejection of each component null hypothesis.

Under an appropriate formulation, testing each component at the specified significance level can control the type I error for the joint claim without the same multiplicity adjustment required by an any-success strategy.

However, the statistical properties depend on the hypotheses, testing procedure, and claims being made.

Researchers should not generalize this observation to mean that every study with multiple outcomes is exempt from multiplicity concerns.

What Happens When Success on Any One Outcome Is Sufficient?

A different situation arises when researchers plan to claim success if at least one of several primary outcomes shows a favorable effect.

Testing each outcome separately at an unadjusted significance level can increase the probability of at least one false-positive claim across the family of hypotheses.

For example, if three independent null hypotheses are all true and each is tested at a significance level of 0.05, the probability of at least one false rejection is approximately 14.3%.

This is an illustrative calculation, not the error rate of every three-outcome study.

Outcome correlations, testing dependencies, and the prespecified decision rule affect the actual error-control requirements.

Which Multiplicity Methods Can Be Used?

Several statistical approaches may be appropriate when multiple primary hypotheses are tested.

Bonferroni adjustment is a straightforward method that allocates the overall significance level across tests.

Holm's procedure can control the familywise error rate while generally being less conservative than simple Bonferroni adjustment.

Prespecified hierarchical or gatekeeping procedures may be appropriate when outcomes have a meaningful order of importance.

Other procedures may account for correlations or more complex hypothesis families.

The appropriate choice depends on the claims, test structure, and statistical assumptions.

Researchers should not automatically apply Bonferroni correction to every set of outcomes, nor should they assume that correlated outcomes eliminate multiplicity concerns.

The U.S. Food and Drug Administration's guidance on multiple endpoints in clinical trials provides detailed examples of these distinctions within its regulatory context.

Multiple Primary Outcomes Affect Sample Size Planning

Sample size calculations should reflect the actual primary success criterion.

If a study requires favorable findings on two co-primary outcomes, the probability of satisfying both criteria may be lower than the power for either outcome individually.

For illustration, suppose each test has 80% power under a specified alternative and their rejection events are independent.

The independence assumption is illustrative. Outcomes measured in the same participants are often correlated, so joint power should be evaluated using the relevant dependence structure.

For an any-success strategy, multiplicity-adjusted thresholds may affect the power of individual and joint claims.

Researchers should therefore avoid calculating sample size for one outcome and assuming that the result is adequate for the entire primary testing strategy.

Outcome Correlation Can Influence Statistical Planning

Primary outcomes may be correlated because they represent related constructs or are measured in the same participants.

For example, conceptual understanding and practical performance may be positively associated.

The degree of correlation can affect joint power and the efficiency of some multivariate testing procedures.

However, correlation does not establish that two outcomes are interchangeable.

Highly correlated measures may still represent distinct educational or clinical dimensions.

Researchers should distinguish the scientific justification for measuring multiple outcomes from the statistical dependence among their observed values.

Multiple Outcomes Should Be Scientifically Distinct Enough to Justify Their Roles

Designating several primary outcomes is difficult to justify when they are merely alternative representations of the same result without a clear reason for separate confirmatory claims.

For example, researchers might propose three nearly identical satisfaction scales as separate primary outcomes.

Unless each captures a distinct essential dimension, the arrangement may increase analytical complexity without improving the scientific question.

Conversely, academic performance and psychological well-being may represent genuinely different dimensions of intervention consequences.

The selection should follow conceptual definitions and stakeholder-relevant decisions.

See what makes an outcome measure appropriate for the measurement considerations underlying this decision.

Why Not Combine Everything Into a Composite Outcome?

A composite outcome can simplify some analyses by combining several components into one defined result.

However, this does not automatically solve the scientific problem of multiple outcomes.

Components may differ in importance, measurement scale, frequency, or responsiveness.

A composite score can improve even when one important component remains unchanged or worsens.

Researchers should justify how components are combined, whether weights are meaningful, and what the resulting score represents.

A composite is appropriate when the combined outcome corresponds to a coherent construct or decision-relevant quantity, not merely because it produces one p-value.

What About Multivariate Outcome Analysis?

Multivariate methods can examine several correlated outcomes jointly.

For example, multivariate analysis of variance may evaluate whether groups differ across a vector of outcomes under its model assumptions.

Other approaches include multivariate regression, global tests, and specialized joint modeling methods.

These analyses may be useful when the scientific question concerns a combined pattern of effects.

However, a significant global test does not automatically establish that every individual outcome improved.

Researchers should distinguish the joint hypothesis being tested from the interpretation of individual outcome effects.

The statistical method must match the prespecified inferential claim.

Multiple Primary Outcomes Require Clear Measurement Timing

Different outcomes may develop or change over different periods.

For example, an educational intervention might influence immediate knowledge acquisition before affecting long-term practical performance.

Researchers may therefore need different assessment times for different primary outcomes.

This can be legitimate when timing follows the scientific objective and anticipated process of change.

However, the time point for each outcome should be specified in advance.

Testing an outcome repeatedly and selecting the most favorable assessment time can introduce additional multiplicity and selective reporting concerns.

Missing Data Can Complicate Joint Outcome Evaluation

When a study has several primary outcomes, participants may provide some measurements but not others.

For example, a student may complete the academic performance assessment but decline a well-being questionnaire.

Consequently, outcome availability can differ across the primary measures.

Researchers should plan how missing information will be handled for each outcome and for the overall success criterion.

Complete-case analysis may unnecessarily discard information in some settings, while more sophisticated methods require explicit assumptions.

Missing-data procedures should align with the estimands and the joint testing strategy.

Statistical methods cannot guarantee that selective missingness has been eliminated.

Prespecification Is Particularly Important

When several outcomes are primary, researchers have more decisions to make about testing order, success criteria, and interpretation.

These decisions should be documented in the protocol and statistical analysis plan before outcome data are examined.

The SPIRIT guidance addresses the specification of outcomes in trial protocols, while CONSORT emphasizes transparent reporting of trial outcomes and analyses.

Applicable registration or regulatory requirements may also govern primary endpoint specification and amendments.

Researchers should document legitimate changes and distinguish them from analyses selected after observing favorable results.

Multiple primary outcomes are methodologically defensible when the inferential framework is transparent, not when the outcome hierarchy is reconstructed after analysis.

Do Multiple Primary Outcomes Apply to Every Research Design?

The terminology is most directly applicable to quantitative studies with defined outcomes, particularly confirmatory intervention research.

Exploratory studies may investigate several outcomes without a single confirmatory hierarchy.

Qualitative studies may instead organize inquiry around central questions, experiences, or processes.

Mixed-methods studies may have primary quantitative outcomes alongside qualitative objectives that do not take the form of endpoints.

Researchers should avoid imposing a clinical-trial endpoint framework on studies for which it is conceptually inappropriate.

The broader distinction is discussed in whether every research study needs one primary outcome.

How Should Conflicting Primary Outcome Results Be Interpreted?

Suppose an intervention improves academic performance but produces no clear improvement in student well-being.

The conclusion depends on the prespecified success criterion.

If both outcomes must demonstrate improvement, the overall co-primary success criterion has not been met.

If the study permits separate claims, the performance finding may support one claim while the well-being result remains inconclusive.

A nonsignificant result does not automatically establish equivalence or absence of harm.

Researchers should examine effect estimates, uncertainty intervals, practical importance, and the specific hypotheses tested.

They should avoid converting mixed findings into an unqualified declaration of success or failure.

Use a Primary Outcome Decision Matrix

No strategy is universally superior. The appropriate choice depends on what the research is intended to establish.

Evaluating an Intervention With Two Essential Educational Outcomes

Specifying the Primary Outcome Strategy

The team chooses a co-primary strategy requiring both outcomes to satisfy their prespecified statistical criteria.

Suppose the hypothetical results are as follows.

These values are hypothetical and are provided solely to illustrate interpretation.

Under the stated co-primary success rule, the study does not meet its overall confirmatory success criterion because the second outcome does not satisfy its required test.

However, the findings do not establish that the intervention has no effect on practical problem-solving. The confidence interval includes both negligible and potentially meaningful benefits.

Interpreting the Evidence

If the team had instead chosen a strategy allowing separate primary claims, the testing and interpretation would require a different multiplicity framework.

The example demonstrates why the outcome strategy must be determined by the scientific objective rather than selected after observing the results.

Common Misconceptions About Multiple Primary Outcomes

Let the Main Scientific Claim Determine the Outcome Strategy

Begin by asking what evidence would be sufficient to support the study's principal conclusion.

If one outcome adequately represents that conclusion, a single primary outcome may be appropriate.

If several dimensions are indispensable, define how their results must be combined or evaluated.

Discuss the strategy with methodological and subject-matter experts before finalizing the protocol.

The key is not whether two primary outcomes are permitted. It is whether the study can support the conclusions those outcomes are intended to establish.

Before Designating Multiple Primary Outcomes

Frequently Asked Questions About Multiple Primary Outcomes

Multiple Primary Outcomes Are Defensible When Their Joint Purpose Is Clear

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.