03 · What You Need to Know
How Stakeholders Can Contribute to Research Outcome Selection
What Does Selecting a Research Outcome Involve?
Outcome selection involves deciding which characteristics, events, behaviors, or experiences will be measured to answer a research question. In intervention research, outcomes commonly represent the consequences that investigators expect an intervention to influence.
Researchers must distinguish the outcome construct from the instrument or procedure used to measure it.
For example, academic writing performance is an outcome. A scored writing task is one possible measurement procedure. Student satisfaction with writing instruction is another outcome, not an alternative instrument for measuring writing performance.
This distinction matters when stakeholders suggest outcomes in everyday language. A request to measure whether an intervention makes learning easier may refer to perceived cognitive effort, accessibility, task completion, or another construct requiring clarification.
Stakeholder involvement can help identify the intended meaning before researchers choose a measurement instrument.
Why Might Researchers and Stakeholders Prioritize Different Outcomes?
Researchers often select outcomes according to theoretical frameworks, prior studies, established instruments, and the requirements of a particular design.
Stakeholders may evaluate the same intervention through their experiences, responsibilities, or practical consequences.
For example, a clinical researcher may prioritize a laboratory marker, while patients emphasize functioning or symptoms. An educational researcher may prioritize test performance, while students are concerned about whether the intervention creates additional workload or improves their ability to learn independently.
Neither perspective is automatically sufficient. A personally meaningful outcome may be difficult to measure reliably, while a technically precise outcome may capture only a narrow part of an intervention's consequences.
The value of stakeholder involvement lies partly in making these differences visible before the protocol is finalized.
Who Should Be Involved in Outcome Selection?
Relevant stakeholders depend on the population, intervention, and research purpose.
They may include prospective participants, patients, students, teachers, caregivers, practitioners, community representatives, service providers, policymakers, or people responsible for implementing the findings.
However, researchers should avoid assuming that one representative speaks for an entire stakeholder group.
For example, students with reliable internet access may prioritize different outcomes from students who experience connectivity or accessibility barriers.
Stakeholder identification should consider whose experiences may otherwise be overlooked and whose decisions the research is intended to inform.
The distinction between consultation and research partnership is also relevant. Stakeholders may contribute through time-limited consultation, advisory groups, or sustained shared decision-making, depending on the project's governance arrangements.
At What Stage Should Stakeholders Help Select Outcomes?
Outcome selection is generally most open to meaningful stakeholder influence during the early design stage, before primary outcomes and analysis procedures are finalized.
At this point, researchers can review existing outcome domains, identify omissions, and assess proposed additions without creating avoidable post hoc changes.
Stakeholder consultation may also be useful during instrument development, pilot testing, and interpretation of findings.
However, changing primary outcomes after data collection begins can raise concerns about selective outcome reporting, particularly when investigators already know the emerging results.
Consequential changes should be justified, documented, and approved where necessary. Stakeholder requests do not remove the obligation to preserve the integrity of prespecified analyses.
What Is the Difference Between Outcome Identification and Outcome Prioritization?
Outcome identification
Generating and clarifying the outcomes that may be relevant to the research question and affected populations.
Outcome prioritization
Determining which identified outcomes warrant particular importance, including possible selection as primary or secondary outcomes.
Stakeholders may identify many relevant outcomes without agreeing about which are most important.
A research team should therefore avoid treating an initial list of stakeholder suggestions as a finalized outcome set. Prioritization requires additional consideration of scientific relevance, measurement feasibility, participant burden, and the intended interpretation.
How Can Researchers Gather Stakeholder Outcome Priorities?
Several approaches may be suitable, depending on the scale of the study and the degree of participation intended.
-
Individual interviews: Useful for exploring experiences, priorities, and outcomes that may not appear in existing instruments.
-
Focus groups: Allow stakeholders to discuss and compare the consequences they consider important.
-
Structured surveys: Can collect priorities from a broader group using consistent questions.
-
Delphi processes: Can support iterative rating and reconsideration of outcomes across stakeholder groups.
-
Consensus meetings: Can facilitate discussion of priorities and the rationale for including or excluding outcomes.
-
Participatory workshops: Can help stakeholders clarify outcome concepts and connect them to the intended research purpose.
These methods are not interchangeable. A Delphi process does not automatically produce representative stakeholder preferences, and consensus does not guarantee scientific validity.
Researchers should select methods according to the decisions being made, the participants involved, and the resources available.
What Are Core Outcome Sets?
A core outcome set is an agreed minimum collection of outcomes that should be measured and reported in studies addressing a particular research area.
Core outcome sets are intended to improve consistency, facilitate evidence synthesis, and reduce problems arising when studies measure incompatible outcomes.
The COMET Initiative provides resources supporting core outcome set development. The COS-STAD recommendations address methodological standards for developing such sets, including consideration of relevant stakeholders and consensus processes.
A core outcome set identifies what should be measured, but it does not necessarily prescribe the instrument used for each outcome.
Researchers must still select appropriate measures and evaluate their measurement properties for the intended populations.
Not every individual study needs to develop a new core outcome set. Investigators should first examine whether a relevant established set exists and whether it is appropriate for their research context.
Can Stakeholders Help Decide the Primary Outcome?
Yes, particularly when the research question concerns consequences experienced by participants or communities.
Stakeholder input may reveal that an outcome initially considered secondary is central to the intervention's practical value.
However, selecting a primary outcome has important methodological implications. The primary outcome ordinarily plays a central role in sample-size planning, analysis, and interpretation.
A proposed primary outcome should be sufficiently aligned with the research question, appropriately defined, measurable with adequate validity and precision, and feasible to collect.
Researchers should also consider its sensitivity to the expected intervention effect and the consequences of missing data.
Stakeholders can contribute to the decision, but a popularity vote alone is not a sufficient scientific basis for selecting a primary endpoint.
What If Stakeholders Prefer Outcomes That Are Difficult to Measure?
An outcome may be important even when no suitable measurement instrument is readily available.
For example, students may prioritize a sense of intellectual independence, but available questionnaires may measure only narrow aspects of perceived autonomy.
Researchers should investigate whether the construct can be operationalized appropriately, whether instrument development is feasible, or whether qualitative methods can address the question.
They should not substitute an unrelated convenient indicator merely to satisfy the appearance of measurement.
Where measurement is not feasible within the current study, the outcome may be identified as a priority for future instrument development or research.
Explaining this limitation to stakeholders is preferable to silently discarding their concern.
How Should Conflicting Stakeholder Priorities Be Managed?
Different stakeholder groups may value different outcomes because they experience different consequences or hold different responsibilities.
For example, teachers may prioritize workload, students may prioritize learning quality, and administrators may prioritize completion rates.
Researchers should not assume that the largest or most institutionally powerful group should determine the outcome hierarchy.
Structured prioritization can make disagreements visible. Separate stakeholder-group ratings may be informative, particularly when an overall average would conceal strong differences.
Consensus procedures should specify how decisions will be made and what happens when agreement cannot be reached.
Where a disagreement remains, researchers may retain distinct secondary outcomes or document why one outcome was prioritized for the primary question.
Should Stakeholder Preferences Be Weighted Equally?
There is no universal weighting rule.
Equal weighting may be appropriate in some participatory processes, while other arrangements may be justified by the purpose of the decision and the responsibilities of the participants.
However, weighting preferences by professional status or organizational authority can marginalize those most directly affected by the intervention.
Researchers should establish decision rules prospectively and explain the rationale for the influence assigned to different groups.
Where stakeholder groups are unequally represented, investigators should consider whether the resulting priorities reflect the intended population of perspectives.
How Can Stakeholders Improve Content Validity?
Content validity concerns whether a measure adequately represents the intended construct for its specified purpose and population.
Stakeholders may identify relevant experiences omitted from an instrument, ambiguous wording, or response categories that do not reflect their circumstances.
For example, a measure of digital learning accessibility might focus on device ownership while overlooking shared-device use, assistive technology compatibility, or the cost of maintaining internet connectivity.
Such contributions can strengthen the conceptual coverage of a measure.
However, stakeholder consultation is one source of validity evidence, not a complete validation process. Reliability, measurement structure, response processes, and other relevant properties may still require evaluation.
Can Stakeholders Recommend Outcomes That Favor a Preferred Intervention?
Yes. Stakeholders may have legitimate interests in particular findings, especially when they developed, funded, deliver, or receive the intervention.
For example, an implementing organization may prefer an outcome likely to demonstrate program success, while participants may emphasize unintended burdens.
Researchers should distinguish the relevance of a proposed outcome from whether it is expected to produce favorable results.
Outcome selection should follow transparent criteria established before outcome data are examined.
Where conflicts of interest exist, they should be disclosed and managed through appropriate governance arrangements.
Stakeholder participation can improve relevance without transferring control over findings or permitting selective outcome reporting.
How Many Outcomes Should a Study Include?
There is no universal maximum number of outcomes, but including every stakeholder suggestion is rarely feasible or methodologically desirable.
Additional outcomes can increase participant burden, data collection costs, missingness, analytical complexity, and opportunities for selective interpretation.
Researchers should establish a justified outcome hierarchy and distinguish primary, secondary, and exploratory outcomes.
Where many statistical hypotheses are tested, an appropriate multiplicity strategy may be necessary.
Stakeholders can help prioritize outcomes, but the research team must also evaluate whether the proposed data collection is proportionate to the study's objectives.
How Should Outcome Selection Decisions Be Documented?
A transparent decision record should identify the stakeholder groups consulted, methods used, outcomes proposed, prioritization criteria, disagreements, and final decisions.
It should also explain why particular outcomes were excluded, retained as secondary outcomes, or selected as primary.
The GRIPP2 reporting checklists provide guidance for reporting patient and public involvement in research, including its aims, methods, results, and influence.
Although developed for patient and public involvement, the underlying principle of documenting participation and its consequences is relevant to other stakeholder-engaged research.
Reporting should not imply that stakeholders made decisions they merely reviewed or that consensus existed when substantial disagreement remained.
How Does Outcome Selection Relate to Other Stakeholder Decisions?
Outcome selection is connected to recruitment, participant burden, intervention design, and data collection feasibility.
An outcome that requires repeated lengthy assessments may be important but difficult to collect from participants with limited time or resources.
Researchers should therefore coordinate outcome decisions with stakeholder assessment of participant burden and stakeholder assessment of study feasibility.
These considerations do not justify discarding important outcomes automatically. They help researchers determine whether the measurement plan can be implemented responsibly.
Watch Out
Do not equate stakeholder consensus with measurement validity or treat the most frequently preferred outcome as automatically suitable for the primary analysis. Stakeholder importance, scientific relevance, measurement quality, and feasibility are related but distinct considerations.
