03 · What You Need to Know
How to Determine Whether One Research Design Can Support Several Questions
What Makes a Study Too Broad?
A study becomes methodologically overextended when its objectives exceed what the proposed design, sample, data, resources, or analytical framework can credibly support.
There is no universal maximum number of research questions or objectives. A large, well-resourced longitudinal investigation may support several related questions, while a small experiment may need a tightly defined primary objective.
The problem is not breadth itself. It is the absence of a coherent relationship between the questions being asked and the evidence being collected.
For example, evaluating the effectiveness of an intervention and exploring participants' experiences of that intervention may fit naturally within a mixed methods design. Investigating the same intervention's effectiveness, national economic implications, institutional governance, and long-term labor market consequences may require several distinct designs and populations.
The central question is whether the research components contribute to one defensible inferential structure or represent separate investigations connected mainly by a common topic.
Do the Research Questions Require Different Types of Evidence?
Different research questions impose different evidential requirements. Questions about causal effects generally require designs capable of supporting causal inference. Questions about lived experience may require qualitative approaches, while questions about implementation may need organizational or process evidence.
These approaches can be integrated within mixed methods research when there is a clear rationale for combining their findings. Their methodological differences do not automatically require separate studies.
Separation becomes more compelling when the questions require fundamentally different data collection systems, units of analysis, time horizons, or sampling frames, and when combining them adds little scientific value.
| Research question |
Likely evidence requirement |
Potential design implication |
| Does an intervention improve achievement? |
Comparable outcome measurements and an appropriate counterfactual |
Randomized or carefully designed quasi-experimental study |
| How do students experience the intervention? |
Detailed accounts of experiences and contextual meaning |
Qualitative study or integrated qualitative component |
| What predicts teachers' adoption? |
Teacher-level characteristics, organizational conditions, and adoption measures |
Observational or explanatory study |
| Does the intervention produce lasting effects? |
Longitudinal measurements and appropriate follow-up |
Longitudinal study or extended follow-up component |
| Is institution-wide implementation feasible? |
Implementation processes, resources, fidelity, and feasibility indicators |
Implementation or feasibility study |
These are illustrative possibilities, not fixed methodological assignments. The appropriate design depends on the exact question, assumptions, available evidence, and intended inference.
Can One Sample Adequately Support All the Objectives?
A common warning sign is that different research questions require different participants or levels of analysis.
For instance, a student-level experiment may provide evidence about academic achievement, but it may not contain enough teachers or institutions to support credible conclusions about organizational adoption.
Having 500 students does not necessarily provide adequate statistical information for an analysis in which the relevant units are 12 teachers or four schools. The effective sample size depends on the research question, clustering, variability, and design.
Similarly, a study powered for its primary outcome may not have sufficient precision for numerous additional outcomes. Researchers should consider whether secondary outcomes require their own statistical power considerations when those outcomes are intended to support important separate conclusions.
If the sample cannot adequately address several principal objectives, narrowing the claims or planning additional studies may be more defensible than relying on underpowered analyses.
When Do Different Time Horizons Justify Separate Studies?
Some questions can be answered shortly after an intervention, while others require months or years of follow-up.
Immediate achievement, sustained knowledge retention, institutional adoption, and long-term career outcomes may require different observation periods and retention strategies.
A single longitudinal design can sometimes address these questions, especially when the same participants and measurements remain relevant. However, extending every component of a study to accommodate one long-term objective may create unnecessary cost and participant burden.
Separate but coordinated studies may be preferable when the long-term question requires a different population, sampling approach, or data source.
How Can Too Many Objectives Weaken Statistical Inference?
When a study treats numerous distinct questions as equally central, researchers may encounter multiple primary outcomes, multiple comparisons, inadequate power, and unclear criteria for success.
These problems do not automatically justify dividing the study. Appropriate multiplicity planning for multiple outcomes may allow a coherent investigation to retain several confirmatory hypotheses.
Nevertheless, statistical adjustment cannot solve every design problem. A single dataset may simply lack the participants, measurements, or causal structure necessary to support several independent claims.
Researchers should distinguish an analysis problem that can be addressed within one study from a research design problem that requires additional evidence.
When Is a Multi-Study Research Program More Appropriate?
A research program consists of coordinated investigations addressing related questions through distinct but complementary designs. The studies may share a conceptual framework while having separate objectives, samples, methods, or data collection periods.
For example, an intervention development program might begin with qualitative needs assessment, proceed to a feasibility investigation, and then evaluate effectiveness in a sufficiently powered trial.
The studies contribute to a broader scientific objective, but each addresses a different evidential task. Findings from an earlier study may inform the design of a later one.
Such sequencing is particularly useful when the intervention, measurement instruments, or implementation procedures are not yet sufficiently established for a definitive effectiveness study.
Guidance from the U.S. National Center for Complementary and Integrative Health distinguishes feasibility research from definitive efficacy testing. Feasibility studies should primarily assess whether a future trial can be conducted appropriately, rather than presenting underpowered hypothesis tests as conclusive evidence of efficacy.
When Is One Integrated Study Better Than Several Separate Studies?
Integration may be preferable when research questions are closely connected, depend on the same intervention and participants, or require joint interpretation.
A mixed methods evaluation, for instance, may combine an intervention outcome analysis with interviews explaining how implementation conditions shaped participants' experiences. Separating these components could weaken the explanatory relationship between the quantitative and qualitative evidence.
Similarly, a well-designed longitudinal cohort may support several related outcomes without requiring a new cohort for every question.
Maintaining one study may also reduce duplication in recruitment, data collection, ethics review, and participant burden. These practical advantages matter, provided the integrated design remains methodologically adequate.
Potential Advantages of Separate Studies
- Clearer research objectives and inferential claims.
- Designs and samples tailored to distinct questions.
- More appropriate measurement and analytical strategies.
- Opportunities to refine methods between sequential investigations.
- Greater clarity about which evidence supports each conclusion.
Potential Limitations of Separate Studies
- Additional recruitment, administration, and research costs.
- Longer timelines when studies depend on one another.
- Possible loss of contextual connections across datasets.
- Difficulty comparing findings from different populations or settings.
- Risk of unnecessary fragmentation or overlapping publications.
What Is the Difference Between Separate Studies and Salami Slicing?
Separating a broad project into scientifically distinct investigations is not the same as dividing one study's findings into multiple minimally differentiated publications.
Salami slicing generally refers to fragmenting a research contribution into multiple publications that contain substantial overlap and offer insufficiently distinct scientific contributions. The concern is especially relevant when the same participants, data, methods, and central conclusions are repeatedly presented as separate contributions.
Legitimate secondary analyses of an existing dataset can support separate publications when they address genuinely distinct questions, use appropriate methods, and transparently disclose the relationship to earlier work.
Separate manuscripts do not necessarily imply separate studies. One study may yield several valid publications, while a coordinated research program may include several genuinely independent investigations.
COPE's publication ethics resources provide guidance on redundant publication and the responsibilities of authors and editors. Researchers should also consult the specific journal's policies concerning overlapping submissions and previously published analyses.
Watch Out
Do not divide a study merely to create additional publications or conceal the fact that several analyses originate from the same participants and dataset. Conversely, do not combine methodologically incompatible questions solely because they share a broad topic. The scientific contribution, not the publication count, should determine the structure.
How Should Researchers Coordinate Several Related Studies?
When separation is justified, the broader project should still have a coherent research rationale. Each study needs a distinct question, appropriate design, defined population, analysis plan, and intended contribution.
Researchers should document how the studies relate to one another, whether they share participants or data, and which findings are intended to inform subsequent investigations.
Where appropriate, separate protocols, ethics approvals or amendments, registrations, and data management arrangements may be necessary. Requirements vary by institution, jurisdiction, funder, and study type.
Shared conceptual foundations can strengthen a research program, but they do not remove the need to establish the validity of each study independently.
04 · A Practical Example
Turning an Overly Broad Educational Technology Project Into a Coherent Research Program
Hypothetical Example
Evaluating an AI-Supported University Learning System
A research team proposes one project with five major objectives: identify students' learning needs, develop an AI-supported learning intervention, evaluate its effects on achievement, investigate teachers' adoption decisions, and assess implementation feasibility across departments.
The original proposal assumes that one student survey and a single-semester intervention can adequately address every objective.
On methodological review, the team recognizes that the objectives require different forms of evidence and cannot all be supported by the proposed sample.
| Focused investigation |
Primary question |
Appropriate evidence |
| Study A: Needs assessment |
What learning difficulties and support needs should inform intervention design? |
Student and teacher interviews, contextual analysis, and relevant needs-assessment data |
| Study B: Feasibility evaluation |
Can the intervention be delivered and evaluated as intended? |
Recruitment, retention, usability, implementation fidelity, and feasibility indicators |
| Study C: Effectiveness evaluation |
Does the intervention improve the prespecified learning outcome? |
An appropriately designed comparison with justified sample size and outcome measurement |
| Study D: Teacher adoption |
Which factors are associated with teachers' willingness or decisions to adopt the system? |
Teacher-level data and an appropriate explanatory or qualitative design |
Establish the conceptual relationship.
The studies share an overarching interest in developing and evaluating an AI-supported learning system, but each addresses a different research objective.
Sequence dependent investigations.
Needs assessment informs intervention development. Feasibility findings help refine procedures before a larger effectiveness evaluation.
Match each design to its question.
The effectiveness study requires an appropriate comparison and sample size. The adoption investigation requires evidence from teachers rather than assuming student outcomes explain teacher decisions.
Retain opportunities for integration.
Findings can be interpreted together in a research synthesis, provided differences in samples, settings, and inferential strength are acknowledged.
Plan transparent dissemination.
Each study should make a distinct contribution and disclose relevant relationships to other investigations, including shared data or participants where applicable.
The team need not necessarily conduct four entirely separate data collection exercises. Some components could be integrated, particularly if they share participants, timing, and compatible methods. The essential improvement is that each research question receives an appropriate evidential foundation.
The broader project remains coherent while avoiding the assumption that one convenient dataset can answer every question about the intervention.
06 · What This Means for You
How to Decide Whether to Keep a Project Integrated or Divide It
Write each major research question as a separate statement and identify the evidence required to answer it. Then compare the questions across their target populations, units of analysis, measurement strategies, time horizons, and inferential objectives.
If the same design can address the questions with adequate rigor and integration contributes to interpretation, retaining one study may be preferable. If the questions require incompatible designs or cannot be supported by the available data, separation deserves serious consideration.
A simple decision framework
If several questions share one population, intervention, and coherent analytical framework
Consider one integrated study with clearly prioritized outcomes and appropriate statistical planning.
If different questions require substantially different populations or units of analysis
Consider separate studies or clearly differentiated components with appropriate sampling strategies.
If intervention development or feasibility remains uncertain
Consider a staged research program before undertaking a definitive effectiveness evaluation.
If qualitative and quantitative evidence must be interpreted together
Consider an integrated mixed methods design rather than separating components solely because their methods differ.
If the main motivation for separation is producing more manuscripts
Reassess whether each proposed publication has a sufficiently distinct question and contribution, and disclose shared data transparently.
When the principal problem is an excessive number of outcomes rather than incompatible research designs, it may be better to reduce outcome proliferation and clarify analytical priorities within the existing study.
When several distinct studies are warranted, create a research program map showing the purpose, design, population, data requirements, and intended contribution of each investigation. Identify dependencies so that later studies can build on earlier evidence without treating preliminary findings as established conclusions.
Finally, distinguish the organization of the research from the organization of its publications. A single coherent study can generate several legitimate analyses, while multiple studies may sometimes be best synthesized in one substantial publication. Scientific design and publication strategy are related decisions, but they should not be confused.
