When Should Qualitative Data Come First in Sequential Mixed-Methods Research?

Should You Understand the Phenomenon Before Trying to Measure It?

You want to investigate how university teachers use generative AI in assessment. A questionnaire seems appropriate for examining patterns across institutions, but you are uncertain whether existing survey items adequately represent the practices you want to measure.

Teachers may interpret responsible AI use differently, distinguish among assessment activities in unexpected ways, or encounter institutional conditions that existing instruments overlook.

Would it be more defensible to conduct interviews first, establish how teachers understand and experience the phenomenon, and then develop a quantitative investigation informed by those findings?

Qualitative-first sequencing can address this problem, but it is not automatically appropriate whenever a study includes interviews and a survey. The methodological justification depends on whether the qualitative findings are genuinely needed to shape the subsequent quantitative component.

Begin Qualitatively When Understanding Must Precede Quantitative Investigation

How Qualitative-First Sequencing Supports Quantitative Research

What Does Qualitative-First Sequencing Mean?

Qualitative-first sequencing involves conducting qualitative inquiry before a quantitative component, with findings from the initial phase informing decisions about what will subsequently be measured, examined, or evaluated.

The qualitative phase may involve interviews, focus groups, observations, document analysis, or other procedures appropriate to the research question.

Its purpose is not merely to collect preliminary opinions. It should produce findings that contribute meaningfully to the conceptualization or design of the quantitative phase.

For example, interviews with teachers may identify different understandings of responsible generative AI use. These findings could inform the development of candidate questionnaire dimensions and items for subsequent quantitative evaluation.

The sequence creates a developmental relationship: the quantitative component is shaped by what researchers learn qualitatively.

In some methodological notation systems, QUAL → quan represents qualitative-first sequencing with qualitative priority, while qual → QUAN represents qualitative-first sequencing with quantitative priority. These conventions vary, so researchers should define the notation they adopt.

When Is Qualitative-First Sequencing Most Appropriate?

Qualitative-first sequencing is particularly defensible when researchers cannot adequately specify the quantitative investigation without first understanding the phenomenon.

This situation may arise when a construct is emerging, an existing instrument is unsuitable for the target population, or an intervention must be developed around insufficiently understood participant needs.

These situations do not automatically require a full mixed-methods study. Researchers should examine existing theory and empirical evidence before concluding that new qualitative exploration is necessary.

How Is Qualitative-First Sequencing Related to Exploratory Sequential Design?

Qualitative-first sequencing is commonly associated with an exploratory sequential mixed-methods design.

In this arrangement, researchers first investigate a phenomenon qualitatively and then use the findings to inform quantitative data collection, measurement, or evaluation.

Fetters, Curry, and Creswell (2013) describe building as a methods-level integration procedure in which findings from one component inform data collection in another.

This developmental relationship is central to many exploratory sequential studies.

However, qualitative-first timing alone does not establish an exploratory sequential design. Researchers must demonstrate what the initial qualitative phase contributed to the subsequent quantitative investigation.

The full logic of an exploratory sequential mixed-methods design includes the purpose of exploration, the transition between phases, and the integration of their contributions.

Why Might Interviews Need to Come Before a Survey?

Questionnaires require researchers to determine which constructs matter, how they should be represented, and how respondents are expected to interpret the items.

When these decisions are made prematurely, an instrument may impose categories that do not adequately reflect the phenomenon or the target population.

Qualitative interviews can help investigate how participants understand relevant concepts and describe their experiences.

For example, researchers investigating teachers' AI assessment practices might initially assume that responsible use is primarily concerned with disclosure. Interviews may reveal additional concerns about verification, accountability, student autonomy, or the appropriateness of AI assistance for different assessment purposes.

These findings could inform the conceptual framework and candidate questionnaire content.

Nevertheless, researchers should not assume that every interview theme represents a distinct quantitative construct. Conceptual development requires interpretation alongside theory and existing evidence.

Can Qualitative Findings Help Identify Quantitative Variables?

Yes. Qualitative findings may reveal behaviors, experiences, or contextual conditions that warrant subsequent quantitative investigation.

For example, interviews about students' experiences with AI-generated feedback might identify differences in how students verify suggestions, revise their work, or decide whether feedback is useful.

These findings could inform candidate variables for a subsequent quantitative study.

However, researchers must distinguish a qualitative theme from an operationalized variable.

A theme may describe a complex process that cannot be adequately represented by a single survey item. Turning it into a quantitative measure requires conceptual definition, appropriate operationalization, and evidence supporting the intended interpretation.

Qualitative exploration can inform measurement decisions, but it does not eliminate the need for measurement theory.

How Should Qualitative Findings Be Translated Into a Quantitative Instrument?

The transition from qualitative analysis to instrument development should be systematic and transparent.

Researchers should identify which findings are relevant to the intended construct, compare them with existing theory, and document how they inform candidate dimensions and items.

A development matrix may be useful for recording the relationship between qualitative evidence, conceptual decisions, and proposed questionnaire content.

Boateng and colleagues (2018) describe scale development as a multistage process involving item development, scale construction, and evaluation.

Their guidance reinforces an important distinction: generating items from qualitative evidence is an early developmental activity, not the completion of instrument validation.

Researchers considering qualitative-informed instrument or intervention development should plan the subsequent evaluation before committing to the overall research design.

Does Qualitative-First Sequencing Establish Instrument Validity?

No. Qualitative inquiry can contribute evidence about construct relevance, content coverage, and how participants interpret proposed items, but it does not establish all aspects of validity.

The Standards for Educational and Psychological Testing emphasize that validity concerns evidence and theory supporting interpretations of test scores for intended uses.

Depending on the instrument, relevant evidence may concern content, response processes, internal structure, relationships with other variables, and consequences of testing.

Reliability and measurement precision also require appropriate evaluation.

For example, a questionnaire developed from interviews may contain items that participants consider relevant but that do not form the expected measurement structure.

Researchers should report the actual evidence obtained rather than describing the instrument as validated solely because its content originated from qualitative findings.

Can Qualitative Inquiry Come First When Developing an Intervention?

Yes. Qualitative inquiry may help researchers understand participant needs, contextual barriers, and implementation conditions before developing an intervention.

Suppose a research team intends to develop a professional development program supporting teachers' responsible use of generative AI.

Interviews could investigate teachers' assessment responsibilities, perceived risks, existing practices, and institutional expectations.

The findings might inform the intervention's content, delivery arrangements, or implementation supports.

However, intervention development should also consider relevant theory and existing evidence. Participant preferences alone do not establish which intervention components will be effective.

Skivington and colleagues (2021), in their updated framework for developing and evaluating complex interventions, emphasize the importance of context, program theory, stakeholder engagement, and appropriate evaluation.

Qualitative findings may contribute to development, but quantitative evaluation must still be designed to address its intended feasibility, implementation, or outcome questions.

Does Qualitative-First Mean Qualitative Research Has Greater Priority?

No. The component conducted first does not necessarily receive greater methodological importance.

For example, qualitative interviews may serve primarily to develop a questionnaire that is subsequently used in a large quantitative investigation. If the central research contribution concerns quantitative measurement or relationships, the quantitative component may have greater priority.

Alternatively, a study may primarily seek to understand a complex professional practice, with quantitative evidence extending the qualitative findings. In that case, qualitative priority may be appropriate.

Some investigations assign relatively balanced importance to both components.

The distinction between methodological priority and sequence should be made explicit in the research proposal.

How Much Qualitative Evidence Is Needed Before Moving to the Quantitative Phase?

There is no universal number of interviews, focus groups, or observations required before quantitative development can begin.

The adequacy of the qualitative phase depends on its research purpose, sampling strategy, analytical approach, and the diversity and depth of information required.

A narrowly defined construct investigated within a relatively homogeneous population may require a different qualitative sampling approach from an emerging phenomenon spanning multiple disciplines or institutional contexts.

Researchers should establish whether the qualitative findings provide a sufficiently defensible basis for the specific developmental decisions being made.

This does not mean that qualitative inquiry must exhaust every possible perspective before quantitative work begins.

Instead, researchers should assess the adequacy of the evidence in relation to the intended construct, intervention, or subsequent research question.

What Should Determine the Transition Between Phases?

The transition should occur when the qualitative findings have been analyzed sufficiently to inform the decisions required for the quantitative phase.

For instrument development, this may mean establishing candidate constructs, dimensions, and item content.

For intervention development, it may mean identifying relevant needs and contextual conditions, then translating them into a coherent intervention specification.

Researchers should document which findings informed these decisions and where theoretical or practical judgment was also required.

Moving to the quantitative phase before completing the necessary qualitative interpretation may weaken the developmental relationship.

Conversely, continuing qualitative data collection indefinitely may delay quantitative evaluation without providing proportionate additional value.

The transition should be justified by the research purpose and the adequacy of the available evidence.

What If the Qualitative Findings Challenge the Original Research Plan?

Qualitative inquiry may reveal that the original construct is poorly specified, that important participant experiences were overlooked, or that the proposed quantitative investigation is not appropriate.

For example, interviews might suggest that teachers do not understand responsible AI use as a single coherent dimension.

The researcher may need to revise the conceptual framework, reconsider the proposed measurement model, or investigate whether separate constructs are more appropriate.

Such changes can be legitimate when they follow from credible evidence and are documented transparently.

However, researchers should avoid presenting emergent changes as though they were predetermined or selectively retaining only findings that support the original plan.

When Is Qualitative-First Sequencing Unnecessary?

Qualitative-first sequencing may be unnecessary when the intended constructs are already well established, suitable measures exist, and the quantitative question can be answered without additional conceptual exploration.

For example, a study estimating the prevalence of a clearly defined behavior using an appropriate established instrument may not require a preliminary qualitative phase.

Similarly, if the primary purpose is to explain particular quantitative findings, beginning with quantitative analysis may provide a more coherent sequence.

If both components can investigate related aspects of a phenomenon during the same period, concurrent collection may be suitable.

Researchers should not add qualitative exploration merely because interviews appear more exploratory or because a mixed-methods label is considered desirable.

What Are the Advantages and Limitations of Qualitative-First Sequencing?

The potential benefits depend on whether qualitative inquiry addresses a genuine developmental need. The limitations become particularly important when researchers underestimate the evaluation work required after the first phase.

Using Teacher Interviews to Develop a Quantitative AI Assessment Instrument

Suppose interviews identify candidate dimensions concerning disclosure of AI assistance, verification of generated content, and responsibility for assessment decisions.

These dimensions could inform item development, but they should not automatically be treated as distinct validated factors.

Subsequent quantitative evaluation may support the proposed structure, indicate substantial overlap among dimensions, or reveal that certain items do not perform as intended.

The justification for qualitative-first sequencing is that the initial inquiry addresses a conceptual gap that must be resolved before the quantitative investigation can be designed adequately.

The value of the mixed-methods arrangement lies in the transparent developmental relationship between phases and the appropriate evaluation of the resulting measurement claims.

Common Mistakes in Qualitative-First Sequential Research

How to Decide Whether Your Study Should Begin Qualitatively

Start by examining whether the constructs, categories, instruments, or intervention components needed for quantitative investigation are already sufficiently established.

If important uncertainties remain, identify what qualitative inquiry could contribute and how the findings would shape the subsequent quantitative phase.

When writing the methodology, specify the qualitative findings needed to inform the quantitative phase, how those findings will be translated into methodological decisions, and what evaluation will follow.

Allow adequate time for qualitative analysis and the developmental transition. A qualitative-first study may require additional instrument construction, pretesting, or intervention refinement before quantitative data collection can begin.

Finally, document how the first phase influenced the second. The methodological justification should be visible in the design and findings, not merely asserted through the sequence label.

Check Whether Qualitative-First Sequencing Is Appropriate

Frequently Asked Questions About Qualitative-First Mixed Methods

Begin Qualitatively When Understanding Must Inform Measurement or Evaluation

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.