03 · What You Need to Know
How to Balance Consistency and Flexibility Across Research Interviews
Standardization and Comparability Are Related but Different
Standardization concerns the extent to which the interview procedure is held constant across participants. This may include question wording, sequence, response options, instructions, and permitted prompts.
Comparability concerns whether the resulting evidence can meaningfully be examined across participants or cases.
Standardization can support comparability, particularly when researchers seek standardized factual responses or numerical estimates. However, identical wording does not guarantee that participants interpret a question in the same way.
Conversely, qualitative interviews can produce analytically comparable accounts even when participants receive different probes, provided that the researcher has a defensible basis for examining their experiences in relation to common research concerns.
The relevant question is therefore not simply whether the words are identical. It is whether the interview procedure produces evidence appropriate for the comparisons and interpretations the study intends to make.
Structured Interviews Usually Require Consistent Questions
Structured interviews use a predetermined protocol, often specifying exact question wording, sequence, response categories, and permissible clarification procedures.
This consistency is especially important when responses will be coded into standardized categories or compared quantitatively.
For example, a researcher conducting interviewer-administered surveys about teachers' professional-development participation may need every eligible respondent to receive the same question about training completed during the previous 12 months.
Changing the reference period for some respondents would create a measurement difference that could undermine comparison.
However, standardized interviewing does not necessarily prohibit every clarification. Interviewers may use approved explanations or neutral prompts specified in the protocol.
The distinction among structured, semi-structured, and unstructured interviews determines how much procedural variation is consistent with the design.
Semi-Structured Interviews Usually Require Shared Topics, Not Identical Conversations
Semi-structured interviews commonly use an interview guide containing core questions, topics, and possible probes.
The guide establishes the substantive areas the researcher intends to explore while allowing the interviewer to respond to each participant's account.
For example, all participants in a study of teachers' generative AI practices might be invited to describe their experiences with AI-supported instructional preparation.
One participant may discuss assessment design, while another emphasizes institutional restrictions. The interviewer can ask different follow-up questions to understand these experiences without abandoning the shared research objective.
Kallio and colleagues (2016) describe the development of semi-structured interview guides as a systematic process requiring attention to the research purpose, preliminary knowledge, guide development, and testing.
The guide should provide sufficient direction for relevant coverage without preventing participants from introducing unexpected information.
Unstructured Interviews May Differ Substantially Between Participants
Unstructured interviewing permits the conversation to develop through participants' accounts and the researcher's evolving understanding.
Questions may emerge from observations, earlier conversations, or the circumstances of a particular participant.
For example, an ethnographic researcher studying informal technology-sharing practices may ask different questions of a newly appointed teacher, an experienced instructional designer, and a department coordinator.
The conversations can remain part of a coherent investigation because they contribute to understanding the same social setting or phenomenon.
However, the researcher must explain how the material will be interpreted and compared. Unstructured interviewing is not an absence of methodological planning.
What Should Remain Consistent in Semi-Structured Interviews?
Consistency can operate at several levels.
The research question and analytical purpose should remain recognizable throughout the study. Core topics should receive sufficient attention where relevant, and the researcher should use a defensible approach to probing and documenting interviews.
However, the precise wording and sequence may vary when participants have different experiences or when a topic emerges naturally earlier in the conversation.
For example, a participant may spontaneously describe institutional support before the interviewer reaches the corresponding guide question. Repeating the question verbatim may be unnecessary if the relevant evidence has already been obtained.
Researchers should distinguish between ensuring that a topic has been adequately explored and requiring that a particular sentence be spoken.
Core Questions and Optional Probes Serve Different Functions
A useful semi-structured guide may distinguish core questions from optional follow-up prompts.
Core questions
Prompts intended to address essential areas of inquiry across relevant participants.
Optional probes
Follow-up prompts used when clarification, elaboration, examples, or exploration of an emerging issue is needed.
For example, a core question might ask, "Can you describe how you decide whether to use generative AI when preparing instructional materials?"
Possible probes could explore institutional expectations, concerns about accuracy, previous experiences, or circumstances in which the participant chose not to use AI.
Not every probe must be asked when the participant has already addressed the issue or when it is irrelevant to their experience.
The degree of flexibility permitted within a semi-structured interview guide should be established before fieldwork and reviewed as the study progresses.
When Should Questions Differ Because Participants Have Different Roles?
Some studies intentionally recruit participants who occupy different positions within a system.
A researcher investigating an institutional AI policy might interview teachers, academic administrators, and instructional technology specialists.
These participants may have different responsibilities and access to information.
Teachers can describe classroom decisions, administrators can explain policy implementation, and technical specialists can discuss support procedures.
Using identical questions for every role may produce irrelevant or speculative responses.
Role-specific interview guides can therefore be appropriate when they retain alignment with the overarching research questions.
Researchers should document which questions were intended for each participant category and avoid treating role-specific responses as though they arose from identical opportunities to provide information.
What If a Question Does Not Apply to a Participant?
Questions should not force participants to describe experiences they have not had.
Suppose an interview guide asks, "How did the AI training program change your assessment practices?" A participant who did not attend the program cannot answer the question as intended.
The interviewer should not pressure the participant to speculate about a nonexistent experience.
Instead, the researcher may establish nonparticipation and, if relevant, explore why the participant did not attend or how they developed their practices through other means.
Such adaptations should remain consistent with the research question and the study's ethical procedures.
They also illustrate why interview questions should be conceptually aligned with research questions without necessarily reproducing the same wording for every participant.
Can Different Questions Still Support Cross-Participant Analysis?
Yes, but the researcher must distinguish differences in experience from differences in the opportunity to discuss a topic.
Suppose 12 participants describe difficulties with institutional AI policies, while eight do not mention the issue.
If all participants were invited to discuss policy experiences, the pattern may support one form of interpretation. If only the first 12 were asked about policies, the absence of discussion among the remaining eight cannot be interpreted in the same way.
Qualitative analysis must therefore consider how the interview material was generated.
A topic's absence from a transcript is not necessarily evidence that the participant did not experience it.
Researchers should preserve sufficient contextual information to interpret what participants were asked and what they volunteered independently.
How Can Interviewers Avoid Leading Participants During Flexible Probing?
Flexibility creates opportunities for deeper inquiry, but it also allows interviewers to introduce assumptions.
For example, asking "Why did the university's restrictive AI policy frustrate you?" assumes that the policy was restrictive and that the participant experienced frustration.
A more neutral prompt would be, "How, if at all, has the university's AI policy affected your teaching?"
Follow-up questions can then explore the participant's actual interpretation.
Researchers should remain attentive to the risk of leading participants toward a conceptual framework, particularly when the study is intended to explore rather than confirm predetermined categories.
Neutrality does not mean avoiding every theoretical concept. It means allowing participants to describe experiences that may support, complicate, or contradict the researcher's expectations.
Should the Interview Guide Change During Data Collection?
In some qualitative methodologies, interview guides may be refined as data collection and analysis proceed.
Early interviews may reveal ambiguous wording, missing topics, or emerging issues that warrant further exploration.
However, substantial changes should be documented and evaluated for their effects on comparability and the study's approved procedures.
A change that clarifies an ambiguous question differs from introducing an entirely new research objective.
Where the methodology permits iterative development, researchers should explain how revisions were informed by emerging analysis.
Changes affecting participant burden, sensitive topics, or the approved scope may require additional ethics review.
How Should Interview Differences Be Documented?
Documentation supports transparent analysis and reporting.
Researchers may retain the interview guide version used, note substantial departures from planned questions, and record relevant contextual circumstances.
Field notes can identify why a particular probe was introduced or why a question was omitted.
For multi-interviewer studies, training and periodic review can help maintain a shared understanding of essential topics and acceptable probing practices.
These procedures do not require every interview to become identical. They make the differences understandable and methodologically accountable.
Does Asking Identical Questions Improve Qualitative Rigor?
Procedural consistency can be valuable, but qualitative rigor is not established solely by identical wording.
Depending on the methodological tradition, researchers may need to demonstrate credibility, reflexivity, contextual sensitivity, analytical transparency, and coherence between the research question and data-generation process.
A rigid interview protocol may improve standardization while preventing exploration of important experiences.
Conversely, unrestricted variation may leave central research questions inadequately addressed.
The appropriate balance depends on the study's epistemological assumptions and analytical objectives.
How Does Question Consistency Affect Sampling and Saturation?
Sampling adequacy and interview consistency are related because the questions asked influence the information available for analysis.
If different participants are systematically asked about different topics, the researcher should consider whether apparent thematic gaps reflect participant experiences or uneven questioning.
Similarly, repeated answers may reflect a narrowly structured interview guide rather than comprehensive understanding of the phenomenon.
When deciding whether additional interviews are needed to achieve saturation, researchers should consider both participant variation and the opportunities created by the interview procedure.
Sample adequacy cannot be evaluated independently of how the data were generated.
