Can an AI Chatbot Generate Follow-Up Questions Dynamically During a Research Interview?

Can AI Decide What to Ask Next Based on a Participant's Answer?

Imagine interviewing teachers about their experiences using generative AI in the classroom. One teacher mentions that AI saves preparation time, while another describes concerns about inaccurate instructional content.

A human interviewer might explore these responses differently. Could an AI chatbot do the same, generating relevant follow-up questions during the interview instead of relying entirely on questions written beforehand?

The technical capability already exists. The more consequential question is whether those dynamically generated questions improve the evidence collected without introducing unintended influences into the interview.

Yes, AI Can Generate Context-Sensitive Follow-Up Questions

How Dynamic AI Interview Probing Works

What Is Dynamic Follow-Up Question Generation?

Dynamic follow-up question generation is a conversational process in which an AI system determines whether to ask an additional question and, when appropriate, formulates that question using information from the ongoing interview.

Unlike a fixed interview script, the system does not need to know every follow-up question before data collection begins. Instead, it uses the participant's previous statements and the interview protocol to determine what information might be explored further.

For example, consider the following exchange:

The follow-up responds to a concern raised by the participant rather than introducing an unrelated topic. It invites a specific example that may help the researcher understand the experience more fully.

Importantly, generating such a question does not mean the AI understands distrust in the same way a human researcher might. It identifies linguistic and contextual relationships that allow it to formulate a plausible next question.

How Does the Chatbot Decide Whether to Ask a Follow-Up?

A controlled AI interviewing system can separate the decision to probe from the generation of the question itself.

This workflow is a possible implementation rather than a universal architecture. Researchers can use different combinations of programmed rules, generative models, and human oversight.

Gårdhus and colleagues (2026), for example, describe AInterviewer, a research platform that combines controlled administration of interview questions with LLM-supported conversational flexibility. The design responds to concerns about standardization and reproducibility in automated interviewing. The published system description illustrates why control over the interview sequence matters even when the chatbot can generate additional questions.

What Types of Follow-Up Questions Can AI Generate?

Different probes serve different methodological purposes. Researchers should specify which types are appropriate for their interview protocol.

These probes may be appropriate in semi-structured interviews, but their usefulness depends on timing and context. Asking for a specific example can be helpful when a participant makes a broad claim. Repeatedly demanding examples after a participant has already provided sufficient detail may instead make the conversation burdensome.

A well-designed AI interviewer should therefore be able to refrain from probing when further questioning is unnecessary.

What Does Research Say About AI-Generated Follow-Up Questions?

Recent research suggests that LLMs can generate context-sensitive probes, although their performance varies by model, interviewing task, and evaluation conditions.

A 2026 study published in Scientific Reports, titled "The AI interviewer: multi-faceted evaluation of adaptive questioning by large language models," examined adaptive interviewing behavior across six language models. The researchers evaluated whether follow-up questions were necessary, context-aware, open-ended, and appropriately formulated, including whether the system could recognize situations in which no follow-up was needed.

The study used a controlled, replay-based design involving human baseline interview transcripts and an LLM-generated interviewee for subsequent probing. This allowed systematic comparison of model behavior, but it did not establish how all systems would perform during unrestricted interviews with real participants.

Another study by Zhang and colleagues (2026), published in the International Journal of Human-Computer Studies, investigated AI-generated follow-up questions delivered by human interviewers. Its findings indicated that AI suggestions could support the breadth and depth of probing when the questions were relevant and appropriately timed.

These studies point toward two distinct applications: an AI system independently conducting follow-up exchanges and an AI system proposing questions for a human interviewer to evaluate. Evidence from one arrangement should not automatically be generalized to the other.

What Makes an AI-Generated Follow-Up Question Appropriate?

A useful follow-up should be relevant to the research question and grounded in what the participant actually said. It should also preserve the participant's freedom to explain the experience in their own terms.

Consider the difference between these two questions after a participant mentions difficulty using AI:

The second question assumes that the difficulty resulted from unreliability and that the effect was negative. Neither interpretation necessarily follows from the participant's original statement.

Researchers should assess follow-up questions for relevance, openness, clarity, neutrality, and consistency with the interview objectives. For sensitive topics, appropriateness also depends on whether the question respects participants' emotional and informational boundaries.

Can Dynamic Probing Change What the Interview Measures or Reveals?

Yes. Follow-up questions influence which experiences participants elaborate, how they frame their answers, and what information becomes available for analysis.

Suppose an AI interviewer consistently asks participants to elaborate on negative experiences but rarely explores positive ones. The resulting transcripts may contain more detailed accounts of difficulties, not necessarily because negative experiences are more common, but because the system pursued them more extensively.

This is particularly consequential when researchers compare themes across participants. Differences in thematic richness may reflect the chatbot's probing decisions rather than differences in participants' underlying experiences.

The issue connects to systematic measurement bias in AI-mediated interviewing. Dynamic questioning can introduce variation into the data collection process even when every participant receives the same initial questions.

How Much Freedom Should the AI Have?

Researchers can restrict AI-generated probes in several ways, including specifying eligible topics, permitted question forms, maximum probing depth, and conditions requiring the system to move forward.

One possible arrangement is to require all participants to receive the same core questions while allowing limited follow-up questions only when clarification or elaboration is needed.

Another arrangement is to let the AI suggest probes to a human interviewer, who decides whether they are appropriate. This preserves human judgment during data collection but may introduce additional cognitive demands or inconsistent acceptance of suggestions.

The appropriate degree of autonomy depends on the study's methodological assumptions and the consequences of inappropriate probing. Greater flexibility should be justified by the information it helps collect, not simply by the system's technical capabilities.

Can AI Recognize When It Should Stop Probing?

A chatbot can be programmed to stop after a predetermined number of follow-ups, when the participant declines to elaborate, or when the system determines that the question has been sufficiently addressed.

However, an AI-generated assessment of answer completeness is itself a fallible judgment. A participant may have provided an adequate response even if the system continues asking for detail.

Researchers should establish stopping rules that do not depend entirely on the model's interpretation. These might include a maximum number of probes per core question, participant-controlled skipping, and explicit instructions to avoid repetitive questioning.

When the topic is emotionally sensitive, the system also needs safeguards against continuing to probe after signs of discomfort or a request to stop. The risk of inappropriate responses to participant distress cannot be addressed solely by making the chatbot sound sympathetic.

Generating Follow-Up Questions During an Interview About AI in Teaching

The follow-up questions help move from a general observation about time savings to a concrete account of verification practices. They also allow the participant to explain the experience without requiring the chatbot to decide whether AI was beneficial or harmful.

Now consider an alternative AI-generated question: "Why do you continue using AI when it frequently produces inaccurate statistics?"

This would be problematic because it assumes frequent inaccuracy and implicitly challenges the participant's decision. A single example of an incorrect output does not establish that such errors occur frequently.

The researcher therefore needs a procedure for detecting and correcting inappropriate probes, not merely a prompt instructing the chatbot to ask good questions.

Misconceptions About AI-Generated Interview Probes

How to Use Dynamic AI Probing Without Losing Methodological Control

Begin with the purpose of each core interview question. Identify the kinds of clarification or elaboration that would help answer your research questions, and specify what the chatbot must not ask.

For studies where probing decisions carry relatively low risk, controlled AI-generated follow-ups may be feasible after adequate testing. For interviews requiring complex interpretation or sensitive interpersonal judgment, human supervision may be more appropriate.

A useful pilot evaluation should examine actual generated questions rather than relying exclusively on the researcher's impression of the completed transcripts. Review whether the chatbot asks unnecessary questions, introduces assumptions, overlooks important responses, or treats similar participant statements inconsistently.

Where feasible, independent reviewers can evaluate a sample of generated probes using a predefined rubric. The process of validating AI-mediated data collection should include the behavior of the interviewing system itself, not only the apparent usefulness of its final transcripts.

Preserve appropriate records of generated questions and participant interactions so that unexpected patterns can be investigated while respecting data minimization and confidentiality requirements.

Before Allowing AI to Generate Interview Follow-Ups

Frequently Asked Questions About Dynamic AI Interview Questions

Dynamic AI Probing Is Possible, but Every Question Has Methodological Consequences

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.