03 · What You Need to Know
How AI Can Help Design Appropriate Survey Response Options
Start With the Information the Question Is Asking For
Response options should follow the measurement task rather than determine it. Before requesting answer choices from AI, identify what respondents are expected to report and how that information will be used.
Consider the question: "During the past 30 days, how often have you used generative AI to prepare teaching materials?"
Agreement options would not directly answer this question. Respondents need to report a frequency, whether through a numerical count or clearly defined frequency categories.
By contrast, the statement "Using generative AI improves my teaching efficiency" asks respondents to evaluate a proposition. An agreement scale may be appropriate if the item is intended to measure a belief about usefulness.
AI may suggest either format, but researchers should determine which one corresponds to the intended variable.
What Types of Response Options Can AI Suggest?
Different research questions call for different answer formats. AI can propose alternatives, but their suitability depends on the information required.
| Response Format |
Example Question |
Possible Response Options |
| Agreement |
Using AI improves my teaching efficiency. |
Strongly disagree to strongly agree |
| Frequency |
How often do you use AI for lesson preparation? |
Never, rarely, sometimes, often, always |
| Satisfaction |
How satisfied are you with the AI training? |
Very dissatisfied to very satisfied |
| Numerical response |
How many AI-related workshops have you attended? |
Nonnegative whole number |
| Single-choice category |
What is your primary teaching role? |
Categories appropriate to the institution |
| Multiple response |
Which teaching tasks have you used AI for? |
Select all applicable activities |
| Open-ended response |
What concerns do you have about using AI in teaching? |
Free-text response |
These examples illustrate possible formats, not universally optimal choices. In particular, verbal frequency labels such as "rarely" and "often" may be interpreted differently across respondents. A defined time period and numerical categories may provide more interpretable information when actual frequency is important.
When AI is used to generate questions from research objectives, the response format should be considered alongside the information each question must collect.
Response Categories Should Be Mutually Exclusive When Only One Answer Is Allowed
For a single-choice categorical question, response options should generally be mutually exclusive. This means a respondent's answer should not fit into more than one category.
Consider a question asking respondents to indicate their years of teaching experience:
- 1–5 years
- 5–10 years
- 10–15 years
A teacher with exactly five years of experience could select either the first or second category. The categories overlap at their boundaries.
A revised set might use "Less than 5 years," "5 to less than 10 years," "10 to less than 15 years," and "15 years or more," assuming the study defines experience continuously and these intervals are appropriate.
AI can flag overlapping boundaries, but researchers should verify the categories against the actual variable definition. The choice of intervals may affect the detail available for analysis.
Response Options Should Cover Reasonable Answers
For a closed-ended question, the available categories should accommodate the reasonable answers respondents may provide. This is often described as making the response set exhaustive.
Suppose a questionnaire asks instructors which AI tools they use and lists only ChatGPT, Gemini, and Copilot. Respondents who use another tool cannot accurately represent their experience unless an appropriate alternative is available.
Depending on the question, adding "Other, please specify" or allowing multiple selections may improve coverage. However, an "Other" category should not become a substitute for identifying important categories relevant to the study.
The American Association for Public Opinion Research recommends careful selection of response categories that are mutually exclusive, cover reasonable answers, and follow a logical order. Its survey best-practice guidance also recognizes that options such as "Don't know" or "Does not apply" may be appropriate in particular circumstances.
AI can assist in identifying missing response categories, but researchers should check whether its suggestions reflect the actual population and study context.
Should You Use Five-Point or Seven-Point Rating Scales?
Generative AI often recommends a five-point agreement scale by default. Yet the number of response categories should be a deliberate measurement decision.
A five-point scale may be easier for some respondents to distinguish, while seven categories may allow finer reported distinctions when those distinctions are meaningful. Neither format is universally superior.
Researchers should consider the construct, respondents' ability to distinguish the categories, established instrument conventions, and the intended scoring approach.
The same consideration applies to the midpoint. "Neither agree nor disagree" may represent a neutral position, but it should not automatically be interpreted as uncertainty, lack of knowledge, or nonapplicability.
For Likert-scale items generated with AI, the response categories should fit the theoretical construct and the response task, not simply follow the model's preferred template.
Distinguish Neutral, Don't Know, and Not Applicable Responses
AI may treat several noncommittal response options as interchangeable, but they communicate different things.
| Response Option |
Possible Meaning |
When It May Be Appropriate |
| Neither agree nor disagree |
The respondent takes a neutral or middle position on the statement. |
When neutrality is a meaningful response. |
| Don't know |
The respondent lacks sufficient knowledge to answer. |
When respondents may genuinely be unable to judge. |
| Not applicable |
The question does not apply to the respondent's circumstances. |
When the activity or experience is not relevant to everyone. |
| Prefer not to answer |
The respondent chooses not to disclose an answer. |
When the question is sensitive or disclosure is optional. |
These distinctions matter during data analysis. A respondent who has never attended an AI training program should not automatically be classified as neutral about its quality.
Similarly, "Don't know" should not be coded as the midpoint of an agreement scale unless the instrument has an explicit and defensible reason for doing so.
Researchers should also distinguish explicit non-substantive responses from unanswered questions in their datasets.
How Should You Prompt AI to Suggest Response Options?
Provide the exact question, the information being measured, the target population, and any relevant analysis requirements. Ask AI to explain the suitability and limitations of its proposed categories.
Illustrative AI Prompt
Reviewing Response Options for a Research Questionnaire
"I am developing a questionnaire for university instructors about their use of generative AI in teaching. Review the following survey questions and recommend appropriate response formats. For each question, identify the variable being measured, suggest response categories, and explain whether respondents should select one answer, multiple answers, or provide a numerical response. Check whether single-choice categories are mutually exclusive and sufficiently comprehensive. Identify where 'None,' 'Don't know,' 'Not applicable,' or 'Prefer not to answer' may be needed. Do not automatically recommend a five-point Likert scale. Flag decisions that require information about the target population or intended analysis."
This prompt encourages AI to consider the measurement task rather than simply produce familiar answer labels.
The researcher should still examine the categories independently, particularly when the question concerns sensitive information or a specialized population.
Can Response Options Influence Survey Results?
Yes. The number, wording, order, and availability of response categories can affect how respondents answer.
Pew Research Center explains that closed-ended response options can shape survey findings because respondents react to the alternatives they are offered. The available choices may influence which considerations become salient and which answers appear acceptable. Its guidance on writing survey questions discusses these design effects.
For example, asking whether respondents are "Very satisfied," "Satisfied," or "Somewhat satisfied" offers only favorable evaluations. Respondents who are dissatisfied have no appropriate category.
Even when positive and negative categories are balanced, response order may influence selection. Depending on the survey mode and purpose, researchers may consider rotating or randomizing unordered categories.
AI can suggest such design choices, but they should be evaluated against established survey methodology rather than applied automatically.
Should You Use Select-All-That-Apply Questions?
Multiple-response questions are useful when respondents may legitimately belong to several categories or engage in several activities.
For example, instructors may use generative AI for lesson planning, creating instructional materials, developing assessments, and providing feedback. A single-choice format would be inappropriate if the researcher wants to identify every activity performed.
However, the method used to present multiple-response options can influence the answers collected.
Research published by Pew Research Center in 2019 compared select-all-that-apply questions with forced-choice formats that ask respondents to evaluate each option separately. The findings demonstrate that these formats can produce different endorsement patterns.
Consequently, the choice between a checklist and separate yes-or-no questions should reflect the research purpose, respondent burden, and evidence concerning the selected format.
Why Response Options Must Be Tested With Actual Respondents
AI can evaluate whether categories appear logically consistent, but it cannot establish that respondents interpret them as intended.
For example, different instructors may interpret "frequently" as several times per week, daily, or whenever a particular task arises. A response set that appears orderly may therefore produce inconsistent interpretations.
Cognitive interviewing can reveal how respondents distinguish categories, while pilot testing can identify practical problems with response selection and questionnaire administration.
These procedures are especially important when the response options have been substantially modified or the questionnaire is intended for a population different from the one for which an instrument was originally developed.
AI can support the broader questionnaire development process, but response-option design still requires human judgment and appropriate evaluation.
