Can Generative AI Generate Likert-Scale Items for a Research Questionnaire?

Can AI Write Likert-Scale Items That Are Suitable for Research?

You need to measure a construct such as academic self-efficacy, technology acceptance, or research anxiety. You ask generative AI to produce 20 Likert-scale items, and within seconds, it returns a list of polished statements accompanied by a five-point agreement scale.

The statements appear relevant, the wording is consistent, and the response options look familiar. You may even ask AI to organize the items according to theoretical dimensions.

But does generating statements in a Likert format mean you have developed an appropriate measurement scale? More importantly, how can you determine whether those statements actually represent the construct you intend to measure?

Yes, AI Can Draft Likert Items, but Their Measurement Quality Must Be Evaluated

How Generative AI Can Support Likert-Scale Item Development

Understand the Difference Between a Likert Item and a Likert Scale

A Likert-type item typically presents a statement and asks respondents to indicate their degree of agreement or another ordered response. A Likert scale, in the stricter psychometric sense, combines responses to multiple items intended to measure an underlying attribute.

For example, "I feel confident conducting statistical analyses" could be presented with response options ranging from strongly disagree to strongly agree. By itself, this is a Likert-type item. Combining several theoretically related items into a score may produce a scale, but only when the scoring and interpretation are justified.

This distinction is important because AI can generate many Likert-type items without establishing that they form a coherent scale. Simply calculating the average of their responses does not resolve that problem.

Define the Construct Before Asking AI to Generate Items

The quality of candidate items depends substantially on how clearly the researcher defines the construct.

Consider academic self-efficacy, which concerns students' beliefs about their capabilities to perform academic tasks. If you ask AI to generate items about academic self-efficacy without specifying the construct definition, it might produce statements about confidence, motivation, enjoyment, study habits, and academic achievement.

Although these concepts may be related, they are not interchangeable. A statement such as "I enjoy studying difficult subjects" primarily concerns enjoyment rather than perceived capability. Including it without justification could weaken the conceptual coherence of the intended scale.

Before generating items, establish the construct's definition, boundaries, and dimensions using relevant theory and literature. Boateng and colleagues' scale-development primer emphasizes identifying the domain of measurement before generating an item pool.

Researchers can use AI to translate research objectives into candidate questions, but construct definitions should guide which statements are retained.

What Information Should You Give AI?

A prompt should identify the target population, construct definition, intended dimensions, response format, and wording constraints. Where relevant, it should also identify the theoretical framework and the situations respondents are expected to evaluate.

This prompt limits the model's freedom to introduce unrelated concepts. It also encourages explicit justification, which can help the researcher identify weak or redundant suggestions.

For self-efficacy, capability-focused response options may sometimes be more directly aligned with the construct than general agreement categories. The response format should therefore be selected deliberately rather than assumed.

How Do You Evaluate Whether an AI-Generated Item Measures the Intended Construct?

Start by examining the meaning of the statement. Does it describe the attribute being measured, or does it refer to a related outcome, preference, behavior, or condition?

For example, consider the following hypothetical items proposed for academic self-efficacy.

The evaluations are illustrative judgments, not empirical findings. Even items classified as potentially aligned would require appropriate review and testing.

This is also why a professional-looking questionnaire can measure the wrong construct. Surface relevance does not necessarily establish conceptual accuracy.

Should You Use a Five-Point or Seven-Point Response Scale?

Generative AI often defaults to five-point agreement categories because they are familiar and easy to generate. However, there is no universal rule that five response categories are always preferable to seven, or that more categories automatically improve measurement.

The choice depends on the construct, respondent population, cognitive demands, established measurement practices, and intended interpretation.

Agreement responses are not always the most suitable format. For instance, asking how confident a student is that they can perform a particular academic task may provide a more direct expression of perceived capability than asking whether they agree with a general statement.

AI can help suggest response options, but researchers should verify their conceptual fit and test whether respondents distinguish the categories as intended.

Watch for Items That Combine Multiple Ideas

AI-generated items can appear polished while containing more than one judgment.

Consider the statement: "I can understand complex readings and explain difficult concepts to my classmates."

A student may feel capable of understanding the material but not confident explaining it to others. Combining the activities makes the response difficult to interpret.

Separating them into two candidate items may improve clarity, provided both activities belong within the construct being measured.

Researchers can ask AI to identify double-barreled questions and suggest revisions. The researcher must still determine whether the resulting statements remain conceptually appropriate.

Should AI Generate Positively and Negatively Worded Items?

AI can easily produce positive and negative versions of statements. For example, it might transform "I can complete difficult assignments" into "I cannot complete difficult assignments."

However, automatically reversing item wording is not necessarily beneficial. Negatively worded statements can introduce confusion, particularly when they contain complicated negations or when respondents answer quickly.

Some researchers use reverse-keyed items to address response tendencies, but this strategy can introduce wording effects that complicate interpretation. Researchers should not assume that a balanced number of positive and negative statements guarantees better measurement.

If reverse-keyed items are retained, their meaning, scoring, and empirical behavior should be examined carefully. A negatively worded statement is not automatically a valid reverse indicator of the same construct.

How Many Items Should AI Generate?

There is no universal number of candidate items that AI should generate. During initial scale development, researchers often prepare a broader item pool than they expect to retain, allowing weaker or redundant items to be removed through subsequent evaluation.

The size of that pool should reflect the construct's breadth, its proposed dimensions, and the intended scale length. Generating 50 nearly identical statements about one aspect of a construct does not compensate for omitting other important aspects.

AI can suggest alternatives, but item retention should be based on conceptual coverage and appropriate evidence rather than an arbitrary target number.

Why AI Cannot Establish Reliability or Validity by Itself

Item generation is an early activity within a broader scale-development process. Boateng and colleagues distinguish the generation and content assessment of items from later procedures involving pretesting, administration, dimensionality, reliability, and validity evaluation.

AI may claim that its generated statements are valid because they correspond to a theoretical definition. Such an explanation is not equivalent to independent content-validity evidence.

Similarly, asking AI to estimate Cronbach's alpha without actual response data cannot establish the internal consistency of a scale. Any numerical result invented from the wording alone would lack an empirical basis.

Even a high alpha obtained from real data would not independently demonstrate validity or unidimensionality. Reliability evidence and validity evidence address related but distinct measurement questions.

The broader question of whether AI can validate a research questionnaire requires careful attention to what evidence is being collected and what interpretations it can support.

Using AI to Develop Candidate Academic Self-Efficacy Items

The result of the AI interaction is an initial item pool, not an established academic self-efficacy scale. The final items and scoring approach must be justified through the development and evaluation process.

Notice that the researcher does not retain items merely because AI grouped them under the correct construct. The conceptual definition determines what belongs in the pool, while subsequent evidence informs which items are suitable for the intended measurement.

Common Mistakes With AI-Generated Likert-Scale Items

Use AI to Expand and Critique the Item Pool, Not to Certify the Scale

AI is most useful when you have already defined the construct and need assistance generating alternative statements, detecting possible wording problems, or comparing response formats.

Before developing a new scale, determine whether an existing instrument adequately measures your construct and whether it can be used for your population and purpose. Developing a new measure is not automatically preferable simply because AI makes item generation convenient.

Keep a record of the generated items, researcher revisions, and reasons for retaining or rejecting statements. This documentation helps distinguish AI assistance from the methodological decisions that determine the final instrument.

Most importantly, remember that item generation and scale validation are different activities. A fluent set of statements may be a useful starting point, but it does not establish the quality of the scores researchers intend to interpret.

Before Using AI-Generated Likert-Scale Items

Frequently Asked Questions About AI-Generated Likert Scales

AI Can Generate Likert Items, but It Cannot Make Them Valid by Writing Them

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.