03 · What You Need to Know
How to Generate Questionnaire Items From Appropriate Sources
Why Does the Source of Questionnaire Items Matter?
Questionnaire items are intended to provide observations about a defined construct. Their wording and content influence which experiences, judgments, beliefs, or behaviors contribute to the resulting scores.
When items are generated without a clear rationale, an instrument may omit essential dimensions or include content that belongs to neighboring constructs.
For example, a questionnaire intended to measure academic self-efficacy might include statements about motivation, study frequency, and examination grades. These attributes may be associated with self-efficacy, but their inclusion requires more justification than a general relationship with academic success.
Item provenance helps researchers establish why particular content was considered and how it relates to the intended measurement domain.
However, a reputable source does not automatically make an item appropriate. An item taken from a well-established scale may be unsuitable for a different construct, population, or research purpose.
What Are the Main Sources of Questionnaire Items?
Item generation may draw on several kinds of evidence. Their relative importance depends on whether the construct is well established, emerging, population-specific, or closely connected to observable experiences.
| Item Source |
Potential Contribution |
Important Limitation |
| Theoretical literature |
Identifies conceptual attributes, dimensions, and expected manifestations |
Theory may not specify language that intended respondents understand |
| Existing instruments |
Provides established item formulations and operational approaches |
Items may be unsuitable for the new population or intended construct |
| Empirical research |
Identifies documented behaviors, experiences, and construct relationships |
Published studies may omit relevant experiences or populations |
| Qualitative interviews |
Reveals participants' language, interpretations, and relevant experiences |
Findings depend on sampling, interviewing, and analytical procedures |
| Focus groups |
Supports discussion of shared and contrasting experiences |
Group interaction may discourage disclosure or privilege dominant perspectives |
| Subject-matter experts |
Contributes disciplinary knowledge and identifies overlooked content |
Expert assumptions may differ from target-population experiences |
| Observations and task analysis |
Identifies behaviors or activities relevant to the measurement domain |
Observable behavior may not represent subjective experiences directly |
| Researcher-generated proposals |
Translates the construct blueprint into candidate items |
Requires independent scrutiny of conceptual relevance and wording |
No single source is mandatory for every instrument. The appropriate combination should provide adequate coverage and support the intended interpretation.
What Is Deductive Item Generation?
Deductive item generation begins with a theoretical or conceptual definition and derives proposed items from its specified attributes or dimensions.
For example, a researcher developing a questionnaire about academic self-efficacy may begin with a theoretically grounded definition of confidence in performing academic tasks.
Candidate items can then be developed for particular task domains, such as planning academic work, understanding difficult material, or completing demanding assignments.
This approach can help maintain conceptual coherence because each item is connected to an established content specification.
However, theory-derived wording may be abstract, overly technical, or insufficiently sensitive to the target population's experiences.
Deductive item generation therefore benefits from subsequent review of relevance and comprehensibility.
What Is Inductive Item Generation?
Inductive item generation develops candidate content from empirical observations, especially qualitative accounts of the phenomenon.
Researchers may interview intended respondents, examine relevant experiences, and identify recurring or theoretically meaningful attributes.
For example, students discussing academic help-seeking may describe hesitation, perceived judgment, difficulty identifying appropriate support, or uncertainty about when assistance is needed.
These accounts may suggest candidate item content that existing theoretical models or questionnaires do not adequately represent.
However, inductive generation should not be reduced to converting every interview quotation into a questionnaire statement.
The researcher must interpret the evidence, establish its relevance to the construct, and determine whether it represents an important aspect of the intended measurement domain.
Should Deductive and Inductive Approaches Be Combined?
They can be complementary.
A deductive approach helps ensure that item generation remains connected to theory. An inductive approach may reveal how the construct is experienced and expressed by the intended population.
For example, theory may identify self-monitoring as a component of self-regulated learning, while student interviews reveal particular monitoring practices in asynchronous courses.
Combining these sources may help generate items that remain conceptually grounded while reflecting relevant contexts.
Nevertheless, using both approaches is not automatically necessary. The decision depends on the maturity of the construct, the availability of existing evidence, and the purpose of the instrument.
Can You Borrow Items From Existing Questionnaires?
Potentially, subject to the instrument's licensing and copyright conditions.
Existing items may provide useful models for operationalizing a construct, particularly when their development and measurement evidence are documented.
However, researchers should distinguish between adopting an established instrument, adapting individual items, and using existing items as conceptual inspiration for new wording.
These activities have different implications for attribution, permissions, scoring, and comparability.
An item that functions appropriately within one scale may not retain the same interpretation when removed from its original context or combined with items from other instruments.
Researchers should therefore review original sources and document the provenance of borrowed or adapted content.
Can You Combine Items From Several Existing Scales?
Yes, but doing so does not automatically create a coherent or validated instrument.
Items from different scales may reflect different construct definitions, recall periods, response formats, or scoring assumptions.
For example, one motivation scale may assess reasons for engaging in academic work, while another assesses perceived importance or confidence.
Combining their items into a single total score could blur theoretically distinct attributes.
Researchers should first establish a common construct framework and evaluate whether each item contributes appropriately to that framework.
Where existing scales are suitable, administering them separately may be more defensible than combining selected items into an untested composite.
How Can Qualitative Interviews Inform Item Development?
Qualitative interviews can help identify relevant experiences, language, distinctions, and circumstances that should inform item content.
Researchers may ask participants to describe how they experience the construct, what behaviors or judgments characterize it, and which situations make it particularly salient.
For example, interviews about online learning autonomy may reveal differences between choosing when to study, deciding which resources to use, and determining how to complete learning tasks.
These distinctions could inform separate candidate dimensions or items.
The quality of the resulting evidence depends on the interview questions, participant selection, analysis, and interpretation.
Researchers should also seek experiences that challenge their initial conceptual assumptions rather than collecting only examples that confirm the proposed framework.
Should Focus Groups Be Used Instead of Individual Interviews?
Focus groups can be useful when interaction among participants helps reveal shared understandings, disagreements, or context-specific terminology.
Individual interviews may be preferable when the topic is sensitive, experiences are highly personal, or respondents may hesitate to disagree publicly.
Neither approach is universally superior.
For example, a focus group may help identify common terminology used by students to describe online collaboration. Individual interviews may better reveal experiences of exclusion or academic anxiety that participants are reluctant to discuss in a group.
The method should be selected according to the type of content evidence needed.
What Role Should Experts Play in Generating Items?
Subject specialists can help identify theoretically important content, clarify distinctions among dimensions, and detect omissions in the proposed item pool.
Measurement specialists may contribute expertise in item wording, response formats, scoring, and psychometric design.
These forms of expertise are complementary but not identical.
For example, an educational technology specialist may recognize important dimensions of AI-supported learning, while a psychometrician may identify problems with double-barreled statements or inconsistent response categories.
However, experts should not be treated as substitutes for the intended respondents.
Expert-generated items still require appropriate evaluation of comprehension and response processes.
The separate question of expert review before pilot testing concerns how proposed items should be evaluated after an initial pool has been developed.
Can Researchers Write Items Based on Their Own Experience?
Researcher experience can be a legitimate source of candidate ideas, particularly when the researcher has substantial knowledge of the relevant population or phenomenon.
For example, an instructor may recognize recurring difficulties students encounter when evaluating online information.
However, personal experience is situated and may not represent the full construct domain or the experiences of other participants.
Researcher-generated items should therefore be treated as proposals requiring conceptual justification and independent review.
The researcher's familiarity with a setting may improve contextual sensitivity, but it does not eliminate the need to examine alternative experiences and interpretations.
Can AI Tools Be Used to Generate Questionnaire Items?
Generative AI systems can assist with brainstorming candidate wording, identifying possible ambiguities, or producing alternative formulations from a researcher-supplied construct specification.
However, generated items should not be assumed conceptually appropriate, original, unbiased, or psychometrically sound.
AI-generated suggestions may combine neighboring constructs, reproduce familiar questionnaire patterns without clear provenance, or introduce assumptions unsupported by the research literature.
Researchers should independently verify theoretical claims, review item content, and evaluate the resulting questions with relevant experts and intended respondents.
Confidential research material or identifiable participant information should not be submitted to external AI services without appropriate authorization and safeguards.
AI assistance may support drafting, but it does not replace the scholarly responsibility for defining the construct and justifying the instrument's content.
How Many Candidate Items Should Be Generated Initially?
There is no universally correct number of initial items or fixed ratio of candidate items to retained items.
An initial pool should be sufficiently broad to represent the intended content domain and allow refinement without eliminating essential dimensions.
However, generating large numbers of nearly identical items may add review burden without improving coverage.
The appropriate pool size depends on construct breadth, dimensionality, intended instrument length, item format, and planned evaluation procedures.
For example, a narrow attitude scale may require fewer distinct content areas than a comprehensive assessment of several competencies.
Researchers should prioritize meaningful coverage over numerical abundance.
How Do You Know Whether the Initial Item Pool Covers the Construct?
Compare the candidate items with the construct blueprint.
Each important dimension should have appropriate representation, while items outside the intended domain should be identified.
A content matrix can help researchers examine this relationship systematically.
| Proposed Dimension |
Candidate Item Content |
Evidence Source |
Review Question |
| Planning |
Setting learning goals before beginning a task |
Theoretical literature and student interviews |
Does the item represent planning rather than general motivation? |
| Monitoring |
Checking comprehension while studying |
Existing research and qualitative accounts |
Does the wording describe a recognizable monitoring activity? |
| Reflection |
Evaluating the effectiveness of completed learning strategies |
Theoretical framework and expert input |
Does the item distinguish reflection from immediate monitoring? |
| Effort regulation |
Maintaining task-directed effort when learning becomes difficult |
Empirical studies and target-population experiences |
Does the item measure regulation rather than simply perceived difficulty? |
This hypothetical matrix illustrates how proposed item content can be connected to conceptual dimensions and evidence sources.
It does not establish that the dimensions are empirically distinct or that the eventual items will function as intended.
Should Every Item Have a Documented Source?
Each item should have a traceable rationale, although that does not mean every original item requires a separate published citation.
An item may originate from a theoretical proposition, an interview theme, a documented observation, or a researcher-generated formulation based on the construct blueprint.
The development record should distinguish direct adoption, adaptation, and original generation.
For adapted items, document the original source and the nature of the modification.
For newly generated items, record the conceptual dimension and evidence that motivated the content.
Such documentation supports transparent review and helps explain why items were retained, revised, or removed.
How Should Copyright and Permissions Be Handled?
Published questionnaire items may be protected by copyright or distributed under specific licensing conditions.
Researchers should verify the instrument's terms before reproducing, translating, modifying, or redistributing its content.
Availability in a journal article or online repository does not automatically establish unrestricted reuse rights.
Some instruments permit academic use or adaptation under specified conditions, while others require authorization.
The relevant legal and licensing requirements vary by instrument and jurisdiction.
Researchers should retain permission records where necessary and provide appropriate attribution. Citation and permission address related but distinct obligations.
What Makes a Candidate Questionnaire Item Well Written?
A candidate item should communicate a sufficiently clear task or judgment that is relevant to the intended construct and appropriate for the target population.
Researchers should examine whether the wording is ambiguous, unnecessarily technical, leading, double-barreled, or dependent on assumptions respondents may not share.
For example, the statement "I confidently and accurately evaluate AI-generated references" combines perceived confidence with claimed accuracy.
A respondent might feel confident without actually evaluating references accurately, making the response difficult to interpret.
Separating these attributes or selecting a performance assessment may be more appropriate, depending on the intended construct.
Item wording should also align with the response options. A frequency response scale may not be appropriate for a statement asking respondents to evaluate the importance of an activity.
Should Items Be Written Positively and Negatively?
Item direction is a design consideration rather than a universal requirement.
Reverse-worded items are sometimes introduced to address acquiescent responding, but they can also create comprehension problems or method-related factors.
The researcher should consider whether the intended respondents can interpret the wording reliably and whether reverse wording serves a justified measurement purpose.
Adding negative statements simply to balance the questionnaire does not automatically improve its validity.
The separate issue of whether to include reverse-worded questionnaire items requires attention to response processes and the instrument's measurement model.
How Should Candidate Items Be Organized Before Expert Review?
Items should be organized according to their intended dimensions and accompanied by sufficient information for reviewers to assess their relevance.
A useful item-development record may include an item identifier, proposed wording, source, intended dimension, conceptual rationale, response format, and revision history.
Researchers may also document alternative formulations and unresolved questions about particular items.
This information helps reviewers evaluate content rather than relying on isolated statements without conceptual context.
The process should remain transparent even when the final instrument contains only a small subset of the original candidate pool.
What Happens After the Initial Item Pool Is Generated?
Item generation is followed by evaluation and refinement.
Relevant activities may include expert review, target-population consultation, cognitive interviewing, pilot testing, and empirical assessment of measurement properties.
These activities address different questions.
Expert review may examine conceptual relevance and coverage. Cognitive interviewing can investigate how respondents understand and answer items. Pilot testing can reveal administration and feasibility problems.
Subsequent statistical analyses may evaluate internal structure, reliability, and other properties appropriate to the intended use.
The sequence should reflect the measurement design rather than treating every stage as a mechanical requirement.
Importantly, item generation does not establish that the proposed items are valid. It creates candidates for systematic evaluation.
Watch Out
Do not construct a questionnaire by collecting attractive items from unrelated scales and assuming that expert approval or a high reliability coefficient will make the resulting total score meaningful. Every item should be connected to a clearly defined construct, and the instrument's content and scoring require appropriate evaluation.
