Were the Variables Measured in the Way Your Question Requires?

The variable exists, but does it actually measure what you need?

You have checked the dataset and found variables corresponding to the main concepts in your research question. That seems promising. Yet a variable called “academic performance” might represent a standardized test score, self-reported grades, cumulative GPA, or performance in a single course. A variable called “technology use” might mean daily screen time, access to a device, frequency of internet use, or use of a particular digital tool.

Those measures are related to the same broad concepts, but they are not interchangeable. The question is therefore no longer simply whether the dataset contains the variables you need. You must determine whether those variables were measured in a way that permits you to answer the question you actually intend to ask.

Variable availability and measurement suitability are different tests

Match the intended construct to the measurement that actually occurred

When collecting primary data, researchers can choose or design measures around their research question. Secondary analysis reverses part of that process. The measurements have already been made, often for purposes different from yours. You inherit those measurement decisions along with the data.

Methodological guidance on secondary analysis therefore emphasizes examining the original assessment tools, questionnaires, codebooks, sampling procedures, coding decisions, and other documentation. Researchers also need operational definitions for the variables they intend to analyze. The fact that a dataset contains a conveniently named column does not establish that the underlying measure corresponds to the construct required by a new study.

Start with the construct, not the variable label

Suppose you want to investigate whether frequent use of generative AI for completing university coursework is associated with academic performance.

You find a variable labelled AIUSE. Before celebrating this unusually cooperative codebook, ask what AIUSE means operationally.

Perhaps respondents were asked, “Have you ever used an artificial intelligence application?” with yes/no response options. That variable does contain information about AI use. But it does not directly measure frequency, use for coursework, use of generative AI specifically, or use during the period corresponding to the academic outcome.

Your conceptual requirement and the available measure therefore differ.

The variable may still support a different research question. What it cannot do is acquire information that was never collected merely because your proposed interpretation requires it.

Inspect the exact question wording and response options

For survey data, the questionnaire is often as important as the data file. SAMHSA's guidance on codebooks notes that documentation may contain exact survey questions, response codes, missing-data codes, skip patterns, frequencies, concept definitions, and information about data collection and quality.

Read the exact wording whenever possible. Small differences can materially alter interpretation:

• “Do you use generative AI?” differs from “Did you use generative AI during the past seven days?”
• “How satisfied are you with your course?” differs from a validated multi-item measure of student engagement.
• “Highest qualification” differs from years of education.
• Self-reported height differs from measured height.
• Current employment differs from employment at any point during the previous year.

Response categories matter as well. A five-category frequency measure may support analyses that a binary yes/no variable cannot. Conversely, collapsing detailed categories during analysis can sometimes be defensible, whereas recovering detail that was never collected is impossible.

Check the source of the measurement

Two variables can appear to represent the same construct while coming from different sources. Academic achievement could come from an administrative record, a standardized assessment, teacher evaluation, or participant self-report. Health status might come from clinical examination, medical records, or respondents' perceptions.

The appropriate source depends on what your research question means by the construct. Self-report is not automatically inferior, nor are administrative records automatically superior. They may simply represent different kinds of evidence and carry different forms of measurement error.

Your interpretation should remain faithful to the source. If the variable records self-reported GPA, for example, describing it simply as “GPA” can obscure a relevant measurement qualification.

Check the reference period

Time is part of measurement. “Currently,” “during the past week,” “during the previous 12 months,” and “ever” define different variables even when the substantive topic is identical.

This becomes particularly important when variables are related analytically. If a study asks whether an exposure predicts a later outcome, you need to understand when each measure was collected and what period it describes. A current outcome paired with an exposure referring to an unspecified lifetime period may not support the temporal interpretation implied by the research question.

The broader feasibility issue of whether the dataset covers the period your question requires should therefore be distinguished from the reference period built into an individual measure. A longitudinal dataset may span ten years while a particular variable still ask only about behavior during the previous seven days.

Understand how constructed variables were created

Not every variable comes directly from a single question. Datasets commonly include composite scores, indices, classifications, recoded variables, scale scores, standardized measures, and variables derived from several source items.

For a constructed variable, inspect its derivation. Ask which source items were used, how responses were combined, whether weighting or transformations were applied, how missing items were handled, and what higher or lower scores represent.

ICPSR's codebook guidance specifically recommends documenting how compiled or constructed variables were created. Without that information, a plausible-looking score may be difficult to interpret responsibly.

Do not assume a measure remains comparable across waves

Longitudinal datasets create another complication. A variable may appear in several waves without being measured identically each time.

Question wording can change. Response categories may be expanded or collapsed. Diagnostic criteria can be revised. Data-collection modes may change. A scale may be replaced or recoded.

Methodological guidance on secondary analysis specifically warns researchers to examine questionnaires and codebooks when key variables are combined across longitudinal waves because assessment and coding methods may change over time.

If your research question involves trends or change, do not assume that identical or similar variable labels establish measurement equivalence.

Watch for skip patterns and conditional questions

A variable can look as though it contains substantial missing data when some respondents were never supposed to answer the question.

Imagine a survey that first asks whether respondents have ever used generative AI. Only those answering yes receive questions about frequency. A blank frequency value among nonusers is structurally different from a user refusing to answer the frequency question.

Codebooks often document these skip patterns. Understanding them is necessary both for interpreting the measure and for later evaluating whether missing data threaten the study.

A proxy variable requires an argument, not merely a resemblance

Sometimes the exact construct you want was not measured, but a related variable is available. Using a proxy is not inherently inappropriate. The problem arises when the proxy and the intended construct quietly become synonymous in the manuscript.

Suppose household income is unavailable but eligibility for a means-tested program is recorded. That indicator may provide information related to socioeconomic circumstances, but it does not become household income. Whether it is a defensible proxy depends on the research question, institutional context, eligibility rules, theoretical rationale, and claims you intend to make.

Measurement suitability is about the claim you want to make

There is rarely a meaningful answer to “Is this a good variable?” without specifying the purpose.

A binary employment indicator may be perfectly adequate for estimating the proportion of participants currently employed. It would be inadequate for a question about job quality. A single age variable may suffice for age adjustment but not for a question requiring developmental stages defined by additional criteria.

Suitability is therefore relational: this measure, for this construct, in this population, for this analysis and interpretation.

When the variable exists but the research question still does not fit

The difficulty is not statistical. No regression model can recover the distinction between generative AI and other digital tools if the original measurement never recorded it.

Common mistakes when interpreting existing measures

Audit each important measure before finalizing the question

Once you identify a candidate variable, trace it back to its measurement source. For a survey item, read the questionnaire. For a test score, inspect documentation describing the assessment and scoring. For administrative data, determine how the field was generated. For a constructed variable, examine the derivation rules.

This evaluation should happen before the research question becomes difficult to change. Where possible, inspect the data before finalizing the research question, while preserving a clear distinction between feasibility checking and searching opportunistically for interesting results.

If the available measurement forces a substantial change in the construct, say what the study now investigates rather than retaining the more appealing original terminology. A narrower question that the data can answer is methodologically stronger than a broader question supported mainly by wishful variable naming.

Check what each important variable really measures

Questions about measurement in existing datasets

The data can support only the construct that was actually measured

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.