Can a Field Publish Many Papers Before Basic Concepts Are Settled?

What if Researchers Are Producing Evidence Before Agreeing on What They Are Studying?

Some research fields look mature from a distance. They have thousands of papers, established journals, measurement instruments, systematic reviews, and increasingly specialized terminology.

Then you start reading closely.

Different authors define the central concept differently. Several instruments supposedly measure the same construct but contain substantially different items. Similar phenomena have different names, while identical terms sometimes refer to different phenomena. Studies are being compared even though their operational definitions barely resemble one another.

Can a field really accumulate a large literature before its basic concepts are settled? Yes, and recognizing this changes how that literature should be interpreted.

Publication Can Outpace Conceptual Development

How Research Accumulates When Its Concepts Are Still Unstable

Every Measurement Begins With a Conceptual Decision

Before researchers can measure something, they need some account of what that something is. A conceptual definition specifies the meaning and boundaries of a construct. An operational definition specifies how the construct will be represented or measured in a particular study.

These are related but not interchangeable. Giving participants a questionnaire does not settle what the underlying construct means. Nor does attaching a familiar label to a numerical score guarantee that the score represents the same concept examined elsewhere.

Methodological work on construct clarity has emphasized that clear conceptual definitions are fundamental to theory development and measurement. Podsakoff and colleagues, for example, describe conceptual definitions as important to scientific progress and identify problems that follow when constructs are inadequately defined.

One Label Can Hide Several Different Concepts

Suppose five research teams all claim to study “engagement.” One defines it as observable participation, another as emotional involvement, another as time spent using a platform, another as self-reported interest, and another combines behavioral, emotional, and cognitive dimensions into one score.

The papers may all contain the word engagement, but they are not necessarily investigating an equivalent construct.

This resembles what methodological literature calls the jingle fallacy: assuming that constructs are the same because they share a name. The reverse problem, often called the jangle fallacy, occurs when researchers treat constructs as different because they have different names even though they substantially overlap.

Both can fragment cumulative knowledge. A literature may appear to contain replications or contradictions that partly reflect differences in terminology rather than differences in the underlying phenomenon.

Researchers Can Build Measures Before the Construct Is Clear

Measurement does not rescue an unclear concept automatically. A scale can produce numerical scores, high internal consistency, and sophisticated statistical models while questions remain about what those scores represent.

This is why construct development normally requires attention to conceptual specification as well as empirical measurement. Reviews of scale-development practice have emphasized defining the construct clearly before generating and evaluating items.

If the conceptual domain changes from one study to another, apparently similar instruments may measure meaningfully different things. Conversely, different instruments may represent overlapping aspects of the same construct.

Construct Proliferation Can Make a Field Look More Developed Than It Is

Research fields sometimes generate increasingly large families of named constructs. Some distinctions are theoretically useful. Others may substantially overlap with existing concepts, differ mainly in terminology, or use familiar labels inconsistently.

This phenomenon is often discussed as construct proliferation. The problem is not simply that a field has many concepts. Scientific progress often requires conceptual differentiation. The concern arises when new constructs are introduced without adequately establishing how they differ from existing ones, or when the same label acquires incompatible meanings across research communities.

Recent methodological discussion has therefore argued that construct proliferation involves not only redundant labels but also semantic divergence and coordination problems in how constructs are defined and used.

Conceptual Instability Can Produce Apparent Empirical Disagreement

Imagine that one study finds a strong relationship between X and “engagement,” while another finds almost none. The obvious interpretation is that the empirical findings conflict.

But what if the first study measured behavioral participation and the second measured emotional involvement? The studies may be answering different questions under the same label.

Before treating inconsistent findings as substantive contradictions, researchers should therefore examine conceptual definitions, operationalizations, populations, contexts, and study designs. Some disagreement is genuinely empirical. Some is partly semantic or methodological.

Conceptual Problems Also Complicate Evidence Synthesis

Systematic reviews and meta-analyses depend on decisions about which studies are sufficiently comparable to address a common question. Conceptual ambiguity makes those decisions harder.

If studies define the central construct differently, a reviewer may need to separate conceptualizations, conduct subgroup analyses, restrict eligibility, or conclude that statistical pooling would obscure rather than clarify the evidence. A pooled number cannot by itself resolve disagreements about what was measured.

This helps explain why rapid growth in publication volume can exaggerate the appearance of evidential maturity. Many studies can exist while the basis for comparing them remains unstable.

Unsettled Concepts Are Not Automatically a Sign of Bad Science

Scientific concepts evolve. Researchers propose definitions, test their implications, identify exceptions, refine boundaries, distinguish related phenomena, and sometimes abandon concepts that no longer prove useful.

Perfect consensus is therefore neither realistic nor always desirable. Competing conceptualizations can generate productive theoretical debate.

The more consequential issue is whether researchers make those differences explicit. A field can progress amid disagreement when competing definitions are clearly articulated and empirically or theoretically examined. Progress becomes harder when incompatible definitions are treated as interchangeable or conceptual differences disappear behind identical terminology.

New Fields May Need Foundational Research Before Another Application Study

When a field is young, researchers may rush toward questions about predictors, outcomes, interventions, adoption, or effects. Yet the more useful study may concern the foundation: What exactly is the phenomenon? What distinguishes it from neighboring concepts? Which dimensions belong to it? How should it be measured?

This is especially relevant when new technologies generate research faster than reliable evidence can accumulate. Novel terminology may enter the literature before researchers have established stable definitions or measures.

Conceptual work is not merely preliminary housekeeping before the “real” empirical research begins. In some fields, clarifying what researchers are actually talking about may be the empirical literature's most pressing problem.

When 80 Studies Use the Same Word but Not the Same Construct

Common Mistakes When Reading a Conceptually Unsettled Literature

Check Conceptual Maturity Before Adding Another Study

When entering a large literature, do not begin by assuming that its terminology is settled. Read definitions closely, especially when different authors appear to use the same construct in different ways.

A useful exercise is to build a concept map before building your hypothesis model. Record how influential or relevant papers define the central construct, which dimensions they include, what they exclude, how they operationalize it, and which neighboring concepts they distinguish from it. Patterns that disappear in ordinary literature notes can become obvious very quickly.

This assessment can also help you judge whether an apparently exciting field is making cumulative progress. A productive emerging field and a research bubble can both generate many papers; one useful question is whether the underlying concepts, methods, and explanations are becoming more coherent as the literature grows.

Before Assuming a Field's Concepts Are Settled

Questions About Conceptual Clarity and Research Maturity

A Large Literature Can Rest on Unsettled Foundations

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.