03 · What You Need to Know
Understanding and Evaluating Coding Changes in Longitudinal Routine Data
What Counts as a Change in Coding Practices?
A coding practice is the set of rules, classifications, and operational procedures used to represent information in a record system. Coding translates clinical diagnoses, procedures, administrative events, or other observations into standardized or locally defined categories.
Changes can occur at several levels. An organization may adopt a different classification system, introduce new categories, revise instructions for selecting codes, or change how staff apply existing rules.
For example, healthcare organizations may transition between versions of the International Classification of Diseases (ICD), while universities may revise how they classify student enrollment, withdrawal, or course completion.
Not every coding change is a change in the coding system itself. An unchanged code may become more common because of staff training, new reimbursement incentives, automated documentation prompts, or revised institutional reporting expectations.
Consequently, researchers should investigate both formal classification changes and changes in how classifications are applied.
Distinguish Coding-System Changes from Coding-Practice Changes
Coding-system change
A revision or replacement of the formal classification structure, such as introducing new codes, removing categories, or changing code definitions.
Coding-practice change
A change in how existing or new codes are assigned, prioritized, documented, audited, or used in routine operations.
The distinction matters because a coding-system transition may be documented through official classification releases, while a change in local practice may be visible only in institutional procedures or the resulting data.
Both can affect the comparability of research variables.
How Coding Changes Produce Artificial Increases or Decreases
Suppose a condition was previously recorded under a broad category. A revised classification introduces several more specific codes, and staff receive additional training to identify the condition.
The number of recorded cases may increase because the new system captures cases that were previously grouped elsewhere or documented less frequently.
Alternatively, a new coding requirement might make a particular diagnosis harder to assign, producing an apparent decline even if the underlying condition remains equally common.
Changes in recording sensitivity, specificity, or coverage can affect observed trends. The resulting bias may be especially consequential when the coding transition coincides with a policy change, intervention, or other event being evaluated.
A recorded increase is therefore evidence of increased recorded occurrence. Establishing that the underlying phenomenon increased requires additional justification.
Why Code Mapping Does Not Always Establish Equivalence
Code mapping, sometimes called crosswalking, connects categories in one coding system to categories in another. It can help researchers harmonize records collected under different classification versions.
However, coding systems do not necessarily contain equivalent categories.
A code in an older system may correspond to several codes in a newer system. Conversely, multiple historical categories may map to one broader contemporary category. Some concepts have no satisfactory direct equivalent.
| Mapping relationship |
What it means |
Research implication |
| One-to-one |
One source code corresponds to one target code |
Potentially straightforward, but definitions still require verification |
| One-to-many |
One historical category corresponds to several newer categories |
Historical records may lack the detail needed to distinguish newer categories |
| Many-to-one |
Several historical categories correspond to one target category |
Harmonization may require loss of detail |
| No reliable equivalent |
The concept cannot be translated adequately |
Combined analysis may be inappropriate for that variable |
Official mapping resources are valuable starting points, but a technical crosswalk does not establish clinical or conceptual equivalence for every research purpose.
For example, the U.S. Centers for Medicare & Medicaid Services provides General Equivalence Mappings for historical ICD-9-CM to ICD-10-CM/PCS transitions. These mappings were developed to support translation between classifications and should not be interpreted as universal guarantees of identical meaning.
Examine Changes in Code Specificity and Diagnostic Criteria
A newer classification may distinguish conditions that an earlier system grouped together. This can improve specificity for recent records while making historical comparisons more difficult.
Suppose a researcher wants to examine a narrowly defined disease subtype. The subtype may have a dedicated code in recent years but may have been included within a broader category previously.
The researcher cannot simply assume that all historical cases in the broader category represent the subtype.
Possible responses include using a broader definition that is reasonably comparable across periods, restricting the study to years with sufficient coding detail, or presenting subtype analyses only for the later period.
The choice should depend on the research question and available evidence, not merely on which option retains the most observations.
Investigate Whether Coding Changes Affect Exposures, Outcomes, or Both
The consequences of coding changes depend on the analytical role of the affected variable.
When the primary outcome is affected, apparent changes in event rates may reflect altered outcome ascertainment. When exposure classification changes, participants may be assigned to different exposure groups under different recording rules.
Changes in covariate coding can also matter. A comorbidity measure may appear to increase because documentation improved, potentially affecting risk adjustment and comparisons across periods.
Researchers should examine the complete operational definitions used to identify exposures and outcomes from routine records, including their code sets, observation windows, and supporting criteria.
Even when the primary outcome code remains unchanged, alterations in related procedures or supporting documentation may affect how frequently it is assigned.
Use Descriptive Analyses to Detect Possible Coding Discontinuities
Plot the frequency or rate of relevant codes over time, ideally using intervals fine enough to examine known transition dates.
Compare patterns before and after the transition, including changes in:
- Overall event counts and population-adjusted rates.
- Individual code frequencies and broader code groups.
- Proportions of unspecified or residual categories.
- Missing or invalid codes.
- Patient, student, or service-user characteristics.
- Related indicators that may provide contextual evidence.
Where relevant, examine whether a discontinuity occurs across all institutions or only in particular settings.
For longitudinal datasets, these checks complement the broader assessment of whether routine data were collected consistently.
Descriptive discontinuities identify possible problems but do not independently establish their causes. A real external event occurring near the transition may produce a similar pattern.
Use Parallel Coding or Validation Data When Available
One useful source of evidence is a period during which records were classified under both the old and new systems. Such dual-coded or parallel-coded observations may help estimate how the transition affects classification.
For example, a sample of records can be independently classified using both coding specifications. Researchers can then compare the resulting case counts, classifications, and agreement.
Other validation strategies may involve chart review, external registries, or stable reference indicators.
The usefulness of these approaches depends on the quality and representativeness of the validation sample. A small or selectively chosen set of records may not capture the full range of coding differences.
When validation evidence is unavailable, uncertainty about the comparability of historical and contemporary classifications should remain explicit.
Choose an Appropriate Strategy for Longitudinal Analysis
There is no single statistical correction for every coding transition. The appropriate response depends on how the coding change affects the variable and whether its consequences can be estimated.
| Situation |
Possible strategy |
Important limitation |
| Definitions can be meaningfully harmonized |
Construct a common variable across periods |
Some detail may be lost |
| Transition date is known |
Model or stratify by coding period |
Period adjustment may not resolve changes in measurement validity |
| Historical categories are less specific |
Use a defensible broader outcome or restrict years |
Reduced specificity or shorter observation period |
| Dual-coded validation records are available |
Estimate classification differences and assess possible calibration |
Requires assumptions about transportability |
| Comparability cannot be established |
Analyze periods separately or revise the research question |
Continuous longitudinal inference may not be defensible |
Segmented regression or interrupted time-series methods may help characterize changes around a known transition. However, a statistical break at the coding date cannot automatically be attributed entirely to coding, particularly when other changes occurred simultaneously.
Including a simple indicator for the coding period may absorb some level differences, but it does not necessarily correct misclassification or changes in the meaning of the underlying variable.
Recognize the Implications for Policy and Intervention Evaluation
Coding transitions are particularly problematic when they coincide with the implementation of a policy or intervention.
Suppose a new clinical protocol begins in the same month that an electronic coding system introduces additional diagnostic prompts. A subsequent increase in recorded diagnoses could reflect the intervention, the documentation change, or both.
If the two changes occur together and no additional information is available, their separate contributions may not be identifiable from the observed time series alone.
Potential approaches include identifying unaffected comparison outcomes, examining external validation sources, studying settings with different transition dates, or narrowing the causal claim.
These approaches require substantive justification. A comparison series is useful only if its own measurement process and relationship to the intervention are appropriate.
Document Historical Coding Decisions Transparently
Researchers should preserve the classification versions used, effective dates, code lists, mapping resources, exclusion rules, and any changes made to analytical definitions.
Explain whether harmonization was based on official crosswalks, clinical or subject-matter expertise, empirical validation, or a combination of these sources.
When analyses use routinely collected health data, the RECORD reporting statement provides relevant guidance on describing codes, algorithms, and database characteristics.
Transparent documentation allows readers to distinguish changes in the underlying phenomenon from changes in its recorded representation, even when some uncertainty cannot be eliminated.
04 · A Practical Example
Evaluating a Diagnosis Trend Across a Coding-System Transition
Hypothetical Example
A Sudden Increase in Recorded Hospital Diagnoses
A researcher examines annual hospital records to investigate changes in the frequency of a particular diagnosis. The hospital adopted a revised diagnostic coding classification at the beginning of the fourth observation year.
Before the transition, approximately 120 qualifying diagnoses were recorded per 10,000 admissions annually. In the first year after the transition, the rate rises to 175 per 10,000 admissions.
The researcher initially considers interpreting this as an increase in disease occurrence.
Step 1: Identify the coding transition.
Review the classification versions, implementation date, and instructions used to identify the diagnosis in each period.
Step 2: Examine the code mapping.
Determine whether the newer classification contains categories that were previously grouped with other conditions. Identify ambiguous or non-equivalent mappings.
Step 3: Investigate the observed increase.
Compare diagnosis rates by department, relevant patient characteristics, and related clinical indicators. Review whether documentation or coding incentives also changed.
Step 4: Evaluate comparability.
Assess whether the historical and contemporary definitions identify sufficiently similar clinical events. If available, use independently reviewed records or parallel coding to examine classification differences.
Step 5: Select a defensible analysis.
If harmonization is adequately supported, analyze the comparable definition across years. Otherwise, consider period-specific estimates or a shorter series using the newer definition.
The observed difference is 55 recorded diagnoses per 10,000 admissions, an approximately 45.8% increase relative to the earlier rate. That arithmetic describes the change in recorded rates, not necessarily the change in underlying disease occurrence.
Without sufficient evidence about classification comparability, attributing the full increase to disease incidence would be unjustified.
