03 · What You Need to Know
How to Evaluate Alignment Between a Registry and Its Target Population
Distinguish the Target Population From the Registry Population
The target population is the population about which you want to draw conclusions. It is defined by the scientific purpose of your study, not simply by the records available to you.
The registry population consists of individuals or units captured by a particular registration system during a specified period. It may represent all eligible members of a defined population, a substantial subset, or a highly selected group.
There may also be an accessible population, meaning the people or units that participating institutions or data systems could potentially identify and include. The final analytic sample may be narrower still because of study-specific eligibility criteria, missing variables, or exclusions.
Target population
The population to which the researcher intends the findings to apply.
Registry population
The individuals or units captured by the registry under its actual coverage and enrollment rules.
Analytic sample
The subset of registry records ultimately included in a particular analysis.
These populations should not be treated as interchangeable. A registry may capture nearly all eligible individuals within its participating institutions while covering only a small or specialized portion of the broader target population.
For example, complete records from one tertiary hospital do not automatically represent patients treated in primary care facilities. Likewise, complete records from one university do not necessarily represent students across an entire national higher education system.
Start by Defining the Population You Want to Make Claims About
Before evaluating registry coverage, write a precise description of the intended target population.
That description may include geographic boundaries, institutional settings, demographic or clinical characteristics, relevant experiences, eligibility conditions, and the time period of interest.
Consider the difference between these two statements:
- All undergraduate students enrolled in higher education institutions in a country.
- First-year undergraduate students enrolled in participating private universities during a specified academic year.
The second population is narrower, but it may correspond more closely to an institutional registry assembled from participating private universities.
Neither definition is inherently preferable. The appropriate choice depends on the research question and the intended contribution.
Researchers should not automatically broaden their population claims merely because the registry contains many records. Population scope must be justified conceptually and empirically.
Examine the Registry's Geographic and Institutional Coverage
Geographic and institutional coverage are often among the most visible sources of population mismatch.
A registry might include hospitals from major cities but few rural facilities. An educational database might contain students from institutions using a particular learning management system while excluding those using alternative platforms.
Such differences can matter when geography or institutional characteristics are associated with the outcome being studied.
For example, internet connectivity, instructional resources, student support services, and institutional admission practices may differ across universities. If these characteristics influence both exposure to online learning and academic outcomes, findings from a narrowly covered registry may not transfer directly to institutions outside its coverage.
Document which locations and institutional types are included, which are absent, and whether their exclusion is relevant to the intended inference.
Compare Eligibility Criteria and Population Definitions
Registries frequently apply enrollment rules established for administrative, clinical, surveillance, or research purposes. These rules may not match the eligibility requirements of your new study.
A disease registry might include only laboratory-confirmed cases. A researcher interested in all individuals with symptoms consistent with the disease would therefore be studying a broader target population than the registry actually captures.
Similarly, a student registry that records only full-time students may be unsuitable for estimating outcomes among all students when part-time enrollment is common and associated with the outcome.
Review the original inclusion and exclusion criteria, diagnostic or administrative definitions, and changes in eligibility over time.
If these criteria changed, assess whether the registry population also changed in ways that affect comparability. Changes in registry definitions over time can create apparent population trends that partly reflect recording practices rather than genuine changes in the underlying population.
Distinguish Coverage Problems From Participation Problems
Coverage and participation are related but conceptually different.
A coverage problem occurs when some members of the target population fall outside the registry's inclusion infrastructure. They may live outside the covered area, attend nonparticipating institutions, or receive services through systems that do not contribute records.
A participation problem occurs when eligible individuals within the registry's coverage are not actually enrolled, recorded, or retained.
| Population issue |
Illustrative situation |
Research implication |
| Undercoverage |
Only urban hospitals contribute data to a registry intended to represent national care. |
Patients treated in rural facilities may be absent from population estimates. |
| Eligibility mismatch |
A registry includes full-time students, but the research question concerns all enrolled students. |
The registry excludes part of the intended target population by design. |
| Selective participation |
Eligible individuals with greater interest in a program are more likely to register. |
Recorded participants may differ systematically from eligible nonparticipants. |
| Selective retention |
Participants experiencing poorer outcomes are less likely to complete follow-up. |
The remaining analytic sample may distort outcome estimates or associations. |
| Temporal mismatch |
Records were collected before a major change in policy or service delivery. |
Historical findings may not directly represent current conditions. |
Investigating differences between registry participants and nonparticipants can help identify selective enrollment. However, participation comparisons alone cannot establish whether the registry adequately covers the broader population.
Which Population Characteristics Should You Compare?
Not every difference between the registry and target population has the same scientific importance.
Prioritize characteristics that are plausibly associated with the outcome, exposure, participation mechanism, or effect being estimated.
Depending on the research question, these might include age, sex, disease severity, socioeconomic circumstances, geographic location, institutional type, baseline performance, service access, or relevant comorbidities.
Where credible population benchmarks exist, compare registry distributions with those benchmarks using consistent definitions and time periods.
For instance, national administrative statistics might provide the proportion of students enrolled in public and private institutions. Those figures could help evaluate the institutional composition of an educational registry.
Comparisons should include both the direction and substantive magnitude of differences. Statistical significance tests alone are often unhelpful for evaluating population balance, particularly in very large datasets where small differences may produce small p-values.
Standardized differences, distributional comparisons, and descriptive summaries may be informative, provided their interpretation remains tied to the research question.
Does a Demographic Difference Automatically Mean the Registry Is Unusable?
No. The consequences of a population difference depend on what you are trying to estimate.
If the objective is to estimate the prevalence of a condition in a national population, systematic underrepresentation of groups with different prevalence rates may substantially distort the estimate.
If the objective is to investigate an association within a clearly defined population, the same demographic difference may be less consequential for that particular estimand, although selection bias, confounding, and effect modification still require consideration.
Even when an association is internally valid among registry participants, its magnitude may differ in the target population if the relationship varies across characteristics that are distributed differently.
This is why representativeness should not be reduced to a single demographic similarity score.
The more useful question is whether the observed differences threaten the specific inference you intend to make.
How Does Population Mismatch Affect Descriptive and Causal Claims Differently?
Descriptive and causal research questions may require different forms of population alignment.
For descriptive estimation, the researcher often wants to estimate a population quantity, such as prevalence, average performance, service utilization, or an outcome rate. Coverage differences can directly affect that estimate when omitted groups have different outcomes.
For causal research, the investigator may seek to estimate the effect of an intervention or exposure. This requires credible identification of the causal effect within the study population and, if broader claims are intended, justification for transporting that effect to another population.
Internal validity and external validity are distinct. A registry study may produce a credible estimate for its observed population without establishing that the estimate applies elsewhere.
Conversely, a registry that closely resembles the target population demographically can still yield biased causal estimates if exposure groups differ in unmeasured confounders.
Population alignment therefore cannot substitute for an appropriate research design.
Can Weighting or Statistical Adjustment Correct Population Mismatch?
Sometimes, but only under assumptions that deserve explicit scrutiny.
Weighting, standardization, calibration, and related transportability methods may help account for observed differences between registry participants and the target population when appropriate external information is available.
For example, if the registry overrepresents one institutional category and reliable population totals are known, weighting may help align the observed composition with the intended population.
However, successful adjustment generally requires that the relevant selection-related characteristics are measured, that suitable target-population information exists, and that there is adequate overlap between the registry and target populations.
If an important subgroup is entirely absent from the registry, ordinary weighting cannot create direct empirical information about that subgroup. Extrapolation would require additional assumptions or evidence.
Watch Out
A weighted registry is not automatically a representative registry. Weighting can address specified observed differences under appropriate assumptions, but it does not establish that unmeasured selection mechanisms or missing population segments have been adequately accounted for.
What If the Registry Population Is Narrower Than Your Intended Population?
A narrower registry population does not necessarily prevent useful research.
You may be able to redefine the target population to correspond to the individuals or institutions genuinely represented by the data. This is appropriate when the narrower question remains scientifically meaningful and the population definition is transparent.
For example, a study originally intended to examine all university students might instead investigate students enrolled in participating private universities during a specified period.
That revision changes the scope of inference. It should be treated as a substantive methodological decision, not merely a wording adjustment made after seeing the results.
Where broader inference is essential, the researcher may need additional sites, supplementary data, or an alternative sampling strategy.
How Do Time and Continued Observation Affect Population Alignment?
Population alignment can deteriorate even when a registry initially captures its intended population well.
Some participants die, withdraw, transfer, stop using services, or otherwise become unobservable. New participants may enter under different conditions, while the broader target population changes over time.
Analyses restricted to individuals who remain observable may therefore represent a selected subset of the original population.
Researchers should examine survivorship and selection bias in existing cohorts when continued inclusion depends on characteristics related to the research question.
They should also assess the relevance of historical cohort data when the intended claim concerns current populations or contemporary practices.
What Evidence Is Needed to Defend Generalizability?
Generalizability is strengthened by a transparent account of how the registry population was assembled and how it relates to the intended target population.
Useful evidence may include registry eligibility documentation, geographic and institutional coverage, recruitment and participation rates, population benchmarks, missingness patterns, and comparisons of relevant characteristics.
When direct comparisons are unavailable, researchers should identify which aspects of population alignment cannot be evaluated and explain the plausible implications.
Reporting guidance such as STROBE recommends describing participant selection and discussing the generalizability of observational findings. RECORD provides additional reporting considerations for studies using routinely collected health data.
Neither checklist establishes that a study is generalizable merely because its reporting is complete. Transparent reporting makes the assumptions and limitations visible to readers.
