How Do You Account for Incomplete Documentation in Research Using Routine Records?

What Should You Do When Routine Records Do Not Document Everything That Happened?

You are analyzing electronic health records, insurance claims, university administrative records, or another routinely maintained database. Some variables contain blank fields. Certain events appear only in narrative notes. Other information may never have been entered because it was not required for the organization's ordinary operations.

Should you exclude incomplete records, replace missing values, treat undocumented events as absent, or proceed with the information available?

The difficulty is that incomplete documentation does not always resemble conventional missing data. An event may have occurred without being recorded, a field may be inapplicable, or relevant information may exist outside the accessible system.

How you interpret these situations can affect the validity of your sample, measurements, and conclusions.

Investigate Why Information Is Missing Before Choosing an Analytical Response

Understanding Incomplete Documentation and Its Research Consequences

Why Routine Records Are Often Incomplete for Research Purposes

Routine records are generally created to support healthcare delivery, educational administration, financial transactions, regulatory reporting, or other operational activities. They are not necessarily designed to capture every variable required by a future research study.

A clinician may document information relevant to treatment but omit details that are not clinically necessary. An instructor may record a student's final grade without documenting every informal learning interaction. An administrative system may retain completed transactions but not unsuccessful attempts.

These omissions do not automatically indicate poor professional practice or defective recordkeeping. Information can be sufficient for its original operational purpose while remaining incomplete for a particular research question.

The relevant question is whether the available documentation adequately represents the variables, events, and population needed for the intended analysis.

Distinguish Different Forms of Incomplete Documentation

Before calculating missing-value percentages, establish what kind of information is absent.

These categories are not necessarily mutually exclusive. A missing structured value may reflect an unperformed assessment, information recorded elsewhere, or a failure to enter an available result.

Identifying the reason matters because the appropriate analytical response depends on what the missing information represents.

Examine How Documentation Is Generated

Understanding the data-generating process is often more informative than inspecting the final dataset alone.

Determine who creates each record, when documentation occurs, whether particular fields are mandatory, and what circumstances trigger an entry.

For example, laboratory results may be recorded only for patients who undergo testing. If testing depends on symptoms, clinical suspicion, or access to care, the observed results represent a selected group rather than a random subset of all patients.

Similarly, a university may record academic support consultations only when students formally book appointments. Informal support received through other channels may not appear.

Relevant evidence can include data dictionaries, operational manuals, workflow descriptions, reporting requirements, system audit logs, and discussions with knowledgeable data custodians.

Where collection procedures changed, examine whether documentation remained sufficiently consistent across the periods and institutions being compared.

Measure Completeness Using Appropriate Denominators

A basic assessment calculates how frequently expected information is present. However, the denominator must reflect the records for which the information should reasonably exist.

For example, calculating the percentage of completed pregnancy-related fields across every patient would produce a misleading completeness estimate if many records are legitimately inapplicable.

Researchers should distinguish eligible records, applicable fields, and observations for which documentation was expected.

Completeness may be evaluated at several levels:

• Variable level: the proportion of applicable observations with a recorded value.
• Record level: the extent to which an individual record contains the required information.
• Event level: whether relevant events are captured by the system.
• Population level: whether the dataset includes the intended individuals or units.
• Temporal level: whether documentation is available throughout the required observation period.

A high proportion of populated fields does not establish that all relevant events or individuals were captured. Field completeness and population coverage are different properties.

Investigate Whether Missingness Is Systematic

Missing information becomes particularly consequential when the probability of documentation depends on characteristics relevant to the study.

For example, patients with more severe symptoms may receive more frequent assessments, making their records more complete. Students experiencing academic difficulties may have more detailed advising records because they receive additional institutional attention.

In such situations, completeness may be associated with the exposure, outcome, or underlying characteristics of the population.

Examine missingness across relevant subgroups, including institutions, departments, time periods, demographic characteristics, exposure categories, and outcome groups where analytically appropriate.

Descriptive comparisons can identify concerning patterns, although they cannot fully establish the missingness mechanism.

Understand MCAR, MAR, and MNAR Without Treating Them as Observable Labels

Missing-data methodology commonly distinguishes three mechanisms:

These mechanisms describe assumptions about the missing-data process, not labels that can generally be established conclusively from observed records.

For example, if laboratory testing is more likely among symptomatic patients, missing laboratory results may be associated with disease status. Whether an MAR assumption is defensible depends on which predictors of testing and disease are observed and incorporated into the analysis.

Researchers should use substantive knowledge of documentation workflows to justify assumptions rather than classify missingness solely from a statistical test.

How Incomplete Documentation Can Bias Research Findings

Incomplete documentation can affect research in several ways.

Selection bias may arise when the analytical sample includes only individuals with sufficiently complete records and the selection process is related to variables relevant to the association being studied.

Information bias may arise when exposures or outcomes are incorrectly classified because relevant information was not recorded.

Reduced precision may result when incomplete observations are excluded, leaving fewer usable records.

Documentation gaps may also affect confounding control when important covariates are missing or incompletely measured.

These consequences are not inevitable in every dataset. Their severity depends on the missingness mechanism, the variables affected, and the intended inference.

For example, a missing administrative contact number may be irrelevant to a study of laboratory outcomes, whereas incomplete documentation of the primary outcome could fundamentally undermine the analysis.

When Is Complete-Case Analysis Appropriate?

Complete-case analysis includes only observations with all variables required for the specified analysis.

It is straightforward to implement and may be reasonable under assumptions that support unbiased estimation of the target association, but those assumptions depend on the model and missingness process.

Complete-case analysis does not always require MCAR for an unbiased estimate of every regression parameter. Nevertheless, restricting analysis to complete records can produce substantial bias when selection depends on relevant outcomes or unmeasured characteristics.

Researchers should compare included and excluded observations using available information, examine the extent of sample reduction, and explain why the resulting analytical population remains appropriate.

Simply reporting that incomplete cases were removed is not an adequate missing-data assessment.

When Can Multiple Imputation Help?

Multiple imputation replaces missing values with multiple sets of plausible values drawn from an explicitly specified predictive model, allowing uncertainty associated with missingness to be reflected in the combined analysis.

It can be useful when the missingness assumptions are defensible and sufficient observed information is available to support the imputation model.

Appropriate implementation requires attention to variable types, relationships among variables, clustering, temporal structure, and compatibility with the substantive analysis.

However, multiple imputation does not automatically repair incomplete event capture, missing individuals, or systematic documentation errors.

If a hospitalization never entered the accessible database, imputing a missing value in an existing record is not necessarily an adequate substitute for identifying the missing event.

Likewise, imputing a diagnosis field does not establish whether a clinical condition was actually present.

Consider Other Analytical Responses to Missing Documentation

Depending on the problem, researchers may consider inverse probability weighting, appropriate missing-data models, linkage to supplementary sources, targeted validation, or sensitivity analysis.

Inverse probability weighting may help address selection related to observed variables when the required assumptions, including adequate positivity and correct model specification, are reasonable.

Record linkage may recover information held in other systems, although linkage introduces its own coverage, privacy, and matching-quality considerations.

When information is structurally absent or cannot be recovered, researchers may need to redefine the variable, restrict the target population, or reconsider the research question.

Choosing among these approaches requires identifying whether the problem concerns missing values, unobserved events, or incomplete population coverage.

Use Sensitivity Analyses to Examine Unverifiable Assumptions

Missing-data assumptions often cannot be verified using the observed dataset alone. Sensitivity analyses help determine how conclusions change under alternative plausible assumptions.

For example, researchers might examine whether results differ between complete-case and appropriately specified imputed analyses, or explore scenarios in which undocumented outcomes are more common in one exposure group.

Such analyses should be grounded in substantive knowledge and plausible ranges of uncertainty.

Agreement across analyses can strengthen confidence in robustness to the assumptions examined, but it does not prove that the underlying records are complete or unbiased.

Report Documentation Limitations Transparently

Describe the extent of incomplete documentation, the variables affected, available explanations for missingness, and the methods used to address it.

Report how many observations were eligible, how many entered each analysis, and why records were excluded where applicable.

For routinely collected health data, the RECORD statement extends STROBE reporting recommendations and addresses relevant characteristics of data sources, coding algorithms, and linkage.

Transparent reporting allows readers to assess whether the conclusions remain credible despite incomplete documentation.

Incomplete Laboratory Documentation in a Hospital Study

The absence of 1,500 laboratory results does not establish that those patients had normal results. Nor does it establish that multiple imputation will recover their true values.

The defensibility of the analysis depends on what is known about why testing occurred and whether the available information can support the assumptions of the chosen method.

Common Mistakes When Handling Incomplete Routine Records

Match Your Analytical Response to the Type of Documentation Gap

Begin by determining whether the missing information is necessary for the intended inference. Then identify what is missing, why it may be missing, and whether the omissions are related to the variables under investigation.

Particular caution is needed when incomplete documentation affects diagnostic status. The absence of a recorded diagnosis does not necessarily establish that a medical condition is absent.

Similarly, missing evidence of treatment or service use may reflect either missing care or missing data. These possibilities should not be conflated.

Where incomplete documentation fundamentally prevents adequate measurement, it may be necessary to reassess the feasibility of the secondary data study rather than rely on increasingly elaborate statistical corrections.

What to Check Before Analyzing Incomplete Routine Records

Frequently Asked Questions About Incomplete Routine Records

Incomplete Documentation Requires More Than Filling Blank Cells

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.