When Can Electronic Health Records or Digital Administrative Systems Support a Defensible Research Study?

Can Records Collected for Clinical or Administrative Purposes Become Reliable Research Evidence?

You want to investigate a question involving patients, students, employees, service users, or institutional processes. Rather than recruiting participants and collecting every observation yourself, you discover that an electronic health record system, student information system, learning management platform, or administrative database already contains relevant information.

The records may cover several years, include thousands of individuals, and document activities that would be difficult or expensive to observe prospectively. Their availability makes a retrospective or secondary-data study attractive.

But the information was not necessarily collected for your research question. Some fields may be incomplete, variables may reflect administrative conventions, and recorded events may not correspond precisely to what occurred in practice.

When can such records support a rigorous investigation? How can you distinguish an extensive database from a dataset that actually provides valid evidence?

Routinely Collected Records Are Suitable When Their Meaning, Coverage, and Quality Support the Intended Inference

How to Evaluate Routinely Collected Digital Records for Research

What Are Routinely Collected Digital Data?

Routinely collected data are information generated through activities such as healthcare delivery, institutional administration, educational operations, or service management rather than collected primarily for a particular research investigation.

Electronic health records may contain diagnoses, medications, laboratory results, encounters, procedures, and clinical observations. Educational administrative systems may record enrollment, course completion, grades, attendance, and institutional transactions.

Other digital systems may document employment, service utilization, financial transactions, or interactions with organizational platforms.

These records can support secondary analysis, but their original operational purposes influence which information is recorded and how it is defined.

Distinguish Electronic Health Records From Administrative Databases

These categories overlap. Healthcare administrative claims, for example, may record diagnoses and procedures but reflect reimbursement rules rather than the full clinical encounter.

The researcher should identify the system's original purpose before interpreting its variables.

Which Research Designs Can Use These Records?

Routinely collected data can support descriptive, cross-sectional, longitudinal, cohort, case-control, and certain quasi-experimental investigations when their structure and quality are appropriate.

For example, historical student records may support a retrospective cohort study of academic progression. Clinical records may support an investigation of outcomes following a documented exposure.

Administrative records may also provide repeated observations before and after an institutional policy change.

However, a particular design is justified by the available evidence, not simply by the existence of dates or large numbers of records.

Temporal ordering, exposure definitions, outcome ascertainment, confounder availability, and the process generating the observations must be evaluated.

What Makes Routinely Collected Data Fit for Research?

Data fitness for purpose concerns whether the records are sufficiently appropriate for the intended scientific use.

These criteria should be evaluated together. High completeness cannot compensate for a variable that measures the wrong construct.

Why Does the Original Purpose of Data Collection Matter?

Clinical and administrative systems are designed primarily to support operational decisions, not necessarily research measurement.

For example, a diagnosis code may be entered to support billing or documentation. Its presence does not automatically establish that a condition was confirmed using the diagnostic criteria required by a particular study.

Likewise, a learning management system may record a student login without establishing whether meaningful learning occurred.

Researchers should distinguish what the system records from the theoretical or clinical concept they wish to investigate.

When the recorded field is a proxy, the study should acknowledge its limitations and seek relevant validation evidence.

How Can Selection Bias Enter Digital Records?

Individuals appear in administrative databases because they interact with particular organizations or systems.

Patients receiving care outside a participating hospital may not be represented. Students who never activate an institutional platform may have little or no digital activity history.

These exclusions may be related to the exposure or outcome under investigation.

For example, estimating the prevalence of a condition using hospital records may overrepresent people who seek or require hospital care.

The researcher must define the source population and explain which individuals have an opportunity to enter the database.

Large numbers of records cannot automatically correct systematic exclusions.

What Is the Difference Between a Missing Value and an Absence of an Event?

Digital records may contain blank fields, zero values, default entries, or no corresponding transaction.

These situations have different meanings.

A missing diagnosis code does not necessarily mean that a patient does not have the condition. An absent learning platform event does not necessarily establish that a student was inactive.

Information may be missing because the activity occurred outside the system, documentation was optional, data were not transferred, or a technical failure prevented recording.

Researchers should examine the recording process before interpreting missing entries as negative observations.

How Can Data Quality Be Evaluated?

Data quality assessment may include checking completeness, plausible ranges, internal consistency, temporal relationships, duplicates, and agreement with external information.

For example, a dataset containing assessment dates before enrollment may indicate coding, linkage, or date-format problems.

Unexpectedly identical values across many records may warrant investigation of defaults or automated population rules.

The harmonized data quality framework developed by Kahn and colleagues emphasizes conformance, completeness, and plausibility as important dimensions of electronic health record data quality.

These dimensions can also inform administrative data evaluation, although their implementation should be adapted to the relevant domain.

Should Researchers Validate Important Variables Against Source Records?

Where feasible and authorized, validating a justified sample of critical variables can help estimate recording accuracy.

For example, a researcher may compare selected diagnosis codes with underlying clinical documentation or verify administrative outcome classifications against original institutional records.

The validation sample should reflect the purpose of the assessment and relevant sources of variation.

Rare outcomes, different institutions, and different software periods may require deliberate consideration.

Manual validation is not always possible, and the underlying source record may itself contain errors. The researcher should explain what the validation procedure can establish.

How Do Changing Definitions Affect Longitudinal Research?

Digital systems may change their coding schemes, field definitions, interfaces, or recording requirements over time.

For example, an institution may revise how it classifies student withdrawals or a hospital may change the coding conventions used for particular encounters.

An apparent increase in an outcome may reflect a recording change rather than a genuine change in the underlying phenomenon.

Researchers should examine data dictionaries, system documentation, policy histories, and relevant version changes.

Where definitions differ, harmonization may be possible, but equivalence should not be assumed.

What Problems Arise When Linking Records?

Record linkage connects information about the same person, organization, or event across datasets.

For example, researchers may link student enrollment records with course activity and academic outcomes.

Incorrect linkage can create false matches, while unsuccessful linkage can exclude relevant observations.

Linkage quality depends on identifiers, matching procedures, data completeness, and the characteristics of the population.

Researchers should document the linkage method, matching criteria, unresolved records, and possible selection effects.

Repeated observations must also be distinguished from independent participants. One individual may contribute many records.

Can Routinely Collected Records Support Causal Inference?

Potentially, but causal inference requires more than historical records and statistical adjustment.

Observational data may contain confounding, selection bias, measurement error, and uncertainty about the timing of exposure and outcome.

For example, students who frequently use an AI-supported learning platform may obtain higher grades because of prior achievement, motivation, course requirements, or other factors.

The association does not independently establish that platform use caused higher achievement.

Researchers should specify the causal question, relevant assumptions, and identification strategy before making causal claims.

Target trial emulation and other causal inference approaches may be useful when the data and assumptions support them, but these methods cannot manufacture information absent from the records.

How Should Privacy and Research Governance Be Addressed?

Routinely collected records may contain sensitive or identifiable information.

Researchers should verify the legal basis for processing, institutional authorization, applicable ethics requirements, data-use agreements, and appropriate safeguards.

Access should be limited to necessary information and authorized personnel. Secure storage, transfer, retention, and disposal arrangements should be established.

Removing direct identifiers does not always eliminate re-identification risks, particularly when detailed longitudinal or demographic information remains.

Requirements vary by jurisdiction and data type. Researchers should consult the relevant institutional and legal authorities rather than assume that secondary data are automatically exempt from review.

What Should Be Reported in a Study Using Routine Records?

Researchers should describe the data sources, source population, inclusion criteria, observation period, variable definitions, data cleaning, linkage procedures, missingness, and analytical assumptions.

The RECORD statement extends the STROBE reporting guidelines for observational studies using routinely collected health data.

It emphasizes transparency concerning database characteristics, coding, linkage, and participant selection.

Researchers working with educational or other administrative records may draw on relevant principles while following reporting standards appropriate to their own discipline.

Transparent reporting allows readers to evaluate how operational records were transformed into research evidence.

When Should a Digital Administrative Dataset Be Rejected?

A dataset may be unsuitable when essential variables are absent, population coverage is seriously misaligned, measurement definitions are unreliable, or the required temporal information cannot be established.

It may also be unsuitable when permissions cannot be obtained or privacy protections cannot be implemented adequately.

For example, a study investigating sustained learning improvement cannot rely solely on platform login counts if no suitable learning outcome is available.

The researcher may need to narrow the question, supplement the records with appropriate primary data, or select another design.

Using University Administrative Records to Study Academic Progression

The resulting study may provide valuable evidence about academic progression within the institution.

However, its conclusions depend on the quality and interpretation of the administrative records. If the definition of withdrawal changed during the observation period, the researcher must account for that change before interpreting trends.

The example illustrates how routinely collected data can support longitudinal research when the underlying recording process is adequately understood.

Common Misconceptions About Electronic Records and Administrative Data

How to Decide Whether Routine Digital Records Can Support Your Study

Begin by specifying the research question and identifying the essential variables, population, and observation period. Then examine how the source system generated and maintained those records.

When definitions vary across systems, examine how changing data definitions affect research validity.

If the study investigates trends, consider whether system changes can produce artificial patterns in the data.

For data quality assurance, assess whether critical variables should be manually validated.

When recruitment is uncertain, evaluate whether existing data provide a defensible alternative to collecting new observations.

The research design should follow the evidence the records can actually provide, not merely the number of observations available.

Before Using Electronic Health Records or Administrative Data

Frequently Asked Questions About Research Using Routine Digital Records

Routine Digital Records Are Research Evidence Only When Their Meaning and Quality Are Established

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.