03 · What You Need to Know
How to Plan Reliable Backup Sources for Essential Outcomes
What is a backup outcome data source?
A backup outcome data source is an alternative means of obtaining information about a predefined research outcome when the preferred measurement source is unavailable, incomplete, or unusable.
For example, a researcher may plan to obtain examination results through direct assessment but also arrange authorized access to official institutional records containing the same assessment scores.
In clinical research, backup sources might include medical records, registries, or independently verified documentation of a specified event. In educational research, they may include official assessment records or appropriately validated administrative data.
The essential distinction is between obtaining the same outcome through another source and substituting a different outcome.
A backup source should preserve the intended construct, time frame, and measurement definition as closely as the research question requires.
Why are primary outcomes especially important to protect?
The primary outcome is central to the study's principal research question and usually influences sample size planning, statistical analysis, and interpretation.
Missing primary outcomes can reduce precision and may introduce bias when observation depends on participant characteristics, intervention experiences, or unobserved outcome values.
For example, if students experiencing academic difficulties are more likely to miss a final assessment, an analysis restricted to observed scores may overrepresent higher-performing students.
A valid backup source may preserve outcome information for some of these participants and reduce dependence on assumptions about their missing values.
However, the value of backup ascertainment depends on its coverage and validity. A source that systematically excludes certain participants or measures a different construct may introduce additional problems.
Protecting primary outcomes should therefore be part of the missing-data strategy established during research design.
Which research outcomes are suitable for backup ascertainment?
Backup ascertainment is most feasible when the outcome has a stable definition and can be obtained from another reliable source.
| Research outcome |
Possible alternative source |
Important limitation |
| Standardized examination score |
Official record of the same examination |
Verify assessment identity, scoring rules, and record completeness. |
| Course completion |
Institutional enrollment or completion records |
Confirm the operational definition and reporting period. |
| Hospital admission |
Authorized medical records or relevant registries |
Assess coverage, event definitions, and record-linkage accuracy. |
| Mortality |
Authorized vital statistics or mortality registries |
Consider registration completeness, reporting delays, and jurisdictional coverage. |
| Platform activity |
Verified system logs or authorized activity records |
Determine whether the system captures the intended activity comprehensively. |
| Self-reported psychological state |
Another validated assessment procedure |
Different instruments or occasions may not produce equivalent measurements. |
These are potential arrangements rather than guarantees of equivalence. Their suitability depends on the outcome definition and the particular sources available.
Some outcomes, especially subjective experiences measured at a specific moment, may have no defensible retrospective substitute.
What is the difference between a backup source and a proxy outcome?
A backup source attempts to obtain the intended outcome through an alternative information channel. A proxy outcome represents the intended construct indirectly through another measurement.
Backup source for the same outcome
Obtaining the identical standardized examination score from an authorized institutional record instead of the original research database.
Proxy outcome
Using a course grade or platform activity score to approximate examination performance when the standardized examination score is unavailable.
Proxy outcomes may be scientifically useful, but they are not automatically equivalent to the original outcome.
A course grade may include attendance, assignments, participation, and instructor judgments. A standardized examination score may represent a narrower assessment of achievement.
Substituting one for the other can change the estimand or introduce measurement differences across participants.
Researchers should not label a different construct as a backup measurement merely because both variables are correlated.
How do you determine whether two sources measure the same outcome?
Begin by comparing the operational definitions of the outcome in each source.
Determine whether both sources represent the same event, construct, time point, scoring procedure, and unit of measurement.
For an examination outcome, relevant questions include whether the sources refer to the same examination version, whether scores are recorded before or after adjustments, and whether institutional records include retakes or supplementary assessments.
Where overlapping measurements are available, researchers can evaluate agreement between sources. Appropriate methods depend on the data type and may include agreement statistics, examination of systematic differences, and inspection of discrepancies.
Correlation alone is insufficient to establish interchangeability. Two sources can be highly correlated while consistently producing different values.
For continuous measurements, agreement methods such as those developed by Bland and Altman may be useful when their assumptions and application are appropriate.
When sources differ meaningfully, researchers should consider whether they can be reconciled through a defensible, prespecified procedure or whether they must be analyzed separately.
Why should source priority be specified before data collection?
When several sources provide information about the same outcome, researchers need a consistent rule for determining which value will be used.
For example, the protocol might designate a verified standardized assessment record as the preferred source and an authorized duplicate record of the same assessment as the backup.
If sources disagree, the team should have a documented process for checking assessment dates, record identifiers, scoring versions, and possible transcription errors.
Without predefined rules, researchers may inadvertently choose whichever source produces a more favorable or convenient result.
Source selection should not depend on the direction or statistical significance of the estimated intervention effect.
Where disagreements cannot be resolved, the uncertainty should be retained and addressed rather than silently replacing one value with another.
Can administrative data reduce missing outcomes?
Administrative records may preserve outcome information when participants do not complete a separate research assessment.
For example, official examination scores may be available for students who decline a follow-up questionnaire. This can reduce outcome missingness if the records contain the same valid outcome and access is permitted.
However, administrative data are generated for operational purposes. Their coverage and definitions may not correspond exactly to the research protocol.
Records may be delayed, incomplete for transfer students, inconsistent across institutions, or affected by changes in administrative procedures.
Researchers should consult data documentation and custodians to determine how the records are generated and maintained.
Administrative availability should not be confused with research validity.
What if the backup source has different coverage across study groups?
A backup source may be more complete for some participants than others.
Suppose one intervention group uses a learning management system that records detailed activities, while the comparison group uses a different platform with less comprehensive logging.
Combining these records without accounting for differences in coverage could create differential measurement error.
Similarly, a clinical registry may capture events occurring in one healthcare network but miss events treated elsewhere.
Researchers should examine whether source availability depends on group assignment, exposure, baseline characteristics, or the outcome itself.
Such differences are related to group-specific missingness patterns and may complicate causal comparisons.
Equal overall coverage percentages do not necessarily establish equivalent observation mechanisms.
How should researchers manage data linkage?
Using an alternative data source often requires linking records to the correct research participants.
Linkage may rely on authorized identifiers, pseudonymized study codes, or other approved procedures.
Incorrect linkage can assign another person's outcome to a participant, while failed linkage can create additional missingness.
Researchers should establish secure procedures for identifier management, access permissions, duplicate detection, and verification of uncertain matches.
Where probabilistic linkage is used, the possibility of false matches and missed matches should be evaluated.
Linkage quality may vary across participant groups, particularly when identifying information is incomplete or inconsistently recorded.
The protocol should describe who performs linkage, which information is accessible, and how confidentiality is protected.
What ethical and legal considerations apply to backup data sources?
Alternative data availability does not automatically authorize its use for research.
Researchers must consider the approved protocol, participant consent or another applicable legal basis, institutional data-governance rules, confidentiality protections, and any relevant data-sharing agreements.
For example, a university's possession of academic records does not necessarily mean every researcher may access those records for a particular project.
Where additional permissions are required, they should be obtained before linkage or extraction.
Researchers should also distinguish intervention discontinuation from withdrawal of consent for further data collection. A participant who stops attending an intervention may still permit outcome ascertainment, while another may withdraw permission for particular forms of follow-up.
The consequences for continued access and retention depend on the consent arrangements and applicable requirements.
Backup data collection must respect these distinctions rather than treating the scientific value of complete outcomes as overriding participant rights.
Can backup sources create measurement bias?
Yes. A backup source may systematically differ from the primary source even when both appear to measure the same construct.
For example, self-reported attendance and electronically recorded attendance may use different definitions. A participant may report attending an online session even when the system did not register a complete connection.
Neither source should automatically be assumed correct without examining its measurement process.
When some participants are measured through one source and others through another, systematic differences between sources can become associated with participant characteristics or study group.
Researchers should assess whether the source itself affects the recorded value and whether appropriate calibration or sensitivity analysis is possible.
A higher outcome completion rate is not necessarily beneficial if it is achieved by combining incompatible measurements.
Should backup sources be used for every secondary outcome?
Not necessarily. The resources required for backup ascertainment should be proportional to the scientific importance of the outcome and the consequences of missingness.
Primary outcomes and other critical measurements may warrant more extensive protection because their absence directly threatens the main research question.
Some secondary outcomes may justify alternative collection procedures, particularly when they concern safety, important subgroup comparisons, or prespecified mechanisms.
However, creating multiple backup systems for low-priority variables can increase cost, administrative burden, and privacy exposure without sufficient scientific benefit.
Researchers should use a variable-level missing-data risk assessment to identify where backup sources are most valuable.
Can backup sources replace missing-data analysis?
No. Even well-designed backup ascertainment may leave some outcomes unavailable.
Participants may be absent from both sources, records may fail linkage, or the alternative source may not contain a valid measurement.
Researchers should distinguish outcomes obtained from primary sources, outcomes recovered from backup sources, and outcomes that remain missing.
When source differences could affect estimation, the analysis may need to account for measurement source or evaluate results under alternative source-handling assumptions.
Missing-data methods may still be necessary for the remaining incomplete observations.
However, multiple imputation and related methods cannot automatically compensate for a backup source that measures a different construct.
What should the protocol specify about backup outcome sources?
A defensible protocol should identify the primary outcome definition, preferred measurement source, acceptable alternatives, and the conditions under which each source may be used.
It should also specify the intended assessment period, data-linkage procedures, quality checks, rules for resolving discrepancies, and relevant permissions.
Where backup sources have not been shown to be interchangeable, the protocol should explain how source differences will be evaluated.
Researchers should document the provenance of each outcome value so that the final dataset can distinguish primary and alternative ascertainment.
These decisions should be made before outcome results are examined whenever feasible. If an unanticipated source becomes necessary during the study, the scientific justification and any required protocol amendments should be documented.
