03 · What You Need to Know
How Registry Participation Affects the Evidence You Can Obtain
Who Counts as a Registry Nonparticipant?
The term nonparticipant can refer to several distinct groups. Researchers should identify which group they mean before comparing participants with nonparticipants.
Some individuals belong to the intended target population but fall outside the registry's geographic or institutional coverage. Others are eligible within participating settings but are never approached, decline consent, or are not recorded because of operational difficulties. Still others initially enter the registry but do not complete subsequent assessments.
These situations involve different selection mechanisms and may require different analytical responses.
| Group |
Why records may be absent |
Primary concern |
| Outside registry coverage |
Location, institution, or service provider does not contribute data |
Undercoverage of the target population |
| Eligible but not enrolled |
Not approached, declined, or registration was unsuccessful |
Enrollment and nonparticipation bias |
| Enrolled but incompletely recorded |
Required variables or events were not captured |
Missing-data and measurement-related selection |
| Lost during follow-up |
Withdrawal, transfer, death, disengagement, or loss of contact |
Attrition and informative censoring |
A registry can have excellent enrollment among eligible individuals at participating institutions while still excluding substantial segments of the broader population. Consequently, alignment between the registry population and the target population must be assessed separately from participation rates.
Why Might Some People Be More Likely to Participate?
Registry enrollment is often influenced by factors that are not random. In clinical registries, participation may depend on disease severity, frequency of healthcare contact, referral pathways, insurance arrangements, or willingness to consent.
In educational registries, students who engage with academic support programs may be more likely to complete voluntary assessments than students who have disengaged from university activities. Institutional recruitment procedures may also favor students who attend regularly or use particular digital platforms.
Participation can therefore be associated with characteristics that also predict the study's outcome.
Importantly, researchers should not assume the direction of these differences. A voluntary health registry might overrepresent people with severe symptoms seeking assistance, but another registry might underrepresent them because participation requires additional time or travel. The actual mechanism must be investigated.
What Is the Difference Between Self-Selection and Selection Bias?
Self-selection describes a participation process in which individuals influence their own likelihood of entering a study or registry. Selection bias refers to distortion of an estimate or association caused by how observations become included in the analysis.
Self-selection can contribute to selection bias, but the two concepts are not interchangeable.
For example, people with stronger health awareness might be more likely to join a voluntary registry. If the research aims to estimate health awareness in the general population, this participation mechanism could produce an upwardly biased estimate.
However, the existence of voluntary enrollment does not establish that every association calculated within the registry is biased. The effect depends on how selection relates to the exposure, outcome, and other relevant variables.
Selection bias is therefore a property of the relationship between a selection process and a particular analysis, not simply a label attached to a dataset.
When Do Participant Differences Distort Descriptive Estimates?
Descriptive research often seeks to estimate a population characteristic, such as disease prevalence, average academic performance, service satisfaction, or the proportion of individuals experiencing a particular outcome.
If registry participation is associated with that characteristic, the observed registry estimate may differ from the corresponding target-population value.
Suppose students experiencing severe academic difficulties are less likely to complete a voluntary institutional survey. The proportion reporting academic difficulties among respondents may underestimate the proportion among all eligible students.
The reverse is also possible. A registry established specifically to support students experiencing difficulties may overrepresent those students relative to the general student population.
The key issue is whether the outcome distribution among participants differs from that among nonparticipants, not merely whether participation is incomplete.
Can Participation Differences Bias Associations Between Variables?
Yes. Selection can distort associations when inclusion depends on variables related to the exposure and outcome, including their causes or consequences.
One important mechanism occurs when researchers condition on a variable influenced by both the exposure and the outcome, or by their respective causes. In causal diagrams, this can create collider stratification bias.
For example, suppose participation in a voluntary registry is more likely among students receiving academic support and also among students experiencing academic difficulties. Restricting analysis to registered students may create or alter an association between support participation and academic difficulties, even when the underlying population relationship differs.
The precise direction and magnitude depend on the selection mechanism. Such bias cannot be diagnosed merely by observing that the registry contains more participants in one category.
Researchers should consider a causal diagram or another explicit account of the selection process when evaluating whether conditioning on registry membership could distort their intended association.
How Can You Compare Participants and Nonparticipants?
The most informative comparison uses data collected independently of registry participation, such as an eligibility log, institutional administrative records, a population register, or an external data source covering both groups.
Where permissible, compare characteristics measured consistently for participants and nonparticipants. Relevant variables might include age, geographic location, baseline condition, institutional category, service utilization, or prior outcomes.
Useful comparisons include distributions, proportions, means, standardized differences, and participation rates across meaningful subgroups.
A participation rate can be expressed as the number of eligible individuals enrolled divided by the number of eligible individuals considered for enrollment, provided the denominator and recruitment process are clearly defined.
However, a high participation rate does not guarantee the absence of bias, and a low rate does not automatically imply severe bias. The relationship between participation and the variables of interest remains central.
If no information about nonparticipants exists, researchers should acknowledge that the selection mechanism cannot be directly evaluated. Comparing participants only with one another does not establish their similarity to nonparticipants.
What If Nonparticipant Data Are Unavailable?
Complete information about nonparticipants is often unavailable, especially when the registry contains only consenting individuals or contributing institutions.
Researchers may still investigate selection indirectly using recruitment documentation, available population benchmarks, institutional coverage information, and published evidence about participation mechanisms.
For example, the demographic composition of a registry could be compared with official population statistics, provided definitions, time periods, and geographic boundaries are compatible.
Such comparisons can reveal observable differences but cannot establish equivalence on unmeasured characteristics.
When important information is absent, sensitivity analyses may help examine how conclusions would change under plausible assumptions about selection. These analyses should be presented as conditional investigations rather than corrections known to recover the true population estimate.
Can Weighting Correct Differences Between Participants and Nonparticipants?
Inverse probability weighting, calibration, and related methods may reduce certain selection-related distortions when sufficient information is available.
For example, if eligible individuals' participation probabilities can be estimated from characteristics measured for both participants and nonparticipants, inverse probability weighting may be used to give greater analytical weight to participants who resemble underrepresented groups.
However, the validity of such adjustment depends on assumptions about the participation model, measured variables, adequate overlap, and the relationship between selection and the outcome.
When participation depends on important unmeasured factors, weighting based only on observed characteristics may leave substantial residual bias.
Extremely small estimated participation probabilities can also produce unstable weights and imprecise estimates. Weight distributions and effective sample sizes should be examined when weighting is used.
Watch Out
Do not describe a registry as representative simply because its records have been weighted to match a few demographic benchmarks. Adjustment is only as defensible as the data and assumptions supporting the selection model.
Does Follow-Up Attrition Create the Same Problem as Initial Nonparticipation?
Initial nonparticipation and follow-up attrition can both produce selection problems, but their timing and mechanisms differ.
Initial nonparticipation determines who enters the registry. Attrition determines which initially enrolled individuals remain observable or contribute data at later stages.
If participants with poorer outcomes are more likely to leave the registry, analyses restricted to those completing follow-up may systematically misrepresent the original cohort.
Researchers should evaluate survivorship and selection bias in existing cohorts rather than assuming that the baseline recruitment assessment remains sufficient throughout follow-up.
Methods for addressing missing follow-up data depend on the design, the missingness process, and the assumptions that can reasonably be defended.
Can Registry Participation Change Over Time?
Participation mechanisms may evolve as institutions join or leave a registry, consent procedures change, recruitment expands, or data collection becomes automated.
A registry that initially recruited volunteers might later capture all individuals receiving a particular service. Differences between early and later records could then reflect changes in participation rather than genuine population trends.
Similarly, changes in registry eligibility definitions may alter who is counted as a participant.
Researchers conducting longitudinal or repeated cross-sectional analyses should investigate whether participation and coverage remain sufficiently comparable across periods.
How Should Participation Bias Be Reported?
Transparent reporting should distinguish the intended population, eligible population, registered participants, and final analytic sample whenever these groups can be identified.
Describe how individuals entered the registry, the available information about those who did not participate, and any adjustments used to address selection.
When participant–nonparticipant comparisons are unavailable, state this limitation directly rather than claiming that the registry is unbiased or representative.
The STROBE reporting framework emphasizes participant selection and potential sources of bias in observational research. RECORD provides additional guidance for studies using routinely collected health data.
Neither reporting framework replaces a substantive assessment of the actual participation mechanism.
04 · A Practical Example
How Voluntary Registry Participation Can Distort an Academic Outcome Estimate
Hypothetical Example
A Student Well-Being Registry With Unequal Participation
A university has 10,000 eligible students and invites all of them to participate in a voluntary well-being registry. Of these, 4,000 enroll.
Among registry participants, 1,200 report substantial academic stress, producing an observed proportion of 30%.
The researcher wants to estimate the proportion experiencing substantial academic stress among all 10,000 eligible students, not merely among registry participants.
Step 1: Establish the participation rate.
The registry enrolled 4,000 of 10,000 eligible students, giving a participation rate of 40%. This figure describes enrollment completeness, not the magnitude or direction of selection bias.
Step 2: Identify the missing population.
The remaining 6,000 eligible students did not participate. Their academic stress outcomes are unknown from the registry.
Step 3: Consider possible selection mechanisms.
Students experiencing severe stress might be more motivated to participate, or they might be less likely to respond because of competing demands. Either mechanism is plausible and requires evidence.
Step 4: Examine what the observed percentage supports.
The 30% figure describes the registered participants. Without further information or assumptions, it cannot be treated as the proportion among all eligible students.
Step 5: Investigate additional evidence.
The researcher could compare available administrative characteristics, evaluate participation patterns across student groups, and consider defensible weighting or sensitivity analyses.
To illustrate the uncertainty, suppose the true proportion experiencing substantial academic stress among nonparticipants were 20%. The population proportion would then be (1,200 + 1,200) / 10,000 = 24%.
If the corresponding proportion among nonparticipants were instead 40%, the population proportion would be (1,200 + 2,400) / 10,000 = 36%.
These are hypothetical sensitivity scenarios, not estimates of the actual nonparticipant outcomes. They demonstrate why the observed 30% cannot independently establish the population proportion.
Without any information about the 6,000 nonparticipants, the purely arithmetic bounds would be 12% to 72%, assuming the 10,000-person eligible population and the observed participant outcomes are correctly enumerated. Those broad bounds emphasize how little can be established without additional evidence.
