03 · What You Need to Know
Participant Exclusion Is Not One Decision but Several Different Decisions
First Ask What “Excluded” Actually Means
Researchers often say that a participant was “excluded” without specifying from what.
A person might be screened out before enrollment because they do not meet eligibility criteria. An enrolled participant might be excluded from one particular analysis because the relevant outcome is missing. A participant might remain in the primary analysis but be omitted from a secondary per-protocol analysis. Another might be removed from every analysis because the researcher later decides the person's results are inconvenient.
Those are not interchangeable situations.
Before evaluating an exclusion, identify when it occurred, which dataset or analysis it affects, what rule produced it, and how the participant is represented in the study's records and reporting.
Eligibility Criteria Can Legitimately Exclude People From a Study
Research designs often define who is eligible to participate. Age ranges, diagnostic criteria, prior exposure, enrollment status, language requirements, contraindications, or other characteristics may be scientifically relevant depending on the research question and ethical framework.
If a participant is discovered to have been ineligible under a legitimate study criterion, some form of exclusion may be appropriate. Exactly how their already-collected data should be handled can depend on the protocol, consent, study design, regulatory requirements, statistical analysis plan, and disciplinary norms.
The important point is that “ineligible” provides a potentially substantive reason. “Their result is inconvenient” does not.
Protocol Deviations Do Not Automatically Require Participant Removal
Participants do not always follow research protocols perfectly. They miss sessions, take prohibited medication, fail to complete an intervention, respond outside a specified time window, or otherwise deviate from the planned procedure.
Whether such participants should be excluded depends on the research design and analysis. In randomized trials, for example, excluding participants after randomization can alter the benefits of random allocation and potentially bias treatment comparisons. A per-protocol analysis answers a different question from an intention-to-treat analysis.
Researchers should therefore resist the intuitive but potentially damaging assumption that “did not follow the protocol perfectly” means “delete this participant.”
Missing Data Does Not Necessarily Mean the Participant Should Disappear
If a participant lacks one outcome, researchers may be unable to include that individual in a particular analysis. That does not necessarily justify removing all of the participant's other valid data or pretending that the person was never enrolled.
Missing-data handling is an analytical problem with methods and assumptions of its own. The correct approach depends on the design, pattern and mechanism of missingness, statistical method, and research question.
Whole-participant deletion can discard considerably more information than the missing value itself.
Attention Checks Need More Thought Than “Pass or Delete”
Attention checks are common in some survey and behavioral research, but their presence does not make every exclusion based on them automatically defensible.
Researchers should consider whether the check validly measures the response-quality problem of concern, whether the exclusion criterion was established appropriately, whether participants had a reasonable opportunity to satisfy it, and whether the rule is applied consistently.
A particularly concerning pattern would be inspecting the outcome first and invoking an attention check only for participants whose responses weaken the hypothesis.
An Extreme Participant Is Not Necessarily an Invalid Participant
Participants can be unusual without being erroneous.
A very high or low outcome may reflect a genuine person at the tail of the population distribution. Removing that participant because the value looks strange can narrow the apparent variation in the population and exaggerate the stability of the result.
Investigate whether the extreme observation reflects measurement error, eligibility problems, data-entry error, protocol failure, or a genuine observation. The distinction between removing invalid data and removing inconvenient data applies just as strongly when the unit being removed is an entire participant.
Prespecified Criteria Reduce Discretion but Do Not Automatically Make Exclusion Correct
Defining exclusion criteria before outcomes are known can reduce opportunities for researchers to tailor the sample to the desired result. Preregistration, protocols, and statistical analysis plans can therefore strengthen the credibility of exclusion decisions.
Still, a prespecified criterion can be poorly designed. Conversely, a legitimate exclusion may become necessary only after an unforeseen problem is discovered.
Prespecification is evidence about the decision process, not a magical stamp of methodological validity.
Changing Exclusion Rules After Seeing Outcomes Requires Particular Scrutiny
Suppose a protocol excludes participants who complete less than 70% of an intervention. After seeing the results, the researcher notices that changing the threshold to 80% removes several participants from the treatment group and makes the effect statistically significant.
The researcher may be able to propose a scientific argument for 80%. But because the preferred outcome was already known, the decision now has an obvious competing explanation.
This does not mean that all post hoc changes are misconduct. It does mean that the researcher should preserve the original plan, explain the change, justify it independently, and consider reporting analyses under both specifications.
Selective Application of the Same Rule Is a Major Warning Sign
Imagine that two participants fail the same attention check. One strongly supports the hypothesis; the other strongly contradicts it. Removing only the second participant is difficult to explain through the attention-check criterion itself.
Similarly, if the researcher labels an extreme low score “invalid” while retaining an equally extreme high score because it supports the desired effect, the stated exclusion rule is not being applied symmetrically.
Selective inclusion and omission of data points has appeared in actual ORI research misconduct findings. The broader lesson is that the scientific rationale should determine who meets the criterion, rather than the participant's contribution to the preferred result.
Participant Exclusion Can Fall Within Falsification
The U.S. Public Health Service defines falsification as manipulating research materials, equipment, or processes, or changing or omitting data or results such that the research is not accurately represented in the research record.
The Federal Research Misconduct Policy also explicitly recognizes that omission can sometimes be appropriate under accepted research practices, while omission is falsification when it misleads readers about the research results.
Accordingly, the question is not simply “Was someone excluded?” It is whether the omission, in context, caused the research record to inaccurately represent the research.
Reporting the Sample Size Accurately Matters
Participant exclusion affects more than the statistical calculation. It can affect descriptions of recruitment, attrition, denominators, participant characteristics, flow diagrams, tables, figures, and methods.
If 150 people contributed data but a researcher selectively analyzes 135, the report should not casually create the impression that only 135 relevant participants ever existed if the omitted participants are material to understanding the research.
Actual ORI findings have included cases involving falsely reported numbers of data points and selective omission or inclusion in analyses. Accurate accounting of the analytical sample is therefore part of accurately representing the research record.
One Participant Can Matter Even in a Large Dataset
Researchers sometimes reason that removing one participant from a large sample cannot matter ethically because the overall result barely changes.
That confuses effect size with integrity.
A legitimate exclusion does not become illegitimate because it changes the conclusion, and an illegitimate exclusion does not become acceptable because it fails to change the conclusion. The primary question remains whether the omission has a defensible basis and the research is represented accurately.
A Poor Exclusion Decision Is Not Automatically Research Misconduct
There is still an important distinction between methodological criticism and a formal misconduct finding.
Under the PHS framework, research misconduct does not include honest error or honest differences of opinion. A finding requires a significant departure from accepted practices of the relevant research community, intentional, knowing, or reckless conduct, and proof by a preponderance of the evidence.
Researchers can therefore disagree legitimately about whether a participant should be included. The mere existence of that disagreement does not establish falsification.
Watch Out
If you decide whether a participant is “invalid” only after learning whether that participant helps or hurts your hypothesis, you have introduced the outcome into the exclusion rule. That does not automatically prove misconduct, but it creates a serious risk of biased and potentially misleading analysis.
