03 · What You Need to Know
How AI-Assisted Participant Safety Monitoring Should Work
What Does Participant Safety Monitoring Involve?
Participant safety monitoring involves identifying, evaluating, documenting, and responding to risks that arise during research participation.
Depending on the study, relevant information may include adverse events, laboratory abnormalities, changes in clinical condition, psychological distress, protocol deviations, or unexpected reactions to research procedures.
Safety monitoring is not limited to clinical trials. Behavioral, educational, and social research may also involve risks requiring appropriate procedures for recognition and response.
For example, a qualitative interview concerning traumatic experiences may require a protocol for responding when a participant experiences significant distress.
The appropriate monitoring arrangements depend on the research design, participant population, anticipated risks, and applicable ethical requirements.
AI assistance should be evaluated within those arrangements rather than introduced as a replacement for them.
How Can AI Contribute to Safety Risk Detection?
AI systems may assist with identifying patterns in structured data, extracting information from clinical notes, reviewing adverse-event narratives, or prioritizing records for closer examination.
For example, a system might flag a combination of laboratory values that warrants assessment under a predefined monitoring protocol.
Another system might identify language in participant reports that could indicate a possible adverse event.
These applications may improve the organization and prioritization of information, particularly when research teams manage large volumes of records.
However, identifying a possible signal is different from determining its clinical significance or deciding what action should follow.
Researchers should distinguish information extraction, risk prediction, clinical assessment, and intervention because each activity requires different evidence and oversight.
What Is the Difference Between a Safety Signal and a Safety Decision?
AI-generated safety signal
An automated indication that available information may warrant further investigation or assessment.
Authorized safety decision
A determination made through the approved monitoring process concerning the significance of a concern and any appropriate response.
An AI flag may identify a record requiring attention, but it does not independently establish that an adverse event occurred or that a particular intervention is necessary.
Similarly, a system's failure to produce an alert does not establish that the participant has no safety concerns.
Researchers should ensure that automated alerts are interpreted within the study's established safety procedures.
This distinction is particularly important when AI contributes to clinical research decisions.
Why False Negatives Can Be Particularly Consequential
A false negative occurs when a system fails to identify a condition or event that it is intended to detect.
In participant safety monitoring, a false negative may result in a concerning record receiving insufficient attention.
For example, an AI system might overlook a serious symptom because the participant described it using terminology not adequately represented in the system's training or validation data.
The consequences depend on the event, the monitoring workflow, and whether other safeguards identify the concern.
Researchers should evaluate sensitivity for the particular safety conditions the system is intended to detect.
However, high sensitivity alone does not establish that a monitoring system is suitable. Other performance characteristics and the consequences of incorrect alerts also matter.
Why False Positives Matter Too
A false positive occurs when a system flags a concern that is not present under the applicable reference standard.
False-positive alerts may lead to unnecessary investigations, additional participant contact, interruptions to research procedures, or avoidable anxiety.
Frequent incorrect alerts may also contribute to alert fatigue, in which reviewers become less responsive to warnings.
Research on clinical decision-support alerts has documented problems associated with excessive or poorly targeted notifications.
However, the appropriate balance between sensitivity and specificity depends on the seriousness of the risks being monitored.
For a potentially life-threatening event, researchers may reasonably prioritize avoiding missed cases, provided the resulting alert burden can be managed safely.
There is no universal threshold that makes every AI safety-alert system acceptable.
Why Overall Accuracy Is Not Enough
Safety events may be relatively uncommon in some research populations.
When an event is rare, a system can achieve high overall accuracy by classifying most observations as not concerning.
For example, imagine a hypothetical dataset containing 10,000 monitoring observations, of which 50 represent events requiring urgent assessment.
A system that never generates an alert would correctly classify 9,950 observations and achieve 99.5% overall accuracy.
Yet it would miss every event requiring urgent assessment.
This hypothetical example demonstrates why researchers should examine sensitivity, specificity, predictive values, and the consequences of particular errors rather than rely exclusively on overall accuracy.
The prevalence of safety events also affects the proportion of alerts that correspond to genuine concerns.
How Should AI Safety Monitoring Be Validated?
Validation should reflect the specific event, participant population, data sources, and workflow for which the system is intended.
For example, a model validated using structured hospital records may not perform equivalently when applied to free-text reports collected through a research survey.
Researchers may need to assess information extraction accuracy, classification performance, timeliness, calibration where relevant, and the consequences of errors.
External validation can help examine performance outside the original development setting.
Prospective evaluation may also be important when the proposed application could affect real-time participant monitoring.
Researchers should distinguish technical validation from demonstrating that a system improves participant outcomes.
A model may detect certain events accurately without necessarily improving the effectiveness of the complete safety-monitoring process.
Why Timeliness Matters
A safety alert may be accurate but still ineffective if it reaches the responsible person too late.
For example, a system might identify a concerning laboratory result after the relevant monitoring window has passed.
Researchers should therefore evaluate the complete pathway from data collection to alert generation, review, and appropriate action.
Relevant questions include how frequently information is processed, who receives alerts, and what happens when the designated reviewer is unavailable.
Where urgent events are possible, escalation procedures should follow the approved protocol and applicable clinical responsibilities.
AI should not create an impression of continuous safety monitoring unless the actual system and staffing arrangements support that capability.
Can AI Detect Safety Concerns That Human Reviewers Miss?
Potentially. Automated systems may identify patterns across large datasets or flag combinations of information that warrant closer examination.
However, whether they improve detection compared with established procedures requires empirical evaluation.
Performance may vary according to data quality, event type, population, and implementation.
Researchers should avoid assuming that an additional AI monitoring layer necessarily improves safety.
An ineffective system could introduce false reassurance, unnecessary alerts, or additional workflow complexity.
The relevant outcome is whether the complete monitoring arrangement provides adequate participant protection.
Why Human Oversight Must Remain Meaningful
Human review is useful only when reviewers can evaluate the underlying evidence and intervene appropriately.
For example, a clinician receiving an AI-generated alert should be able to examine the relevant record, assess the participant's circumstances, and determine the appropriate response under the approved protocol.
A reviewer who receives only a risk label without sufficient supporting information may be unable to evaluate the recommendation.
Similarly, a reviewer who lacks authority to escalate a concern may provide limited protection.
Effective oversight therefore requires relevant competence, information access, decision authority, and clear responsibilities.
These considerations are consistent with meaningful human oversight in AI-assisted research.
What About AI Monitoring of Psychological Distress?
Some research studies involve participants discussing trauma, discrimination, self-harm, or other sensitive experiences.
Researchers may consider using AI to identify language suggesting distress or possible safety concerns.
However, linguistic indicators are not equivalent to clinical assessment.
A participant may use ambiguous language, culturally specific expressions, or indirect descriptions that an AI system misinterprets.
False-positive classifications can also lead to inappropriate assumptions about participants.
Researchers should therefore establish ethically appropriate, human-led response procedures rather than relying on unvalidated automated interpretation.
Where a study involves foreseeable psychological risks, the approved protocol should identify appropriate monitoring and escalation responsibilities.
Participant Privacy Is Part of Safety Monitoring
Safety monitoring may require access to highly sensitive participant information.
Processing these records through external AI services can introduce confidentiality, security, and data-protection risks.
Researchers should examine the provider's processing arrangements, access controls, retention practices, and any secondary use of submitted information.
They must also determine whether the proposed processing is consistent with participant consent, approved protocols, institutional agreements, and applicable law.
Removing names does not necessarily eliminate reidentification risks.
A safety-monitoring tool should not be considered appropriate solely because it performs well on a detection benchmark.
Does AI Safety Monitoring Require Ethics or Regulatory Review?
Additional review may be necessary when AI changes the approved monitoring process, affects participant-related decisions, or introduces new data-processing arrangements.
The requirements depend on the study, institution, jurisdiction, and intended use.
Some AI applications may also fall within medical-device or clinical decision-support regulatory frameworks.
Researchers should consult appropriate institutional and regulatory personnel rather than assuming that general permission to use AI covers safety-related applications.
The World Medical Association's 2024 Declaration of Helsinki provides relevant ethical principles concerning participant protection and responsible medical research.
These principles do not automatically prohibit AI assistance, but they reinforce the need to prioritize participant rights, safety, and well-being.
Why Safety Monitoring Needs Ongoing Evaluation
An AI system's performance may change when participant characteristics, documentation practices, data sources, or model versions change.
Researchers should therefore determine whether ongoing monitoring is needed to detect performance deterioration or unexpected failures.
For consequential applications, this may involve reviewing missed events, incorrect alerts, response times, and changes in subgroup performance.
Investigators should also document significant system changes and determine whether additional validation or approval is required.
The National Institute of Standards and Technology's AI Risk Management Framework emphasizes managing AI risks throughout the system lifecycle.
Initial validation should not be assumed sufficient for every future use.
When Should AI Safety Monitoring Be Avoided?
AI may be inappropriate when its detection performance cannot be adequately established, when missed events could cause serious harm, or when the workflow lacks reliable human review and escalation.
It may also be unsuitable when necessary data processing is unauthorized or the system cannot be integrated safely into established monitoring procedures.
Researchers should not replace required safety assessments with an unvalidated general-purpose language model.
If adequate safeguards cannot be established, the appropriate decision may be to restrict or avoid AI use for the monitoring task.
This reflects the broader principle of risk-proportionate AI use in high-stakes research.
