03 · What You Need to Know
How to Evaluate Privacy and Security Before Selecting an AI Research Tool
Why Security Depends on the Research Task
The protections required for an AI tool depend on the information it will process and the consequences of unauthorized access, disclosure, modification, or loss.
For example, generating code from synthetic data presents different concerns from uploading identifiable health interviews.
A service suitable for public literature summaries may be inappropriate for restricted participant records.
Researchers should therefore begin with the task and information classification rather than the provider's general reputation.
The relevant question is whether the complete processing arrangement meets the study's requirements.
Privacy and Security Are Related but Different
Privacy
Concerns the appropriate and authorized processing of personal information, including purpose, transparency, disclosure, retention, and applicable individual rights.
Information security
Concerns protecting information and systems against unauthorized access, alteration, disclosure, disruption, or loss.
A system can use strong encryption while processing information for a purpose that the researcher is not authorized to permit.
Conversely, an authorized processing purpose may still involve inadequate security safeguards.
Researchers should evaluate both dimensions where relevant.
Begin by Classifying the Research Information
Before selecting an AI service, researchers should identify the information the task requires.
Relevant categories may include public information, internal institutional records, confidential research materials, personal information, and highly restricted data.
Classification should consider direct identifiers and contextual information that could permit reidentification.
For example, removing names from interview transcripts does not necessarily eliminate references to unique experiences or identifiable roles.
The required safeguards depend on the information's actual sensitivity and governing restrictions.
Which Security and Privacy Features Should Researchers Examine?
| Assessment area |
Question to investigate |
Potential concern |
| Data retention |
How long are prompts, files, outputs, and logs retained? |
Retention incompatible with research obligations |
| Model training |
Can submitted information be used to improve models? |
Unauthorized secondary processing |
| Access controls |
Who can access the information? |
Excessive permissions or unauthorized disclosure |
| Encryption |
How is information protected in transit and at rest? |
Exposure during storage or transmission |
| Subprocessors |
Which additional entities may process information? |
Unexamined third-party processing |
| Data location and transfers |
Where may processing occur? |
Additional jurisdictional obligations |
| Deletion |
What information can be deleted, and under what conditions? |
Uncontrolled retained copies |
| Audit logging |
Can consequential access and operations be traced? |
Insufficient accountability |
| Incident response |
How are security incidents detected and handled? |
Delayed containment or notification |
| Connected applications |
What external files or systems can the AI access? |
Unintended exposure of unrelated records |
The importance of each area depends on the research task. These questions are a practical assessment framework rather than a universal certification standard.
Does Encryption Make an AI Tool Safe?
Encryption can protect information during transmission and storage, depending on its implementation.
However, encryption does not determine whether the provider is authorized to process the information.
It also does not necessarily prevent access by parties who possess legitimate system permissions.
Researchers should examine encryption alongside access controls, authentication, contractual arrangements, and other safeguards.
Security should be assessed as a system of controls rather than a single technical feature.
What Does a No-Training Commitment Actually Mean?
Some AI providers offer arrangements under which customer information is not used to train or improve models.
This may address an important concern about secondary data use.
However, restrictions on model training do not necessarily mean that submitted information is never stored, logged, accessed, or processed for other permitted purposes.
Researchers should examine the actual contractual terms and service configuration.
A no-training statement should not be interpreted as a complete confidentiality or deletion guarantee.
Why Data Retention Deserves Attention
Research information may be subject to institutional retention schedules, ethics conditions, funding agreements, or legal requirements.
An AI service may apply different retention rules to uploaded files, conversation histories, application logs, and other records.
Researchers should determine whether these arrangements are compatible with the project's obligations.
For example, a study may require identifiable recordings to be deleted after a specified period.
Using an external AI service should not create uncontrolled copies that undermine that requirement.
Researchers should verify deletion mechanisms and avoid assuming that removing a conversation from an interface deletes every associated record.
Who Can Access Information Submitted to the AI System?
Access may be available to the researcher, authorized collaborators, institutional administrators, service personnel, or subprocessors, depending on the system and agreement.
Researchers should determine which parties can access the information and for what purposes.
Access controls should follow the principle of least privilege, limiting permissions to what is necessary for authorized tasks.
For connected AI applications, permissions may extend beyond information explicitly entered into prompts.
The scope of these integrations should be evaluated before activation.
Why Provider Terms Must Be Examined for the Specific Service
AI providers may offer consumer, enterprise, educational, and application programming interface services with different processing conditions.
Researchers should not assume that protections described for one service automatically apply to another.
The relevant agreement, account configuration, and administrative settings should be identified.
For example, an institutionally contracted AI environment may provide safeguards unavailable through an individual's consumer account.
However, even an approved enterprise service may restrict particular categories of confidential information.
Authorization should be confirmed for the actual research use.
What Is a Data Processing Agreement?
A data processing agreement may establish contractual obligations between organizations concerning the processing of personal information.
Its requirements depend on the applicable legal framework and the parties' actual roles.
For example, Article 28 of the European Union General Data Protection Regulation addresses contractual requirements for processing carried out by a processor on behalf of a controller.
Researchers should not assume that every AI provider acts in the same legal capacity for every service.
Where an agreement is required, the responsible institutional personnel should evaluate whether it covers the intended processing.
When Is a Privacy Impact Assessment Necessary?
Some processing activities may require a formal privacy or data protection impact assessment.
Under GDPR Article 35, a data protection impact assessment is required for certain processing likely to result in high risk to individuals' rights and freedoms.
Other jurisdictions and institutions may establish different assessment requirements.
Not every AI research task automatically requires a formal assessment.
Researchers should consult the responsible privacy or data protection office when the proposed activity involves substantial or uncertain risks.
What Should Researchers Consider Under Philippine Data Protection Law?
In the Philippines, Republic Act No. 10173, the Data Privacy Act of 2012, establishes the principal legal framework for personal information processing.
Relevant considerations include transparency, legitimate purpose, proportionality, lawful processing, and appropriate security measures.
Researchers should also consider applicable National Privacy Commission issuances and institutional policies.
The law should not be interpreted as requiring separate AI-specific approval for every research interaction.
However, processing personal or sensitive personal information through AI may require evaluation of the lawful basis, disclosure arrangements, security safeguards, and organizational responsibilities.
Where requirements are uncertain, the institution's data protection officer or qualified legal personnel should provide guidance.
Does Institutional Approval of an AI Tool Establish Suitability?
Institutional approval may provide evidence that a service has undergone relevant procurement, security, or governance review.
However, approval may apply only to specified tasks, data classifications, or configurations.
For example, a university may approve an AI service for drafting administrative documents while restricting its use with identifiable participant data.
Researchers should establish whether the authorization covers the particular information and processing purpose.
General tool approval and task-specific authorization should not be treated as interchangeable.
Can Locally Hosted AI Provide Better Protection?
Locally deployed AI may reduce reliance on external processing services and provide greater control over some information flows.
However, local deployment is not automatically secure.
Risks may arise from inadequate access controls, insecure storage, unpatched software, inappropriate logging, or unauthorized connections.
Researchers should evaluate the actual implementation and institutional responsibilities.
Local processing may be preferable for some restricted datasets when the necessary safeguards can be established.
The decision should follow the complete risk assessment rather than the physical location of the model alone.
How Do AI Integrations Change the Security Assessment?
AI systems may connect to institutional repositories, research databases, shared drives, or external applications.
These integrations can expand the information available to the system.
Researchers should examine whether the connection grants access to unrelated confidential records or permits actions beyond the intended task.
Relevant safeguards may include restricted permissions, authentication, monitoring, and appropriate review of connected services.
The security assessment should cover the complete workflow, including connected tools, rather than only the underlying language model.
Why Prompt Injection Matters for Connected Research Systems
Prompt injection occurs when untrusted content attempts to influence an AI system's behavior contrary to its intended instructions.
For example, a retrieved document may contain text directing an AI assistant to disclose information or perform an unauthorized action.
The risk is particularly relevant when AI systems can access connected files, execute tools, or perform operations across applications.
Researchers should evaluate whether the system adequately separates untrusted content from authorized instructions.
Appropriate permissions, review procedures, and technical controls may reduce the consequences of such attacks.
Simply instructing the model to ignore malicious content should not be treated as a complete security control.
How Should Researchers Assess Security Certifications?
Security certifications and independent assurance reports may provide useful evidence about an organization's controls.
However, their relevance depends on scope, validity, and the services covered.
For example, a certification covering one operational environment may not establish that every AI product or integration offered by the provider is included.
Researchers should examine whether the assurance applies to the selected service and processing arrangement.
Certification may support a security assessment, but it does not independently establish research ethics approval or permission to process particular data.
What Should Researchers Do if Provider Terms Are Unclear?
Unclear terms can prevent researchers from establishing whether information processing is authorized.
The appropriate response may involve obtaining clarification from the provider, consulting institutional procurement or data protection personnel, or selecting another service.
Researchers should not infer favorable processing conditions merely because the provider does not explicitly describe a particular risk.
Where essential information remains unavailable, the system may be unsuitable for confidential research.
Uncertainty about a provider's arrangements should be treated as a governance issue rather than ignored for convenience.
How Should Researchers Prepare for Security Incidents?
Research teams should know how to respond if information is exposed through unauthorized AI processing, compromised credentials, or insecure integrations.
Relevant procedures may include containment, documentation, notification to responsible institutional personnel, and assessment of reporting obligations.
The required response depends on the information, incident, jurisdiction, and institutional policies.
Researchers should not assume that deleting an uploaded file or conversation fully resolves an exposure.
Incident response responsibilities should be established before consequential processing begins.
What Is Residual Security Risk?
Residual risk is the risk remaining after relevant safeguards have been applied.
Encryption, access controls, contractual commitments, and monitoring may reduce particular risks without eliminating them entirely.
Researchers should evaluate whether the remaining risks are acceptable for the information and intended task.
Some processing may remain inappropriate even when a provider offers substantial technical safeguards.
For example, a contractually prohibited disclosure cannot automatically be justified by strong encryption.
When Should Researchers Avoid an AI Tool?
Researchers should not submit restricted information to an AI system when essential authorization or protection cannot be established.
They should also reconsider use when provider terms are incompatible with research obligations, security controls are insufficient, or connected permissions create unacceptable exposure.
Alternatives may include an approved institutional service, an appropriately controlled local deployment, synthetic examples, or conventional analytical software.
The preferred option should satisfy both the scientific requirements and the information protection obligations of the study.
