03 · What You Need to Know
Where Confidentiality Risks Remain in Locally Deployed AI
Local processing changes the exposure pathway, not the sensitivity of the data
Local inference means that a model generates outputs on computing infrastructure controlled by the researcher or institution. If the entire workflow operates without external processing, confidential inputs need not be transmitted to a commercial model provider.
This can be a meaningful privacy advantage, especially when external processing is prohibited or requires additional authorization. However, the information remains confidential regardless of where it is processed.
A transcript containing a participant's name, health history, or recognizable personal circumstances does not become anonymous because it is analyzed offline. Nor does an unpublished manuscript become publicly shareable because a locally installed model has summarized it.
The privacy advantages of running AI locally should therefore be understood as reductions in particular exposure pathways, not as elimination of confidentiality obligations.
Which parts of a local AI workflow can contain sensitive information?
Researchers often focus on the original input file while overlooking information created during processing.
Depending on the software and configuration, a local AI application may create prompt histories, generated responses, temporary documents, diagnostic logs, indexes, embeddings, or cached files.
These materials may contain direct identifiers, verbatim quotations, sensitive classifications, or sufficient contextual information to identify research participants.
| Information location |
Potential confidentiality risk |
Appropriate safeguard |
| Original research files |
Unauthorized reading, copying, or modification |
Restricted access, approved storage, and encryption where appropriate |
| Prompt and response histories |
Retention of sensitive passages or identifying details |
Controlled logging, access restrictions, and retention limits |
| Temporary files and caches |
Unexpected copies outside the approved research directory |
Inspect application storage and establish secure cleanup procedures |
| Embeddings and retrieval indexes |
Potential disclosure of information derived from confidential documents |
Protect derived representations and evaluate their disclosure risks |
| Backups and synchronized folders |
Persistence or transmission of information beyond the intended environment |
Review backup destinations, synchronization, and retention arrangements |
| Exported AI outputs |
Accidental disclosure of identifiable or sensitive information |
Review outputs before sharing and apply appropriate access controls |
Not every application creates all these materials. Researchers should inspect the actual software rather than assuming either that every risk is present or that none exists.
Why application logs deserve particular attention
Logs help developers and administrators diagnose problems, monitor system performance, and reconstruct processing events. In research environments, however, logging can create additional records of sensitive information.
A log may record only technical metadata, such as timestamps and processing duration. Another application may retain full prompts, generated responses, file paths, or error messages containing confidential excerpts.
Researchers should establish what is logged, where logs are stored, who can access them, and how long they are retained.
Disabling all logging is not necessarily appropriate. Some records may be needed for security monitoring, research auditing, or reproducibility. The objective is to retain information that serves a legitimate purpose without unnecessarily duplicating confidential content.
Where feasible, technical metadata can be separated from identifiable research inputs. Any retained sensitive logs should receive protections consistent with their contents.
Can an offline computer still expose confidential research data?
Yes. Disconnecting a computer from the internet removes certain network communication pathways, but it does not prevent every form of unauthorized access or disclosure.
Possible exposure routes include compromised software installed before disconnection, removable storage devices, physical theft, shared user accounts, unauthorized local access, and files copied when the system is later reconnected.
Offline systems also require maintenance. Security updates may need to be transferred through controlled procedures, and researchers must consider how software dependencies and model files are obtained.
An isolated environment can be valuable for particularly sensitive research, but its security depends on how isolation is established and maintained.
It would be inaccurate to describe a computer as completely secure merely because it has no active internet connection.
Why access controls matter even when only one researcher uses the model
A locally deployed AI system may run on a shared workstation, laboratory server, or institutionally managed computer. Other users or administrators may have legitimate access to parts of the environment.
If accounts share permissions, confidential inputs and generated outputs may be visible to people who are not authorized to examine the research data.
Access controls should reflect the approved research roles. This may involve separate accounts, appropriate authentication, restricted directories, and controlled administrative privileges.
Researchers should also consider physical access, unattended sessions, and the handling of exported files.
The NIST SP 800-53 security and privacy controls provide an authoritative reference for organizational approaches to access control, audit, identification, system protection, and related safeguards. The appropriate controls must be selected according to the actual risk rather than applied mechanically.
Can locally generated AI outputs reveal confidential information?
Yes. AI-generated outputs may reproduce names, quotations, unusual experiences, or combinations of characteristics that identify individuals.
For example, a model asked to summarize interviews may produce a passage describing a participant's program, workplace, family circumstances, and unusual academic experience. Even without the participant's name, the combination may be recognizable to someone familiar with the setting.
Confidentiality review must therefore extend beyond the original dataset.
Researchers should assess whether summaries, classifications, extracted information, and analytical notes contain details that require protection.
Generated text should not be assumed anonymous merely because the model has paraphrased the original material.
Do embeddings and vector databases create confidentiality concerns?
They can. Some local AI workflows use retrieval-augmented generation, in which documents are processed into representations that support information retrieval.
These representations may be stored in a vector database or another searchable index.
Although an embedding is not the same as the original document, researchers should not assume that it is anonymous or incapable of revealing sensitive information. Disclosure risks depend on the representation, access conditions, associated metadata, and possible attacks.
Researchers should treat derived representations of confidential data as potentially sensitive until an appropriate assessment supports a different classification.
Deleting the original document while retaining an unprotected retrieval index may leave relevant information accessible within the system.
Can prompt injection compromise a local research workflow?
Potentially, especially when a model is connected to tools or permitted to perform actions.
Prompt injection occurs when untrusted content attempts to redirect a model's behavior away from the authorized task. For example, a document being analyzed might contain instructions telling an AI assistant to disclose other files or change its processing procedure.
Text inside a research document should be treated as data, not as an instruction that automatically overrides the researcher's intended workflow.
A local model that only generates text within a restricted environment has a different risk profile from an AI agent that can read directories, execute commands, access networks, or export files.
Researchers should restrict tool permissions, separate trusted instructions from untrusted content, and avoid giving AI applications broader access than the task requires.
Local deployment does not make a tool-connected AI system immune to malicious instructions embedded in documents.
Does local fine-tuning introduce additional confidentiality risks?
It may. Fine-tuning modifies a model using additional training data, which can include research materials.
If confidential information is used during fine-tuning, researchers should consider whether the resulting model could retain or reproduce sensitive content under particular conditions.
The risk depends on the training procedure, dataset, model, and subsequent access arrangements. It should not be assumed that every fine-tuned model memorizes all its training examples, nor that memorization is impossible.
Research by Carlini and colleagues demonstrated that training data can sometimes be extracted from language models under particular experimental conditions. This establishes a relevant class of risk, not a universal extraction rate for every locally trained model.
Researchers should distinguish ordinary inference, where inputs are processed without necessarily changing model weights, from fine-tuning, where information may influence the resulting model parameters.
Before fine-tuning on confidential data, assess whether the procedure is authorized and whether access to the resulting model requires additional restrictions.
Why deletion and retention require more than removing a file
Research protocols may specify retention periods for identifiable data and procedures for secure disposal.
Deleting a transcript from the main research folder does not necessarily remove copies from backups, temporary storage, synchronized directories, application histories, or retrieval indexes.
Researchers should identify where data and derived materials are retained before establishing deletion procedures.
Secure disposal requirements depend on the storage medium, technical environment, institutional policies, and applicable obligations.
For example, deleting files on encrypted storage, managing encryption keys, and handling backups may involve different procedures from removing ordinary documents from an unprotected directory.
The relevant question is whether the retention and disposal arrangement adequately addresses the locations where confidential information actually exists.
Can local AI processing violate research ethics or data-protection requirements?
Yes. A secure processing environment does not independently establish permission to process information for a particular purpose.
Researchers must consider participant consent, the approved research protocol, institutional data-governance requirements, and applicable law.
In the Philippines, Republic Act No. 10173, the Data Privacy Act of 2012, provides a framework governing personal information processing. Its applicability and requirements depend on the processing circumstances.
Researchers should not assume that keeping personal data on a local device removes obligations concerning lawful processing, proportionality, security, and data-subject rights.
Where the proposed AI use materially changes an approved analytical procedure or data-processing arrangement, institutional ethics or data-protection review may be necessary.
The distinction is important: technical confidentiality controls address how information is protected, while ethical and legal authorization concerns whether and under what conditions it may be processed.
How should researchers evaluate local AI confidentiality risks?
A practical assessment begins with the information being processed and the people or systems that could access it.
Researchers should identify the original data, derived files, model outputs, storage locations, user permissions, network connections, and relevant retention arrangements.
They should then consider plausible exposure scenarios and the consequences of each.
For example, unauthorized access to publicly available article abstracts may have limited confidentiality consequences. Exposure of identifiable interviews involving vulnerable participants could be considerably more serious.
The NIST Privacy Framework provides a structured approach to privacy risk management, while the NIST AI Risk Management Framework supports context-sensitive evaluation of AI-related risks.
When the research involves particularly sensitive data, the assessment should involve appropriate institutional information-security, ethics, or data-protection personnel.
Researchers considering different AI deployment arrangements should compare their actual safeguards rather than treating local and cloud labels as complete descriptions of risk.
