Can Research Prompts Be Used to Train or Improve AI Models?

Could What You Type Into AI Be Used to Improve the Model?

A researcher asks an AI system to critique an unpublished argument, interpret a methodological problem, edit a manuscript passage, or analyze research material. The response appears within seconds. But another question remains: could the researcher's input later contribute to improving the AI system itself?

Sometimes it can. Sometimes the applicable service explicitly excludes customer content from model training. In other cases, the answer depends on a setting controlled by the user or organization.

The phrase "AI uses your data" is therefore too imprecise to answer the question. Researchers need to know what the particular provider means by training, model improvement, product improvement, feedback, and related forms of data use.

It Depends on the Service, Account, and Settings

Model Training Is Only One Possible Use of Your Research Prompt

AI models can be improved using data from real interactions

AI providers may improve models using various sources of data, which can include eligible user interactions. Such data can help developers evaluate model behavior, identify weaknesses, improve safety, or train later versions of models.

OpenAI currently states that content submitted through its services for individuals, such as ChatGPT, may be used to improve model performance depending on the user's settings. This content can include prompts, responses, images, and files. Eligible personal accounts can turn off "Improve the model for everyone," after which new conversations are not used to train ChatGPT.

Google similarly states that when Keep Activity is enabled in personal Gemini Apps, activity can be used to provide, develop, and improve Google services, including training generative AI models. Google allows users to manage whether future chats are used to improve Google AI through Gemini Apps activity controls.

These policies can change. The relevant question is therefore not whether consumer AI systems have ever used conversations for improvement, but whether your current service and configuration permit your content to be used in that way.

Business and institutional services may operate differently

Researchers should be particularly careful about extrapolating from a personal AI account to an institutionally provided one, or vice versa.

OpenAI currently states that, by default, it does not use inputs or outputs from ChatGPT Business, Enterprise, Edu, Healthcare, or its API Platform to train or improve its models. Microsoft states that prompts, responses, and organizational data accessed through Microsoft Graph in Microsoft Copilot are not used to train foundation models under its enterprise data protection arrangements.

These are product-specific commitments, not a universal characteristic of enterprise AI. An institution considering AI for research should examine its actual contractual and technical arrangement rather than assume that an "enterprise" label automatically produces a particular data policy.

The table illustrates why there is no meaningful universal answer such as "Chatbots train on everything you type" or "AI companies do not train on your prompts." Both statements flatten product-specific arrangements into slogans.

Storage does not prove that training occurs

A provider can retain a conversation without using that conversation to train a foundation model. Microsoft Copilot under enterprise data protection is a clear example: Microsoft states that prompts and responses are logged and retained for organizational capabilities such as audit and eDiscovery, while also stating that those prompts and responses are not used to train foundation models.

Similarly, turning off model improvement does not necessarily prevent a conversation from being stored. OpenAI states that conversations can remain in ChatGPT history after "Improve the model for everyone" has been turned off.

Researchers should therefore investigate whether prompts are stored separately from whether they may contribute to model improvement.

Turning off training is useful, but it is not a confidentiality waiver

Suppose you disable a setting that allows your conversations to improve the model. That can materially change how eligible content is used. It does not, however, automatically make confidential research appropriate to submit.

The information still has to be processed to generate the response. Depending on the service, some retention may still occur. Other authorized forms of access or processing may remain possible. More fundamentally, your ethics approval, participant consent, confidentiality agreement, institutional policy, collaborator arrangement, or contract may prohibit the disclosure regardless of whether the provider trains on the content.

Human review can intersect with model improvement

Some services use human reviewers in processes related to safety, evaluation, or improvement. Google states that a subset of Gemini Apps chats can be reviewed by human reviewers, including trained service providers, to help improve Google services and keep the service safe. Its privacy documentation advises users not to submit confidential information they would not want a reviewer to see or Google to use to improve its services.

This is particularly relevant to researchers because seemingly ordinary prompts may contain confidential passages copied directly from a dataset, manuscript, proposal, or review file.

Files and other uploaded content can matter too

Do not interpret "prompt training" as applying only to the words typed into a chat box. Depending on the product and settings, eligible content can include files, images, or other material submitted during an interaction.

OpenAI's consumer-data documentation, for example, states that eligible content used to improve model performance can include prompts, responses, images, and files. Google likewise explains that Gemini Apps activity can include files, videos, screens, and photos shared with the service.

For researchers, uploading an entire unpublished document can therefore present a much larger disclosure than asking a generic methodological question.

Feedback can have separate rules

Another subtlety is that voluntarily providing feedback can sometimes change how an interaction is handled. Some providers distinguish ordinary service use from content deliberately submitted to help improve the system.

Researchers should therefore inspect any feedback mechanism before attaching sensitive material or providing detailed corrections containing research information. A service that excludes ordinary organizational content from training by default may still provide explicit mechanisms through which customers can opt in to sharing data for improvement.

"Training" itself can be used loosely

In casual discussion, people often use "training" to describe almost any process through which an AI service learns, evaluates performance, personalizes responses, or improves a product. Provider documentation may distinguish foundation-model training from evaluation, safety monitoring, personalization, retrieval, fine-tuning, product analytics, or other forms of processing.

That distinction matters. A statement that data are not used to train foundation models does not necessarily answer every question about how the service processes those data.

The broader lifecycle of research information submitted to AI therefore remains relevant even when the training question has been resolved.

The Same Research Prompt Can Have Different Data-Use Rules

The correct decision therefore requires two separate questions: Can this content be used for model improvement? and Am I permitted to provide this content to the service in the first place?

Common Misunderstandings About Research Prompts and AI Training

Verify Model-Improvement Rules Before Submitting Non-Public Research

The training question matters most when the information in your interaction is not already public. A generic question about statistical power carries very different consequences from uploading an unpublished dataset or describing a patentable result.

For confidential and unpublished material, the safest approach is to decide whether disclosure is appropriate before worrying about the quality of the AI response. The later discussion of the safest default for confidential or unpublished research information provides a conservative starting point.

Check Whether Your Research Prompt Can Be Used for AI Improvement

Questions About Research Prompts and AI Training

Do Not Assume Either That Every Prompt Trains AI or That None Do

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.