03 · What You Need to Know
A Researcher's Readiness Framework for Serious Generative AI Use
First, Know What Kind of Output You Are Looking At
The most important conceptual shift is simple: an AI response is not one uniform kind of thing.
An AI assistant might generate text from its model, summarize a document you supplied, retrieve information from the web, invoke a calculator, execute code, query an external source, or combine several of these operations behind one conversational interface.
Those outputs do not deserve identical treatment.
If the system performs a calculation using a dedicated computational tool, you can evaluate that calculation. If it retrieves a paper, you can inspect the paper. If it generates an explanation from a large language model, you need to distinguish linguistic plausibility from factual verification.
Before relying on an AI answer, ask: What operation produced this output?
Generated Does Not Mean Retrieved
This distinction deserves to become almost automatic.
A generated citation is not necessarily a citation retrieved from a bibliographic database. A generated description of a study is not necessarily based on that study. A generated quotation is not necessarily a quotation found in a source. A generated policy explanation is not necessarily the current policy of the responsible organization.
Generative systems can produce highly specific false information. NIST identifies confabulation as a core generative-AI risk, referring to confidently presented erroneous or false content. Its Generative AI Profile also identifies risks involving data privacy, harmful bias, human-AI interaction, and information integrity.
This does not mean every generated statement is unreliable. It means the act of generation is not itself evidence that the statement has been verified.
Watch Out
Never allow the surface properties of an answer, including confidence, detail, academic vocabulary, numerical precision, citations, or polished formatting, to substitute for evidence that the answer is correct.
Know the Difference Between Generation and Verification
Generative AI can be excellent at proposing possibilities. Research requires an additional step: determining which possibilities survive contact with evidence.
Generation
Producing a candidate answer, explanation, classification, code segment, interpretation, citation, summary, or other output.
Verification
Establishing through appropriate independent evidence or testing whether the consequential output is accurate, valid, suitable, and supported.
If AI generates a reference, verify it in an authoritative bibliographic source and inspect the actual publication. If it generates code, execute and test the code. If it proposes an interpretation, return to the data, method, theory, and literature. If it describes a journal policy, check the current official policy.
Asking the same system, “Are you sure?” can sometimes prompt useful reconsideration. It is not independent verification.
Decide How You Will Verify the Output Before You Rely on It
A particularly useful habit is to plan verification before asking AI to perform a consequential task.
Suppose you want AI to classify 5,000 research records. Before running the classification, decide how performance will be evaluated. Will a human-coded validation set be used? Which errors matter most? What level of performance is acceptable? Will certain ambiguous cases receive human review?
If you generate analysis code, decide how you will test it before the output becomes part of the study.
If you cannot identify a credible verification strategy, that is important information about whether the task should be delegated to AI at all.
This principle follows directly from the distinction between tasks generative AI can technically help with and tasks for which its outputs can be used responsibly.
Understand the Consequence of Being Wrong
Verification should be proportional to consequence.
| AI use |
If the output is wrong |
Typical scrutiny needed |
| Suggesting a paper title |
The title may be poor or misleading |
Check accuracy and fit |
| Improving researcher-written prose |
The meaning or strength of a claim may change |
Compare carefully with the intended meaning and evidence |
| Summarizing a source |
The literature may be misrepresented |
Check important statements against the source |
| Generating references |
False or irrelevant evidence may enter the manuscript |
Verify existence, metadata, relevance, and support |
| Generating analysis code |
The reported findings may change |
Inspect, execute, test, and validate the implemented method |
| Classifying primary data |
The evidence base may be systematically distorted |
Validate performance, investigate errors and bias, and document the procedure |
| Interpreting findings |
The study may reach unsupported conclusions |
Ground interpretation in actual evidence, method, theory, uncertainty, and scholarship |
There is little value in applying an elaborate validation protocol to every disposable brainstorming prompt. Equally, a quick read-through is inadequate when AI determines the data entering a primary analysis.
Know What You Are Allowed to Put Into the System
Before uploading anything, consider the information itself.
Research materials may contain identifiable participant information, confidential records, unpublished findings, proprietary data, copyrighted material, embargoed results, peer-review manuscripts, grant proposals, institutional documents, or other restricted information.
Whether such material can be processed by a particular AI service depends on the actual service, deployment, contractual arrangements, ethics approval, participant consent, institutional rules, applicable law, licensing conditions, and other obligations.
UNESCO's guidance calls for a human-centred approach to generative AI and specifically emphasizes data privacy, human agency, ethical validation, inclusion, and capacity building.
The relevant question is therefore not “Can I upload this file?” The upload button already answers that technical question. Ask instead: Am I permitted to provide this information to this particular system under these conditions?
Do Not Assume “De-Identified” Means Risk-Free
Removing obvious identifiers can reduce privacy risk, but it does not automatically settle whether data may be provided to an external AI system.
Residual combinations of attributes may still permit re-identification. Ethics approval or participant consent may constrain where information can be processed. Contracts or institutional rules may impose additional restrictions.
De-identification is therefore one possible safeguard within a broader data-governance decision, not a universal permission slip.
Know Which Decisions Must Remain Yours
Generative AI can suggest almost every component of a research project. That does not mean researchers should surrender every component.
AI can propose research questions, methods, hypotheses, coding categories, statistical explanations, themes, interpretations, and conclusions. Those suggestions can be valuable.
But core scholarly responsibilities should remain under meaningful human control, particularly final methodological judgment, ethical responsibility, authoritative verification, interpretation, and conclusions.
A useful test is whether you can defend the decision without saying, “That is what the AI recommended.”
Know Enough About the Method to Evaluate AI-Generated Work
Generative AI can lower the technical barrier to performing sophisticated tasks. A researcher can obtain Python code without being an experienced programmer, generate a structural equation model specification without having implemented one before, or receive an explanation of an unfamiliar qualitative methodology.
This can support learning and interdisciplinary work.
It can also create an evaluation gap.
If you cannot determine whether the generated analysis is methodologically appropriate, your approval does not become meaningful oversight merely because you are human.
You do not need to be the world's leading expert in every method you use. Research has always relied on collaboration and specialist expertise. But someone responsible for consequential work needs enough competence to evaluate it properly.
Understand That AI Can Be Wrong in Ways You Do Not Expect
Researchers sometimes learn one AI risk and then guard only against that risk.
They check for fabricated citations but overlook altered numerical values in a summary. They protect privacy but accept inappropriate analysis code. They verify factual claims but fail to notice that generated editing changed “associated with” into “predicted” or “caused.”
NIST's Generative AI Profile describes a broader risk landscape including confabulation, data privacy, harmful bias or homogenization, human-AI configuration, information integrity, and other concerns.
This is why generative AI safeguards should be designed around the actual failure mode rather than reduced to the instruction “check for hallucinations.”
Do Not Confuse Better Writing With Better Research
Generative AI can dramatically improve the appearance of a manuscript.
It can make arguments smoother, methods sound more sophisticated, limitations appear more complete, and conclusions more polished.
Those can be genuine communication improvements.
They do not automatically improve sampling, measurement, research design, analysis, evidential support, or inference.
Indeed, poor research can be made to look more rigorous than it really is when generated methodological language outruns what the study actually did.
Before accepting an AI-improved passage, ask what changed: the research, the explanation of the research, or merely the prose?
Know the Difference Between Productivity and Quality
Generative AI can make research faster. Faster is useful.
But a literature summary produced in two minutes is not better because it took two minutes. Code written instantly is not more valid. A manuscript completed earlier is not automatically more rigorous.
Research quality improves only when something substantively valuable improves: an error is detected, an alternative explanation is examined, documentation becomes more accurate, an analysis becomes reproducible, communication becomes clearer without distortion, or another relevant quality criterion is strengthened.
This distinction matters when evaluating whether generative AI actually improves research quality rather than merely increasing output.
Know When AI Is Generating and When It Is Using External Tools
Modern AI assistants increasingly combine language models with search, document retrieval, code execution, calculators, databases, and other tools.
This can improve reliability substantially for particular tasks, but it also makes the interface deceptive in a benign sense: two answers appearing in the same chat window may have been produced through completely different mechanisms.
A numerical answer produced by executed code should be evaluated differently from arithmetic generated directly as text. A literature record retrieved from an external database should be evaluated differently from a citation produced from the model's learned patterns.
Researchers should therefore understand the workflow well enough to identify where evidence, retrieval, computation, and generation enter the answer.
Know That AI Systems and Policies Change
Generative AI is not a stable category of software with one permanent set of capabilities.
Models change. Applications add search and tool use. Data-handling arrangements differ across deployments. Institutional rules evolve. Journals revise disclosure policies. Funders develop guidance.
The European Commission updated its living guidelines again in May 2026 to reflect technological developments and emerging research risks, while retaining principles such as accountability, transparency, responsibility, and research integrity. The update also added recommendations concerning third-party AI use during meetings or information management and risks from instructions hidden from human oversight.
For this reason, advice remembered from an earlier AI tool, publisher policy, or institutional memo should not automatically be assumed current.
Check the Policies That Actually Govern Your Research
There is no single global AI policy for researchers.
Relevant requirements may come from:
- your university or research organization;
- an ethics or institutional review body;
- a research funder;
- a journal or publisher;
- a research consortium or data provider;
- a professional or disciplinary body;
- contracts, licenses, or data-use agreements;
- applicable national or regional law.
These rules can address different issues and need not use identical definitions of AI assistance, disclosure, confidentiality, authorship, or acceptable use.
Verify current requirements from the responsible authority rather than asking a general-purpose AI system to decide which policy applies.
Know What Needs to Be Documented
Documentation should be proportionate to the role of AI.
If AI suggests a title that you never use, retaining a detailed record may serve little research purpose. If AI classifies primary data or generates code used in the final analysis, the system, procedure, validation, and human review may be important for reproducibility and methodological evaluation.
Depending on the task, useful records may include:
- the AI system or relevant model;
- the task assigned to it;
- important prompts or instructions;
- the data or documents supplied;
- relevant settings or tool connections;
- generated outputs that entered the research process;
- validation procedures;
- human corrections or decisions;
- dates or versions where system changes could matter.
Do not document merely to create an impressive AI appendix. Preserve what someone would reasonably need to understand, evaluate, reproduce, or audit the consequential part of the workflow.
Know the Difference Between Documentation and Disclosure
These concepts are related but not identical.
Documentation
Maintaining records of how AI was used so the workflow can be understood, checked, reproduced, or audited where necessary.
Disclosure
Communicating AI use to readers, journals, institutions, funders, participants, or other relevant parties when required or methodologically appropriate.
You may need internal documentation even when a particular use does not require a manuscript disclosure. Conversely, a publisher may prescribe a disclosure format that does not capture every technical detail preserved in your research records.
Follow the relevant authority's requirements rather than assuming one generic AI statement satisfies every purpose.
Know Who Is Responsible Before Something Goes Wrong
If AI-generated code affects the analysis, who checks it? If a research assistant uses AI for classification, who validates the classifications? If a collaborator uploads shared data to an external system, who determines whether that was permitted?
These questions should not first appear during an integrity investigation.
Research teams should establish responsibilities for consequential AI use in advance. As explained in the question of responsibility when AI contributes to a research error, accountability may be distributed across researchers, teams, institutions, and other actors, but AI itself does not become the accountable scholar.
Know That “Human in the Loop” Is Not Enough
A human can be present and still contribute almost no meaningful oversight.
If a researcher automatically accepts AI classifications, copies generated interpretations, or approves code they cannot understand, the workflow technically contains a human while functionally depending on the AI.
NIST identifies human-AI configuration risks including automation bias and over-reliance. UNESCO similarly emphasizes protecting human agency and developing the human capacity needed to evaluate generative AI rather than allowing technology to displace human judgment.
The relevant question is not whether a human clicked the final button. It is whether an appropriately informed human exercised meaningful judgment.
Know When Not to Use Generative AI
AI literacy includes knowing when another method is better.
Do not use a language model for a calculation when a reliable computational tool is more appropriate. Do not ask it to invent references when a scholarly database can retrieve them. Do not upload restricted data merely because processing them manually would take longer. Do not delegate a methodological decision you cannot evaluate.
Sometimes the best AI workflow is no AI workflow.
This is not technological conservatism. It is ordinary research-tool selection.
A Serious Research Workflow Should Survive the “Without the AI” Test
Imagine that the AI conversation disappears tomorrow.
Could you still explain where your evidence came from? Could you reproduce the analysis from your data and code? Could you justify the method? Could you identify the sources supporting your claims? Could you explain why the conclusions follow?
You do not necessarily need to reproduce every generated sentence manually. The test is whether the intellectual and evidential basis of the research remains defensible without treating the AI conversation itself as authority.
If the answer is no, the AI may have become more than an assistant. It may have become an unsupported dependency in the chain of evidence.
