03 · What You Need to Know
Teaching Researchers to Recognize the Boundaries of Appropriate AI Use
Knowing How to Use AI Is Different From Knowing Whether to Use It
Researchers may become proficient in prompting, evaluating generated text, and using AI-assisted analytical tools without developing sufficient judgment about when those tools are appropriate.
Technical competence concerns whether a researcher can operate a system and interpret its outputs.
Appropriateness concerns whether using the system is justified within a particular research context.
For example, a researcher may know how to upload interview transcripts and request thematic analysis. That knowledge does not establish permission to process confidential participant material through an external service.
Similarly, a researcher may know how to generate statistical code without understanding whether the proposed analytical procedure is appropriate.
Training should therefore distinguish technical capability from ethical authorization and methodological suitability.
Why Responsible AI Training Should Include Non-Use Decisions
AI education often emphasizes productive applications, such as literature exploration, academic writing, coding, and analytical assistance.
These applications can be valuable, but a curriculum focused exclusively on successful use may create the impression that every research problem should involve AI.
Responsible research requires selecting methods and tools according to their suitability for the task.
Sometimes the appropriate decision is to use AI with verification. In other circumstances, it is to use a different tool, seek specialist expertise, or proceed without AI.
The UNESCO Guidance for Generative AI in Education and Research (2023) emphasizes human-centered governance and the need to address risks associated with generative AI.
Similarly, the NIST AI Risk Management Framework (2023) provides a structured approach to governing, mapping, measuring, and managing AI-related risks.
These frameworks support teaching researchers to evaluate use cases rather than treating adoption as an automatic objective.
What Does It Mean to Decide Not to Use AI?
Non-use does not necessarily mean rejecting AI technology in general.
It may mean avoiding a particular system, refusing to upload certain information, excluding AI from a specific decision, or postponing use until safeguards are established.
For example, a researcher may use AI to explain a statistical concept while declining to accept its recommendation about the final analytical model without methodological review.
A laboratory may permit AI-assisted manuscript editing but prohibit the use of unapproved external services for confidential participant records.
These decisions reflect boundaries between acceptable assistance and inappropriate delegation.
The distinction is important because responsible AI use is not a binary choice between unrestricted adoption and complete prohibition.
1. Avoid AI When Its Use Violates Applicable Requirements
The clearest reason not to use a particular AI system is that the proposed activity conflicts with binding institutional, contractual, ethical, or legal requirements.
For example, a university may prohibit uploading confidential research information to unapproved external platforms.
A research ethics approval may specify how participant data can be stored and processed.
A publisher or funder may establish restrictions concerning particular forms of AI assistance.
Researchers should identify the actual requirements before using the technology.
AI-generated interpretations of institutional rules should not substitute for consulting the authoritative policy or responsible office.
Where requirements are unclear, the appropriate response is to seek clarification rather than assume that technological availability constitutes permission.
2. Avoid Unauthorized Processing of Confidential Research Data
Research datasets may contain identifiable participant information, sensitive interviews, unpublished institutional records, or confidential intellectual property.
Uploading such material to an AI service may introduce risks associated with access, retention, secondary processing, and disclosure.
These risks depend on the provider, system configuration, contractual protections, and applicable institutional arrangements.
Removing direct identifiers may reduce risk but does not necessarily eliminate the possibility of re-identification.
Researchers should not upload restricted information until the proposed processing has been appropriately authorized.
Where AI-assisted practice is educationally useful, synthetic or appropriately approved datasets may provide an alternative.
3. Avoid Delegating Decisions That Require Qualified Human Judgment
Some research decisions require professional expertise, contextual understanding, or formal authority.
Examples may include determining participant eligibility, interpreting clinically significant information, deciding whether an adverse event requires action, or approving consequential changes to a research protocol.
AI may sometimes support these activities through appropriately validated and authorized systems.
However, it should not independently assume responsibilities assigned to qualified researchers or oversight bodies.
For example, a generative AI response should not be treated as sufficient authority to change participant exclusion criteria.
Researchers should identify who is accountable for the decision and whether the system has been evaluated for the intended purpose.
The objective is meaningful human oversight rather than ceremonial approval of automated recommendations.
4. Avoid AI When Errors Could Cause Serious Harm Without Adequate Safeguards
The consequences of error should influence whether AI is appropriate.
A mistaken suggestion about manuscript wording generally presents different risks from an incorrect recommendation affecting participant safety.
High-stakes tasks may require validated systems, independent review, formal authorization, and additional safeguards.
Where these protections are unavailable or insufficient, avoiding AI may be the more defensible decision.
The NIST AI Risk Management Framework emphasizes that AI risks depend on context and potential impacts.
Researchers should therefore evaluate the specific application rather than assume that a system's general performance establishes suitability for every use.
5. Avoid AI When It Would Replace the Competency Being Assessed
Research education sometimes requires students to demonstrate particular abilities independently.
For example, a research methods examination may assess whether students can formulate research questions, evaluate study designs, or interpret statistical outputs.
If AI supplies the reasoning being assessed, the resulting performance may not provide adequate evidence of the student's competence.
AI restrictions can therefore be justified by assessment validity.
However, this does not mean AI should be prohibited throughout research education.
Other activities may explicitly assess students' ability to evaluate automated recommendations.
The distinction is central to deciding when AI restrictions are appropriate in research methods training.
6. Avoid AI When the Researcher Cannot Evaluate Consequential Outputs
A researcher may encounter an AI recommendation that exceeds their methodological expertise.
For example, a student unfamiliar with advanced statistical modeling may receive a technically sophisticated analytical recommendation.
The explanation may appear convincing, but the student may lack the knowledge required to evaluate its assumptions.
In such circumstances, AI can still be used for preliminary learning if permitted.
However, the researcher should not implement consequential recommendations without appropriate independent verification or qualified assistance.
The important boundary concerns using an unverified recommendation to make a scientific decision, not necessarily consulting AI for an explanation.
7. Avoid AI When Its Output Cannot Be Adequately Verified
Some AI-generated claims can be checked against original publications, computational results, or authoritative records.
Others may concern information for which reliable verification is unavailable.
For example, an AI system may confidently describe the motivations of research participants without sufficient supporting evidence.
It may also claim that no previous study has investigated a particular topic without conducting an appropriately scoped literature search.
Researchers should avoid treating such outputs as established findings when independent verification is not possible.
AI may still help generate questions or hypotheses, but these should be distinguished from verified conclusions.
8. Avoid AI When It Conflicts With the Research Methodology
The suitability of AI assistance may depend on a study's methodological and epistemological commitments.
For example, reflexive thematic analysis emphasizes the researcher's active role in developing interpretations through engagement with the data.
Braun and Clarke (2022) discuss the importance of methodological coherence in thematic analysis.
Automatically accepting generated themes without substantive researcher interpretation may be inconsistent with the intended analytical approach.
Other qualitative methods may permit more structured automated classification under appropriate conditions.
The question is not whether AI can produce codes or themes, but whether its role is compatible with the selected methodology.
Researchers should avoid forms of delegation that undermine the intellectual commitments of their research design.
9. Avoid AI When Its Use Creates Unmanageable Reproducibility Problems
AI-assisted research workflows may be affected by model updates, nondeterministic outputs, unavailable configurations, and changes in external services.
These characteristics do not automatically make AI-assisted research irreproducible.
However, researchers should determine whether consequential procedures and outputs can be adequately documented and reconstructed.
For example, using AI to generate code that is subsequently tested, preserved, and executed in a documented environment may support reproducible analysis.
By contrast, relying on an inaccessible sequence of generated interpretations without preserving the underlying evidence or decision process may create serious reproducibility concerns.
Where adequate records cannot be maintained, researchers should reconsider the proposed AI workflow.
10. Avoid AI When It Adds Complexity Without Meaningful Benefit
AI is not necessarily the most appropriate tool for every research task.
A simple spreadsheet formula, database query, established statistical function, or direct examination of an original document may be more reliable and transparent.
For example, asking a language model to calculate a straightforward descriptive statistic may introduce unnecessary uncertainty when the calculation can be performed directly using verified software.
Researchers should evaluate whether AI offers a meaningful advantage relative to available alternatives.
The decision to avoid AI can reflect methodological efficiency rather than technological resistance.
How Should Researchers Evaluate Whether AI Is Appropriate?
A practical decision framework can examine the task, authorization, potential consequences, verification requirements, and availability of alternatives.
| Decision question |
Reason for concern |
Possible response |
| Is the proposed use authorized? |
Institutional, ethics, or contractual restrictions |
Do not proceed without appropriate authorization |
| Will sensitive information be processed? |
Confidentiality and data-protection risks |
Use an approved environment or avoid the upload |
| Could an error cause substantial harm? |
Participant safety or consequential scientific decisions |
Require enhanced safeguards or avoid AI |
| Can the output be independently verified? |
Unsupported or inaccurate recommendations |
Verify before use or withhold the claim |
| Does AI replace a learning objective? |
Compromised assessment of independent competence |
Restrict AI during the relevant assessment |
| Is AI methodologically appropriate? |
Conflict with research design or epistemology |
Modify or avoid the proposed assistance |
| Is a simpler alternative available? |
Unnecessary complexity or reduced transparency |
Use the more appropriate method or tool |
The framework is intended to support judgment rather than produce an automatic permission decision.
Formal institutional requirements and research ethics conditions remain authoritative.
Why Researchers Should Learn to Distinguish Risk From Prohibition
A risky activity is not necessarily prohibited, and a permitted activity is not necessarily risk-free.
For example, AI-assisted code generation may be permitted while still requiring testing and methodological review.
Conversely, uploading confidential participant information may be prohibited under a particular research protocol even when the researcher believes the technical risk is small.
Training should distinguish formal restrictions from discretionary judgments about acceptable risk.
This prevents researchers from treating personal confidence as sufficient authorization.
How Can Non-Use Decisions Be Taught Through Case-Based Learning?
Case-based instruction allows researchers to examine realistic situations involving competing considerations.
For example, trainees might evaluate whether AI should be used to classify sensitive interview data, recommend participant exclusions, or generate code for a high-stakes analysis.
They should identify the relevant obligations, potential benefits, risks, and available alternatives.
Some scenarios should have clear restrictions, while others should involve genuine uncertainty.
The objective is to develop defensible reasoning rather than memorization of a fixed list of forbidden tasks.
Why Training Should Include Cases Where AI Is Appropriate
A curriculum consisting entirely of examples where AI should be avoided may encourage indiscriminate rejection.
Researchers should also examine situations where AI provides legitimate assistance under appropriate safeguards.
For example, using AI to explain publicly available methodological terminology may be reasonable when the explanation is checked before consequential use.
Similarly, generating preliminary code from synthetic data may be appropriate when the implementation is tested and reviewed.
Training should develop calibrated judgment rather than an assumption that non-use is always preferable.
How Can Researchers Demonstrate Competence in Non-Use Decisions?
Assessment should examine whether researchers can explain why AI is appropriate, conditionally appropriate, or inappropriate for a particular task.
Participants may be asked to identify applicable requirements, evaluate risks, propose safeguards, and justify alternatives.
A strong response should recognize uncertainty where the available information is insufficient.
For example, a trainee should not invent a university prohibition when no actual policy has been supplied.
Instead, the trainee should identify the need to consult the responsible authority.
This approach evaluates research judgment rather than confidence in a particular answer.
