03 · What You Need to Know
How to Decide Whether an AI Tool Is Suitable for the Population Being Studied
Known Poor Performance Is a Methodological Warning
Researchers select instruments and analytical procedures based partly on their suitability for the intended population.
For example, a questionnaire validated in one language should not automatically be assumed valid in another.
The same principle applies to AI systems.
If an AI classifier performs poorly on a particular language, population, or institutional context, researchers should investigate whether the limitation affects their study.
Ignoring relevant performance evidence can undermine the validity of the resulting measurements and conclusions.
However, the seriousness of the concern depends on what the tool is asked to do and how its outputs are used.
Performance Must Be Evaluated for the Specific Research Task
An AI system may perform poorly on one task while remaining useful for another.
For example, a language model may struggle to classify culturally specific expressions but perform adequately when formatting non-sensitive bibliographic records.
Similarly, a model with weak multilingual sentiment classification may still help organize a research schedule.
Researchers should distinguish general model limitations from task-specific suitability.
The relevant question is whether the system can perform the intended research function with sufficient reliability under the conditions in which it will be used.
Population-Specific Performance Matters More Than General Reputation
AI tools are often described using aggregate benchmark scores or broad claims about their capabilities.
These results may not adequately represent the population included in a particular study.
For example, a model evaluated primarily on English-language academic material may not demonstrate equivalent performance on Taglish interviews.
Researchers should avoid assuming that strong overall performance establishes reliability for every language or population.
Where possible, they should examine AI performance across relevant population subgroups before relying on the tool.
Researchers Should Distinguish Poor Performance From Insufficient Evidence
There is an important difference between a tool known to perform poorly and a tool whose performance has not been adequately evaluated.
In the first case, evidence indicates a relevant limitation.
In the second, the evidence may be insufficient to establish suitability.
Neither situation justifies an unsupported claim that the tool is reliable.
However, the appropriate response may differ.
Known poor performance may warrant corrective action or replacement, while insufficient evidence may warrant additional validation before use.
External Validation Is Important When Populations Differ
A model developed or evaluated in one population may perform differently in another.
For example, a student-risk prediction model trained using conventional university programs may not generalize adequately to fully asynchronous learners.
Differences in attendance expectations, assessment structures, and platform behavior may affect predictions.
Researchers should evaluate the model using data that represent the intended study population.
External validation can help establish whether performance transfers to a new institution, population, or time period.
However, validation should also examine whether the target outcome and measurement procedures remain appropriate.
Researchers Should Establish Minimum Performance Requirements
Before relying on an AI tool, researchers should identify what level of performance is necessary for the intended task.
For classification, this may involve sensitivity, specificity, or other error measures.
For probabilistic prediction, calibration and discrimination may be relevant.
For translation, the requirement may concern preservation of meaning and avoidance of consequential omissions.
For qualitative coding, the criteria should reflect the analytical methodology rather than assume that every passage has one objectively correct interpretation.
Performance requirements should be justified according to the scientific purpose and consequences of errors.
Known Performance Disparities Can Threaten Research Validity
If an AI system performs poorly for the study population, its outputs may systematically misrepresent participants or observations.
For example, an AI sentiment classifier may overlook dissatisfaction expressed indirectly in Filipino.
If researchers use those labels to estimate satisfaction levels, the resulting statistics may be distorted.
The problem becomes particularly serious when classification errors differ across comparison groups.
Researchers should evaluate whether the known limitation could affect the study's main findings.
The potential for biased AI tools to produce biased research conclusions should inform tool selection.
Consequential Decisions Require Stronger Safeguards
The acceptability of an AI tool depends partly on how its outputs influence decisions.
Using a model to suggest preliminary search terms differs from using it to determine participant eligibility or allocate important opportunities.
Errors in low-stakes exploratory tasks may be relatively easy to identify and correct.
Errors in consequential decisions may create exclusion, unequal treatment, or other harms.
Researchers should therefore apply safeguards proportionate to the potential consequences.
Where the system cannot demonstrate adequate reliability, it should not independently control high-impact decisions.
Human Review May Reduce Risk but Does Not Guarantee Suitability
Researchers may attempt to address poor AI performance through human verification.
Qualified reviewers can identify incorrect labels, restore omitted meanings, and challenge unsupported interpretations.
However, reviewers may also overlook systematic errors or become influenced by plausible AI suggestions.
Human review is particularly limited when reviewers lack the language proficiency or disciplinary expertise needed to evaluate the outputs.
Researchers should consider whether human oversight can adequately address the identified AI limitations.
A statement that every output was reviewed does not independently establish reliability.
Independent Verification Can Be More Important Than Model Selection
In some workflows, the AI tool serves only as a preliminary assistant.
For example, AI-generated qualitative codes may be treated as suggestions that researchers independently reconsider against original transcripts.
In this situation, the final analytical decisions may not depend directly on the model's classifications.
However, AI suggestions can still influence researcher attention and interpretation.
Researchers should design verification procedures that reduce inappropriate reliance on the model and preserve access to original evidence.
The extent of verification should reflect the role of AI and the methodological risks involved.
Restricted Use May Be More Defensible Than Complete Adoption
An AI tool does not necessarily need to be accepted or rejected for every possible research task.
Researchers may restrict its role to activities for which adequate reliability can be demonstrated.
For example, a multilingual AI system that performs poorly on culturally specific interpretation might still assist with non-interpretive document organization.
Similarly, a prediction model may be suitable for exploratory analysis but not for determining individual eligibility for institutional support.
Researchers should define which uses are permitted and which require independent procedures.
Restricted use should be based on evidence rather than convenience alone.
Alternative AI Tools Should Be Evaluated Comparatively
Replacing one AI system with another may improve performance, but the alternative should not be assumed superior without evaluation.
Researchers should compare candidate tools using relevant data, reference criteria, and performance measures.
For example, two multilingual models may differ in their ability to interpret code-switched responses.
A comparative evaluation can help determine which system is more suitable for the intended task.
However, models may share limitations because of similar training resources or evaluation procedures.
Researchers should examine actual task performance rather than relying exclusively on provider claims.
Non-AI Procedures May Be More Appropriate
Researchers should not assume that an AI-based procedure is necessary simply because one is available.
Traditional statistical methods, manual coding, established instruments, or qualified human translation may provide more defensible alternatives for particular tasks.
For example, a small collection of culturally nuanced interview responses may be better examined directly by qualified researchers than processed through a poorly performing automated classifier.
However, manual procedures also have limitations, including inconsistency and human bias.
The appropriate choice depends on comparative evidence, methodological requirements, and available resources.
Retraining and Fine-Tuning May Help but Require Validation
Researchers may consider improving an AI system using additional task-relevant examples.
Fine-tuning or other adaptation procedures may improve performance for the intended population.
However, additional training does not guarantee that measurement problems or conceptual misunderstandings have been corrected.
Researchers should evaluate adapted models using suitable held-out data.
They should also examine whether improvements for one population introduce new errors elsewhere.
Where sensitive research data are involved, training procedures must comply with applicable consent, ethics, and data-governance requirements.
Prompt Engineering Cannot Reliably Correct Every Population-Specific Limitation
Clearer prompts may improve some AI outputs.
For example, providing a detailed codebook may help a generative model classify responses more consistently.
However, prompt revision cannot guarantee that the system understands culturally specific expressions or performs equally across languages.
Researchers should evaluate whether revised instructions actually improve relevant performance.
Repeatedly modifying prompts until outputs appear plausible is not equivalent to independent validation.
Prompt improvements should be tested against appropriate reference evidence.
Researchers Should Consider Whether Poor Performance Is Systematic
An AI tool may make occasional errors without exhibiting a recurring disadvantage for the study population.
Conversely, errors may cluster around particular linguistic, demographic, or contextual characteristics.
Researchers should examine the distribution and nature of errors.
For example, a model may consistently misinterpret indirect criticism or particular code-switching patterns.
Identifying systematic failure modes can help determine whether mitigation is feasible.
The evaluation should distinguish demonstrated patterns from plausible but unverified explanations.
Performance Requirements Should Reflect the Study's Methodology
Different research traditions require different forms of validation.
A structured content analysis using predefined categories may evaluate classification consistency against reference judgments.
A reflexive thematic analysis may emphasize interpretive engagement, analytical transparency, and the relationship between themes and original evidence.
Researchers should not impose one universal numerical accuracy requirement on every AI-assisted task.
Instead, they should identify what constitutes defensible performance within the chosen methodological framework.
Known Limitations May Require Changes to the Research Design
When an AI tool performs inadequately for the study population, researchers may need to revise more than the software choice.
For example, a study relying on automated multilingual sentiment classification may need additional bilingual reviewers or a different analytical strategy.
A prediction study may need revised outcome definitions or additional validation data.
Researchers should examine whether the existing design can support the intended conclusions after the limitation is identified.
Continuing with an unsuitable procedure merely to preserve the original research plan may undermine methodological integrity.
Researchers Should Not Exclude Participants to Accommodate a Weak AI Tool
One possible response to poor model performance is to remove observations that the system cannot process reliably.
However, excluding participants because their language, communication style, or circumstances are poorly represented by the AI tool can introduce selection bias.
For example, removing Taglish responses to simplify automated analysis may systematically exclude relevant participant experiences.
Researchers should consider whether alternative processing or verification procedures can preserve those observations.
Any exclusion criteria should be scientifically justified and transparently reported.
The limitations of a chosen tool should not silently redefine the population the research claims to represent.
Data Privacy and Research Ethics Still Apply
Improving population-specific AI performance may involve collecting additional data or sharing material with external services.
Researchers should consider participant consent, confidentiality, institutional approval, and applicable data-protection requirements.
For example, uploading identifiable interview transcripts to an external AI service may create risks unrelated to model accuracy.
Researchers should not assume that performance improvement justifies unrestricted processing of participant information.
Technical suitability and ethical authorization are separate requirements.
Document the Rationale for Using or Rejecting the Tool
Researchers should record the evidence informing AI tool selection.
Relevant information may include the intended task, model version, population characteristics, validation results, known limitations, alternative procedures, and safeguards.
If the tool is used despite known performance disparities, the justification should explain why the remaining limitations are acceptable for the intended use.
Where reliability cannot be established, the report should describe how AI use was restricted or replaced.
Documentation supports accountability but does not substitute for adequate validation.
Tool Suitability Should Be Reassessed Over Time
AI systems may change through model updates, revised configurations, or altered retrieval resources.
The research population or data collection conditions may also change.
A tool previously found unsuitable may improve, while a previously validated tool may perform differently after an update.
Researchers should evaluate whether existing validation evidence remains applicable.
Reassessment is particularly important when AI outputs influence consequential research decisions.
