03 · What You Need to Know
How Research Design Changes When AI Behavior Is the Phenomenon of Interest
What does it mean for AI to be the object of research?
AI becomes the object of research when the study seeks to investigate characteristics, capabilities, limitations, or behavior of an artificial intelligence system.
For example, researchers may examine whether a language model generates accurate citations, produces biased classifications, follows instructions, or responds consistently to repeated prompts.
In these studies, AI outputs are not merely tools used to complete another research activity.
They constitute observations relevant to the research question.
A researcher studying citation hallucinations may collect model-generated references and verify whether the cited publications exist.
The observed phenomenon is the AI system's citation-generation behavior under specified conditions.
How does studying AI differ from using AI in research?
Using AI in research means employing an AI system to assist with a task such as literature screening, data processing, coding, or writing.
Studying AI means investigating the system itself.
| Dimension |
Using AI as a research tool |
Studying AI as the research subject |
| Primary research interest |
A phenomenon outside the AI system |
The AI system's behavior, performance, or characteristics |
| Role of AI outputs |
Intermediate assistance or analytical material |
Primary observations or experimental outcomes |
| Model configuration |
Important for documenting consequential assistance |
Often central to the research design |
| Prompt design |
May support task completion |
May function as an experimental condition |
| Repeated testing |
Useful for checking reliability |
May be necessary to estimate response variability |
| Evaluation |
Verifies the research task was completed appropriately |
Measures defined properties of AI behavior |
| Generalization |
Concerns the substantive research question |
Concerns tested models, tasks, settings, or model families |
The distinction matters because a researcher cannot assume that methods suitable for using an AI tool are sufficient for evaluating the tool scientifically.
What becomes the unit of analysis in AI research?
The unit of analysis depends on the research question and experimental design.
Researchers may analyze individual model responses, prompts, tasks, conversation sessions, model versions, or combinations of these elements.
For example, a study investigating citation hallucinations might treat each generated citation as an observation.
However, several citations generated within one response may share dependencies.
Likewise, repeated responses to the same prompt may not constitute independent observations in the statistical sense.
Researchers should identify the unit of analysis explicitly and account for relevant clustering or repeated measurements.
Failure to do so may produce misleading estimates of uncertainty or exaggerate the precision of performance comparisons.
Why does model identification become methodologically important?
Different AI models may behave differently even when given identical tasks.
Models can differ in training, architecture, alignment procedures, available tools, and other implementation characteristics.
Consequently, findings obtained from one model should not automatically be attributed to another.
Researchers should document the provider, model identifier, and available version information.
Where a model version cannot be determined precisely, that limitation should be reported.
The question of recording the exact AI model version is particularly important for studies intended to support reproducibility or longitudinal comparison.
Why do testing dates matter?
Commercial AI systems may change through model updates, modified system instructions, revised safety mechanisms, or changes in supporting infrastructure.
Such changes may occur without researchers receiving complete technical information.
For example, a language model that produces a particular error in October may behave differently when the same prompt is tested several months later.
Researchers should therefore document when observations were collected.
For longitudinal studies, testing dates may become part of the analytical design.
However, recording the date alone does not guarantee that the underlying model state can be reconstructed.
Testing time provides important provenance even when exact computational reproducibility is unavailable.
How does prompt design become part of the research method?
Prompts determine what information and instructions the model receives.
In studies of AI behavior, prompt wording may influence the observed outcome.
For example, asking a model to explain a statistical concept may produce different responses from asking it to provide a concise answer suitable for a novice student.
These differences may be relevant to the research question.
Researchers should therefore treat consequential prompts as methodological materials rather than informal conversational instructions.
Prompt construction, selection, piloting, and revision should be documented where they affect the findings.
In experimental studies, prompt variants may function as independent variables.
Should prompts be standardized?
Standardization is often useful when researchers want to compare model responses under controlled conditions.
For example, two models may receive the same set of research methodology questions to examine differences in accuracy.
However, identical wording does not always create functionally equivalent conditions across models.
Different systems may have different instruction hierarchies, tool access, context limits, or formatting conventions.
Researchers should therefore explain whether the study prioritizes identical inputs, comparable task conditions, or optimized performance under each model's intended use.
The appropriate choice depends on the scientific objective.
Standardization should support fair comparison rather than become an assumption that all AI systems interpret prompts identically.
Why can conversation history affect the results?
Many generative AI systems respond to the current prompt within a broader conversational context.
Previous messages may influence terminology, assumptions, response style, and substantive content.
For example, a model that has already discussed a particular statistical method may answer a subsequent question differently from a model receiving that question in a fresh conversation.
Researchers should therefore decide whether conversation history is part of the experimental condition.
If the objective is to examine independent responses, fresh sessions may be appropriate.
If the objective concerns extended dialogue, the preceding conversation should be documented and controlled as far as feasible.
The choice should follow the research question rather than convenience.
Why is repeated testing important?
Generative AI systems may produce different outputs when given the same prompt repeatedly.
Variation can arise from sampling procedures, computational implementation, contextual differences, or changes in the deployed system.
For example, a model may answer a factual question correctly in some runs but generate an incorrect citation in others.
Evaluating only one response may therefore provide an incomplete picture of performance.
Researchers may need repeated trials to estimate variability or the frequency of particular outcomes.
However, the appropriate number of repetitions depends on the task, expected variability, statistical precision, and available resources.
There is no universal repetition count suitable for every AI study.
Does setting temperature to zero guarantee reproducibility?
No. A low or zero temperature setting may reduce sampling variability in systems that expose this parameter, but it does not universally guarantee identical outputs.
Other sources of nondeterminism may remain, including computational implementation, backend changes, and model updates.
Some commercial interfaces do not expose generation settings at all.
Researchers should document available parameters and evaluate actual response stability rather than assume deterministic behavior.
The distinction between controlling generation settings and establishing reproducibility is important when interpreting repeated AI tests.
How should researchers select AI evaluation tasks?
Tasks should represent the capability or behavior the study intends to evaluate.
For example, a study of AI assistance in academic research may include citation verification, methodological explanation, statistical interpretation, or research ethics scenarios.
However, selecting only easy questions may inflate performance estimates.
Conversely, selecting unusually adversarial or ambiguous tasks may produce results that do not represent ordinary use.
Researchers should define the target task domain and justify how evaluation items were selected.
Where possible, tasks should be sampled systematically or drawn from appropriate benchmarks.
Researchers should also distinguish exploratory test development from the final evaluation set.
What is the reference standard in AI evaluation?
A reference standard is the criterion against which model outputs are evaluated.
For factual questions, it may involve independently verified answers supported by authoritative sources.
For classification tasks, it may involve carefully developed reference labels.
For open-ended responses, evaluation may require a rubric and qualified human judgment.
However, some research tasks do not have a single uncontroversial correct answer.
For example, evaluating an AI-generated qualitative interpretation may require attention to methodological orientation and interpretive defensibility rather than simple exact-match scoring.
Researchers should explain how the reference standard was established and why it is appropriate.
Can researchers use another AI model as the evaluator?
Potentially, but doing so introduces additional methodological questions.
An AI evaluator may help score large numbers of responses, classify errors, or apply a structured rubric.
However, the evaluator may exhibit its own biases, inconsistencies, or preferences for particular response styles.
It may also share training patterns with the model being evaluated.
Researchers should therefore validate AI-based evaluation against appropriate independent reference judgments where feasible.
Agreement between two AI systems does not automatically establish correctness.
The evaluation method itself should be treated as a component requiring scrutiny.
How should researchers evaluate partially correct AI responses?
Binary correct-or-incorrect scoring may be insufficient for complex tasks.
For example, an AI response explaining a statistical method may correctly describe its purpose while misstating an important assumption.
Classifying the entire answer as correct could conceal a consequential error.
Classifying it as entirely incorrect may obscure the portions that were accurate.
Researchers may use multidimensional rubrics that distinguish factual correctness, completeness, relevance, and other task-specific properties.
However, rubric criteria should be defined clearly and applied consistently.
Where human judgment is involved, researchers should address evaluator training, disagreement, and uncertainty appropriately.
How should hallucinations be evaluated?
Researchers should define the type of unsupported or incorrect content relevant to the study.
For example, a citation hallucination may involve a reference that does not correspond to a real publication.
Other hallucinations may involve unsupported factual claims, invented quotations, or incorrect descriptions of evidence.
The evaluation should distinguish these error types where their consequences differ.
Researchers should also specify the denominator used to calculate error rates.
A hallucination rate per response is not necessarily equivalent to a hallucination rate per factual claim or generated citation.
Clear operational definitions are necessary for meaningful comparisons across studies.
How does research validity change when studying AI?
Validity remains central, but its application depends on the AI-related research question.
Construct validity concerns whether the evaluation measures the intended AI capability or behavior.
Internal validity concerns whether observed differences can reasonably be attributed to the experimental conditions.
External validity concerns whether findings generalize to other tasks, models, settings, or periods.
For example, a model's performance on multiple-choice research methodology questions may not establish its ability to conduct rigorous methodological reasoning in an open-ended research setting.
Researchers should therefore avoid equating performance on a narrow benchmark with competence across an entire research domain.
What is benchmark contamination?
Benchmark contamination occurs when evaluation material has influenced model development or is otherwise available to the system in ways that compromise the intended independence of testing.
For example, a model may have encountered publicly available benchmark questions during training.
Strong performance on those questions may then partly reflect prior exposure rather than generalization to unfamiliar tasks.
However, the presence of a benchmark online does not by itself prove that a particular model trained on it.
Researchers should assess plausible contamination risks and avoid unsupported claims about training-data access.
Independent or newly constructed evaluation items may help reduce some risks, although they introduce their own validation requirements.
How should researchers compare different AI models?
Comparative studies should establish conditions that are fair relative to the intended research question.
For example, models may differ in their access to retrieval tools, context windows, or specialized capabilities.
Researchers should decide whether these features are part of the system being evaluated or variables that should be controlled.
Testing one model with external retrieval while denying it to another may confound model performance with tool availability.
However, disabling a model's normal capabilities may also create an artificial comparison.
Researchers should explain the comparison framework and interpret results accordingly.
The objective is not necessarily identical treatment in every technical detail, but a defensible comparison of the intended systems or capabilities.
Should human performance be used as a comparison?
Potentially, when the research question concerns AI performance relative to human researchers or professionals.
However, the human comparator should be appropriate for the task.
For example, comparing a language model with first-year students may answer a different question from comparing it with experienced researchers.
Time limits, access to reference materials, task instructions, and evaluation procedures may also influence the results.
Researchers should avoid describing AI as superior to humans without specifying the population, task, conditions, and performance measure.
Average accuracy alone may not capture differences in error severity or the need for human oversight.
Can AI research findings become outdated?
Yes. Rapid model development can affect the continued applicability of findings.
A study may accurately describe a model's performance during data collection while becoming less representative of its later deployed behavior.
However, this does not necessarily make the original research invalid.
The findings remain evidence about the tested system under the documented conditions.
Researchers should distinguish historical validity from current applicability.
Longitudinal testing or replication across model versions may be appropriate when the research question concerns changing capabilities.
How does reproducibility differ in AI research?
Reproducibility may be difficult when researchers cannot access the exact model version or deployment environment used in the original study.
Commercial providers may update systems, modify internal instructions, or discontinue older models.
Even when the model remains available, nondeterministic outputs may prevent exact response reproduction.
Researchers should therefore distinguish reproducing the procedure, reproducing the distribution of outcomes, and obtaining identical outputs.
Preserving prompts, response records, model identifiers, settings, and testing dates can support independent examination.
However, documentation cannot always reconstruct a proprietary system's historical state.
The limitations should be reported rather than concealed.
Should researchers archive every AI response?
The appropriate preservation strategy depends on the research design, data sensitivity, and practical constraints.
For many AI evaluation studies, preserving the complete set of tested prompts and generated outputs is valuable for auditability.
It allows others to examine scoring decisions, identify overlooked errors, and assess whether the published examples represent the full dataset.
However, responses may contain confidential information or material subject to access restrictions.
Researchers should apply appropriate storage, redaction, and access controls.
Where full public release is not possible, a documented restricted archive or other verification arrangement may be appropriate.
How should researchers handle unsuccessful prompts?
Unsuccessful prompts may be important research observations rather than inconvenient development artifacts.
For example, a researcher evaluating citation generation should not report only prompts that produced valid references while excluding those that produced fabricated citations.
However, exploratory prompt development should be distinguished from the final evaluation protocol.
Researchers may refine prompts during piloting and then freeze the procedure before formal testing.
If prompt variants are selected based on evaluation outcomes, the selection process should be documented.
Selective reporting can distort performance estimates and create an overly favorable impression of the model.
How does human oversight affect AI evaluation?
Human oversight may be necessary for verifying outputs, interpreting ambiguous responses, and evaluating consequential errors.
For example, a model may generate a statistically correct explanation that is inappropriate for the educational level or context of the intended user.
Such judgments may require domain expertise.
Researchers should explain who evaluated the outputs, what criteria they applied, and how disagreements were addressed.
They should also distinguish independent human evaluation from AI-assisted scoring.
Where evaluation requires substantial human correction, that effort may be relevant to claims about the model's practical usefulness.
What ethical issues arise when AI is the research subject?
Research on AI may involve privacy, intellectual property, platform access conditions, and potential harm from generated content.
For example, prompts containing confidential participant information may expose that information to an external service.
Studies involving human evaluators may also require appropriate ethical review, particularly when participants are exposed to sensitive material or their judgments are collected as research data.
Researchers should distinguish studies conducted entirely on computational outputs from those involving human participants or personal information.
The presence of AI does not automatically remove ethical or legal obligations.
Institutional requirements should be evaluated according to the actual research activities.
Can researchers generalize findings from one AI model to all generative AI?
Generally, not without broader evidence.
A finding obtained from one model may depend on its architecture, training, deployment configuration, or specific prompting conditions.
For example, observing a high citation hallucination rate in one model does not establish that every generative AI system has the same rate.
Researchers should specify the evaluated system and avoid using broad labels that imply unsupported generalization.
Studies involving several models may support wider conclusions, but model selection and representativeness still require justification.
The scope of the claim should reflect the diversity of systems and conditions actually examined.
How should studies of AI systems be reported?
Researchers should document the AI system, research tasks, prompts, experimental conditions, evaluation criteria, and analytical procedures.
Relevant information may include available model versions, testing dates, generation parameters, tool access, conversation history, and repeated-trial procedures.
Where human evaluators are involved, the manuscript should describe their selection, training, and evaluation responsibilities.
Results should report relevant performance variation, uncertainty, errors, and limitations.
Researchers should also clarify whether findings concern a particular deployment, model version, task domain, or broader class of systems.
Transparent reporting helps distinguish reproducible methodological evidence from isolated demonstrations of AI capability.
