03 · What You Need to Know
How AI Can Increase Research Complexity Without Strengthening the Evidence
Complexity Is Not the Same as Methodological Sophistication
Research complexity refers to the number and interaction of decisions, procedures, dependencies, assumptions, and sources of uncertainty involved in conducting a study.
Methodological sophistication refers to the appropriateness and technical development of methods used to address a scientific question.
A sophisticated method may be necessary when the research problem demands it. For example, a multilevel model may be appropriate when observations are nested within institutions and the analysis must account for that structure.
By contrast, using a generative AI system to perform a predefined arithmetic operation may increase workflow complexity without improving the underlying calculation.
The important distinction is whether complexity serves an identifiable scientific purpose.
AI Introduces Additional Decisions That Researchers Must Justify
Incorporating generative AI into research may require choices about the provider, model, prompt wording, input preparation, generation settings, output selection, validation, and human intervention.
Each decision can influence the resulting output.
For example, an AI-assisted qualitative classification procedure may produce different categories depending on whether the prompt requests inductive coding, applies a predefined framework, or supplies examples of expected labels.
These decisions may be methodologically legitimate. However, they require explanation when they materially affect the analysis.
If the research task can be performed adequately without introducing these additional choices, researchers should consider whether the generative component is necessary.
Why Additional Components Create More Opportunities for Error
A research workflow may contain data preparation, analysis, interpretation, and reporting stages. Adding AI introduces another operation whose behavior must be understood and evaluated.
Errors may arise from incorrectly formatted inputs, misunderstood instructions, unsupported generated claims, inappropriate code, or inconsistent output interpretation.
These possibilities do not establish that AI-assisted workflows are always less reliable. A carefully validated AI system may improve performance in some applications.
Nevertheless, every consequential component requires appropriate quality assurance.
Where AI provides no compensating benefit, the additional opportunities for error become difficult to justify.
When AI Creates an Unnecessary Verification Burden
Verification is necessary whenever research outputs require independent support, but AI can introduce additional checking that a simpler procedure would not require to the same extent.
Consider a researcher who asks a chatbot to calculate the median of 500 observations. The resulting number must be checked against the dataset using a reliable calculation.
If the researcher already has access to statistical software capable of calculating the median directly, the AI-generated answer may add a redundant step.
Similarly, AI-generated bibliographic references require verification of their existence and accuracy. A reference-management system importing metadata from an authoritative source may offer a more direct procedure for the same task.
The relevant comparison concerns the complete validated workflow rather than the speed of the initial output.
How Does AI Affect Research Reproducibility?
Generative AI systems may produce variable outputs even when researchers submit identical prompts. Model updates, generation settings, and computational conditions can further affect results.
This variability can complicate reproducibility when AI outputs function as research measurements or analytical evidence.
For example, a study using AI to assign sentiment categories to student reflections may need to preserve the model identifier, prompts, classifications, and validation records.
A deterministic rule-based procedure may be easier to reproduce when the classification criteria are explicit and sufficiently simple.
However, deterministic procedures are not automatically valid. Their rules must still represent the intended construct appropriately.
Researchers should evaluate whether AI introduces reproducibility problems that are disproportionate to its contribution.
When AI Makes Methodological Reporting More Difficult
Research reporting should provide enough information for readers to understand how evidence was generated and analyzed.
When AI materially influences the procedure, researchers may need to document model identification, prompts, input preparation, output handling, validation, and human decisions.
This documentation can be essential for transparency, but it also increases reporting requirements.
If AI is used to perform a task already supported by a straightforward, documented procedure, the additional reporting burden may offer little scientific benefit.
By contrast, when AI provides a necessary analytical capability, detailed reporting is part of making that capability scientifically defensible.
Why Black-Box Dependencies May Increase Complexity
Some AI services provide limited information about their internal operation, model changes, or deployment environment.
Researchers may therefore depend on a provider whose system cannot be fully inspected or preserved.
Such dependencies can complicate long-term maintenance, reproducibility, and interpretation of unexpected results.
For example, a model update may alter classifications midway through a longitudinal study. The research team may then need additional analyses to determine whether observed changes reflect participant responses or changes in the classification system.
A fixed, documented alternative may avoid some of these difficulties.
However, proprietary or black-box systems are not inherently unsuitable. Their acceptability depends on whether the intended use can be adequately validated and documented.
How Can Researchers Compare Complexity Across Workflows?
A useful comparison examines both the scientific benefits and the additional obligations introduced by each procedure.
| Dimension |
Questions to examine |
Possible evidence |
| Methodological suitability |
Does the method address the research question appropriately? |
Study design, assumptions, methodological literature |
| Accuracy |
Does AI improve or reduce correctness? |
Reference data, error analysis, independent calculations |
| Reproducibility |
Can the procedure be repeated or adequately reconstructed? |
Version records, scripts, prompts, archived outputs |
| Verification burden |
How much checking is needed? |
Review time, error rates, validation records |
| Data protection |
Does the workflow introduce additional information-handling risks? |
Provider terms, institutional authorization, security assessment |
| Maintenance |
What happens when tools or models change? |
Version dependencies, update procedures, monitoring requirements |
| Scientific contribution |
Does AI improve the evidence or answer a question otherwise difficult to address? |
Comparative validation, research objectives, output quality |
The comparison should not be reduced to counting software components. A workflow with several well-defined stages may be more defensible than a shorter process whose assumptions and outputs cannot be evaluated.
When Is Additional AI Complexity Scientifically Justified?
Complexity may be justified when AI enables a capability that is necessary or demonstrably useful for the research objective.
For example, a validated language model may help classify large volumes of unstructured text when explicit rules cannot adequately capture the relevant meanings.
A specialized machine-learning system may identify patterns that simpler models fail to represent, provided the performance advantage is established through suitable evaluation.
AI may also support accessibility or reduce substantial manual workload while maintaining acceptable accuracy and oversight.
The justification should identify the benefit and explain why the additional complexity is proportionate to it.
Using AI merely because it is available is not a methodological rationale.
When Does Complexity Conceal Weak Methodological Understanding?
AI-generated code and explanations can make advanced procedures appear accessible before the researcher understands their assumptions.
A researcher may produce a sophisticated model without being able to explain its specification, estimation, or interpretation.
The resulting complexity may obscure rather than resolve weaknesses in the research design.
For instance, adding machine-learning predictions to an observational study does not automatically strengthen causal inference. Predictive accuracy and causal identification address different scientific questions.
Researchers should be able to explain why each consequential analytical component is necessary and what evidence supports its use.
If a procedure cannot be defended without appealing to the authority of the AI system, additional methodological learning or consultation is needed.
Why a Hybrid Workflow May Be Better Than Full AI Automation
Researchers do not necessarily need to choose between using AI for every stage and avoiding it completely.
A hybrid workflow can allocate tasks according to the strengths and limitations of different tools.
For example, AI might help generate an initial script while established statistical software performs the calculation. The researcher then inspects the code and verifies the results.
Alternatively, AI might suggest preliminary qualitative categories while researchers conduct the final interpretation using an explicit coding procedure.
These arrangements can be useful when each component has a clear purpose and the additional steps improve the workflow.
However, hybrid designs should also be evaluated for unnecessary duplication or verification costs.
When Should Researchers Remove AI From an Existing Workflow?
AI should be reconsidered when it repeatedly introduces errors, creates unstable outputs, increases documentation requirements without benefit, or makes the procedure harder to defend.
Researchers may also discover that an established tool can perform the task more efficiently or transparently.
Removing AI does not necessarily mean abandoning the scientific objective. It may simply mean selecting a better-suited implementation.
For ongoing projects, changes should be documented, particularly when earlier AI outputs have already influenced data processing or analysis.
If AI-generated results are replaced, researchers should preserve an appropriate audit trail and ensure that the final reported findings are based on the validated procedure.
Can Research Quality Be Measured by Workflow Complexity?
No. The number of analytical stages, software packages, or AI models used does not establish research quality.
Research quality depends on the appropriateness of the design, the credibility of the evidence, the validity of the analysis, and the transparency of the resulting claims.
A simple procedure may be scientifically rigorous when it adequately answers the question. A complex procedure may also be necessary when the phenomenon or data structure demands it.
The relevant issue is whether the complexity is justified, not whether the workflow appears technologically advanced.
