03 · What You Need to Know
How AI-Assisted Coding Affects Research Methods and Disclosure Requirements
What Does Research Data Coding Mean?
Research data coding involves assigning labels, categories, values, or other representations to observations according to a defined analytical purpose.
In qualitative research, coding may involve identifying concepts, meanings, experiences, or patterns in interviews, documents, and open-ended responses.
In quantitative research, coding may involve transforming responses into numerical variables, classifying observations, or preparing categories for statistical analysis.
These activities differ in their assumptions and analytical purposes.
For example, assigning a numerical value to a fixed-response questionnaire item is not the same as interpreting the meaning of a participant's narrative.
AI systems may assist with either task, but the methodological implications and verification requirements are not necessarily identical.
What Counts as AI-Assisted Data Coding?
AI-assisted coding occurs when an artificial intelligence system contributes to assigning, recommending, organizing, or interpreting codes applied to research data.
Examples include using a language model to classify open-ended survey responses, applying a machine-learning classifier to documents, or generating preliminary qualitative codes from interview transcripts.
Some systems assign labels according to a predefined coding framework. Others suggest new categories based on patterns identified in the data.
The distinction matters because applying established categories and generating an interpretive framework involve different kinds of analytical judgment.
Researchers should identify the actual function performed rather than describing every AI-assisted activity simply as automated coding.
Why Does AI-Assisted Coding Require Methodological Transparency?
Coding decisions can influence the findings produced by a study.
If an AI system incorrectly classifies participant responses, the resulting frequency distributions or thematic patterns may be misleading.
Similarly, an AI-generated coding framework may emphasize some concepts while overlooking others.
Readers need sufficient information to understand how categories were developed, how observations were assigned to them, and what role human researchers played.
A generic statement that AI was used during analysis may not explain these procedures adequately.
Methodological transparency therefore concerns the actual coding process, not merely the identity of the software.
What Do ICMJE and COPE Say About AI-Assisted Data Coding?
The International Committee of Medical Journal Editors (ICMJE) recommends disclosure of AI-assisted technologies used in submitted research and identifies methodological AI use as an activity to be reported in the methods section.
The Committee on Publication Ethics (COPE) similarly emphasizes transparency about AI tools used in research activities, including data collection and analysis.
Neither organization provides a universal coding protocol applicable to every qualitative or quantitative study.
However, their guidance supports the principle that consequential AI-assisted analysis should be described accurately and that human authors remain responsible for the results.
Researchers should also follow discipline-specific methodological standards and the journal's current AI-use policy.
Is AI Coding the Same as AI-Assisted Qualitative Analysis?
Not necessarily. Coding may be one component of qualitative analysis, but qualitative interpretation often extends beyond assigning labels.
For example, a researcher may use AI to classify responses according to a predefined set of categories without asking the system to develop themes or interpret participant experiences.
Another researcher may use AI to generate themes, explain relationships among codes, and draft interpretive findings.
The second activity involves a broader analytical role.
Researchers should distinguish coding assistance from interpretation because the latter may introduce additional concerns about contextual meaning, theoretical assumptions, and unsupported inference.
The requirements for disclosing AI-assisted qualitative analysis may therefore extend beyond the documentation needed for coding alone.
What Is the Difference Between Deductive and Inductive AI Coding?
Deductive coding applies categories established before or independently of the analysis. Inductive coding develops categories through engagement with the data.
AI systems may assist with both approaches, but the role they perform differs.
AI-assisted deductive coding
Using AI to assign observations to a predefined coding framework developed and justified through the research design.
AI-assisted inductive coding
Using AI to propose or develop categories from the data, potentially influencing the conceptual structure of the analysis.
In deductive coding, verification may focus on whether the system applied the established definitions consistently and accurately.
In inductive coding, researchers may also need to evaluate whether the suggested categories adequately represent the data and fit the study's analytical approach.
These distinctions should be reflected in methodological reporting when they are consequential to the findings.
Does Human Review Remove the Need to Disclose AI Coding?
No, not automatically.
Human review may improve the reliability or credibility of AI-assisted coding, but it does not mean the AI system played no role.
For example, a researcher may ask AI to assign preliminary codes and then inspect every coded response before finalizing the dataset.
The final decisions may be human-approved, yet the AI still contributed to the analytical workflow.
Where the journal requires disclosure of methodological AI use, that assistance should be reported.
The statement should explain the human verification accurately rather than suggesting that all coding was performed independently by humans.
What Does Human Verification of AI Coding Require?
The appropriate verification procedure depends on the coding task, research design, and consequences of classification errors.
For a predefined classification task, researchers may compare AI-assigned labels with a human-coded reference set.
They may examine disagreement patterns, assess category-specific errors, and determine whether the system performs adequately for the intended purpose.
For interpretive qualitative coding, verification may involve returning to original excerpts, examining contextual meaning, considering contradictory cases, and documenting decisions to revise or reject AI suggestions.
A single numerical agreement statistic may not capture all aspects of qualitative credibility.
Researchers should therefore avoid applying quantitative classification metrics mechanically to every qualitative methodology.
Verification must be appropriate to the epistemological and analytical assumptions of the study.
Should Researchers Calculate Inter-Rater Agreement for AI Coding?
Inter-rater agreement may be useful in some coding designs, but it is not universally required or methodologically appropriate.
For studies using predefined categories and reliability-oriented coding, agreement measures such as Cohen's kappa or Krippendorff's alpha may help evaluate consistency.
However, the choice of statistic depends on the number of coders, coding structure, measurement level, and analytical assumptions.
In interpretive qualitative approaches, coding differences may be treated as opportunities for analytical reflection rather than errors requiring numerical reconciliation.
Researchers should not assume that calculating agreement between AI and a human coder automatically validates the resulting interpretation.
Nor should they describe AI-human agreement as inter-rater reliability without explaining how the AI system functioned and what the measure represents.
What If AI Codes Every Response but Humans Review Only a Sample?
Sampling-based verification may be appropriate in some research designs, but its adequacy must be justified.
For example, a researcher may manually code a representative subset of responses and compare the labels with AI classifications.
The resulting evaluation may provide information about estimated classification performance.
However, a small or unrepresentative sample may fail to reveal systematic errors affecting particular categories or participant groups.
Researchers should consider the prevalence of categories, the consequences of misclassification, and whether the validation sample reflects the data being coded.
The methods section should explain the verification procedure accurately.
A researcher who reviewed only 10% of the records should not state that all AI-generated codes were individually verified.
Can AI Coding Introduce Bias Into Research Findings?
Potentially. AI systems may classify responses differently according to language, dialect, writing style, or contextual information.
They may also favor categories that resemble common patterns in their training data or prompts.
For example, a system might interpret students' concerns about unreliable internet access as low motivation, even when the responses primarily describe infrastructural barriers.
Such errors could distort the frequency or meaning of reported categories.
Researchers should examine whether classification errors are concentrated in particular response types or groups.
However, bias should not be assumed solely because AI was used. Its presence and significance require evaluation in the specific research context.
Does AI Coding of Numerical Data Require Disclosure?
It depends on what the AI system actually did.
Routine numerical recoding according to fixed rules may resemble conventional data-processing operations, although journal policies may still require disclosure of particular AI tools.
For example, converting a five-point Likert response into a predetermined numerical code is different from asking a generative AI system to infer categories from ambiguous textual responses.
When AI makes substantive classification decisions that influence the analysis, methodological transparency becomes more important.
Researchers should distinguish deterministic transformations from model-based judgments and report the procedure according to the applicable policy.
What About AI Coding of Interview Transcripts?
Interview transcripts often contain contextual information that is important to qualitative interpretation.
An AI system may assign a code based on an isolated phrase while overlooking the meaning established by the surrounding conversation.
For example, a participant who says they rarely use an online platform may be describing poor internet access rather than a lack of interest in learning.
Researchers should evaluate whether the coding procedure preserves the relevant context and reflects the chosen qualitative methodology.
Where AI assists with coding, the methods section should explain the role of the system and how the researchers assessed its suggestions.
The disclosure should not imply that automated classification alone establishes a valid qualitative interpretation.
Does AI Coding of Multilingual Data Require Additional Care?
Yes. Coding multilingual responses may involve linguistic and cultural distinctions that are not adequately represented by automated classifications.
For example, Filipino expressions describing interpersonal relationships, obligation, or social accommodation may not map neatly onto English-language categories.
An AI system may assign a label based on an approximate translation rather than the participant's intended meaning.
Researchers should consider language-specific validation, contextual review, and the role of translation in the coding process.
If AI translation was used before coding, both procedures may need methodological documentation.
The relationship between AI-assisted translation and research disclosure is especially important when translated data become the basis for qualitative findings.
What Privacy Risks Arise When AI Codes Research Data?
AI-assisted coding may involve uploading interview transcripts, survey responses, documents, or other research materials to external services.
These materials may contain identifiable or confidential information.
Researchers should determine whether the intended service is authorized under the study's ethics approval, participant consent, institutional rules, and applicable data-protection requirements.
De-identification may reduce risks but does not necessarily eliminate them, particularly when detailed narratives contain indirectly identifying information.
Researchers should also examine relevant data retention, access, and processing arrangements.
Disclosure in the published manuscript does not retrospectively authorize an impermissible data transfer.
What Information Should an AI Coding Disclosure Include?
The appropriate reporting detail depends on the significance of the AI assistance and the journal's requirements.
| Reporting Element |
Purpose |
| AI tool or model |
Identifies the system used for coding. |
| Data coded |
Explains which responses, documents, or observations were processed. |
| Coding framework |
Clarifies whether categories were predefined or generated during analysis. |
| AI task |
Describes whether the system suggested, assigned, or revised codes. |
| Human verification |
Explains how researchers checked or modified the outputs. |
| Validation results |
Provides relevant evidence of classification performance or analytical credibility. |
| Limitations |
Identifies uncertainties or constraints affecting interpretation. |
Additional details, such as model versions, prompt templates, or coding settings, may be relevant when required for reproducibility.
The statement should reflect the actual procedure rather than provide a generic assurance that human oversight occurred.
Where Should AI-Assisted Coding Be Disclosed?
When AI contributes to research data coding, the methods section is generally the appropriate location for describing the procedure.
A journal may also require a separate AI-use declaration.
For example, the methods section may explain the coding framework, AI classification procedure, human verification, and final analytical treatment.
The designated AI statement may identify the tool and summarize its role.
These reporting mechanisms should be consistent and should not conceal methodological details behind a brief acknowledgment.
What Should an AI Coding Disclosure Statement Say?
A useful statement should describe what the AI system did and what the researchers actually verified.
Illustrative Methods Disclosure
AI-Assisted Coding of Open-Ended Survey Responses
"An AI system was used to assign preliminary categories to open-ended survey responses according to a predefined coding framework. Two researchers independently reviewed the AI-generated classifications against the original responses and resolved discrepancies through discussion. Final coding decisions were made by the research team."
This example is appropriate only if the stated procedure actually occurred.
Researchers should identify the specific tool and provide additional details required by the journal.
If only a subset of responses was reviewed, the statement should describe that sampling procedure rather than claiming comprehensive verification.
Does Disclosure Make AI Coding Methodologically Valid?
No. Disclosure and methodological validity are separate questions.
A researcher may fully disclose the use of an AI coding tool while applying an inappropriate coding framework or failing to identify systematic classification errors.
Conversely, carefully validated AI-assisted coding may still require disclosure under the journal's policy.
Researchers must establish that the coding procedure is appropriate for the research question and analytical approach.
Transparency enables readers to evaluate the method, but it does not substitute for methodological justification.
Should Researchers Preserve AI Coding Records?
Maintaining relevant records can support transparency, reproducibility, and later verification.
Useful materials may include the coding framework, prompts or instructions, AI-generated labels, human revisions, validation results, and final coding matrices.
The appropriate level of retention depends on institutional requirements, research risks, and the role of AI in producing the findings.
Researchers should also consider confidentiality and data-retention obligations when preserving prompts or outputs containing participant information.
The broader question of maintaining records of AI use in research becomes particularly relevant when AI-assisted coding materially influences published results.
What If the Journal Does Not Have a Specific AI Coding Policy?
Researchers should review the journal's general AI-use policy and ordinary methodological reporting requirements.
Even without a dedicated AI-coding rule, readers may need to understand how the coding procedure was performed.
If uncertainty remains about the disclosure requirement, the editorial office may be consulted.
The researcher should describe the actual coding activity rather than ask only whether AI tools are permitted.
Policy silence does not eliminate the need for an accurate and sufficiently detailed methods section.
