Which Data-Cleaning Decisions Must Be Explicitly Specified by the Researcher?

Which Data-Cleaning Decisions Should Never Be Left to AI Guesswork?

You ask an AI assistant to clean a dataset containing 500 student questionnaires. It identifies missing responses, unusual scores, repeated records, and inconsistent academic program labels. The tool offers to correct these problems automatically.

But who decides whether a missing response should be imputed, whether an outlier should be excluded, or whether two categories should be combined? Can the AI make these decisions based on what appears statistically reasonable?

Many cleaning operations are computationally straightforward once their rules have been defined. Establishing those rules, however, can involve assumptions about measurement, sampling, and the evidence that the research intends to preserve.

Researchers Must Specify Decisions That Affect Data Meaning or Inclusion

Which Data-Cleaning Rules Require Explicit Researcher Decisions?

Why Is a Data-Cleaning Decision Different From a Data-Cleaning Operation?

A cleaning decision establishes what should happen to particular observations and why. A cleaning operation implements that decision through software or manual processing.

For example, a researcher may decide that questionnaire responses outside the permitted range of 1–5 must be flagged for source verification. A Python script can then identify every value violating that rule.

The script performs the operation. The researcher establishes the rule and its methodological justification.

This distinction becomes particularly important when generative AI is used because a chatbot may propose both the rule and the code that executes it. Researchers should evaluate these separately.

Which Decisions Are Most Consequential?

Decisions that alter the meaning, availability, or analytical representation of observations generally warrant explicit specification.

The table identifies areas requiring researcher authority, not a universal requirement to approve every individual cell change manually. An approved deterministic rule may be applied to many records, provided its execution is properly verified.

Who Defines What Counts as an Invalid Value?

The researcher should establish validation rules using the instrument, codebook, collection protocol, and relevant measurement definitions.

For example, an academic self-efficacy item may permit integers from 1 to 5. A value of 8 violates that coding rule, but the rule does not establish whether the intended response was 3, 4, or something else.

The researcher must specify whether invalid values are flagged, corrected from authentic records, or handled as unresolved missing information.

AI can help implement checks for missing and impossible values, but should not invent permissible ranges or replacement values.

Who Decides How Missing Values Are Treated?

Missing-data treatment is a methodological decision because different procedures rely on different assumptions and can affect statistical inference.

Researchers should establish how missingness is represented, whether source recovery is possible, which analytical methods can accommodate incomplete observations, and whether imputation is appropriate.

If imputation is selected, its model, predictors, assumptions, and diagnostic requirements should be documented.

Rubin's foundational work on missing data demonstrates why the missingness mechanism matters to inference. Van Buuren's work on multiple imputation further illustrates that generating replacements requires more than selecting plausible numbers.

Consequently, automatic missing-data imputation should execute an approved analytical strategy rather than substitute for one.

Who Determines Which Duplicate Records Are Removed?

The researcher must define the unit of observation and the conditions under which repeated records are considered unintended duplicates.

For example, a participant appearing twice in a cross-sectional questionnaire may require investigation. The same participant appearing twice in a pretest-posttest study is ordinarily expected.

AI can identify repeated identifiers or matching responses, but it should not independently decide that identical records must be deleted.

When duplicate submissions are confirmed, the researcher should specify the retention rule, including how incomplete or conflicting submissions are handled.

Who Determines Whether Outliers Should Be Excluded?

Outlier screening and exclusion should be treated as separate decisions.

Researchers may specify an interquartile range rule or regression influence diagnostic to identify observations requiring investigation. However, a flagged value is not automatically erroneous.

The decision to correct, retain, or exclude an observation should depend on the research design, available source evidence, and analytical justification.

NIST's guidance on outliers recognizes that unusual observations may arise from errors or legitimate variation. Researchers should not remove them merely because they fall outside a conventional statistical threshold.

Where analytical conclusions depend strongly on unusual cases, sensitivity analyses may be more informative than automatic deletion.

Who Determines Recoding and Category Grouping Rules?

Researchers should specify the intended meaning of every transformed variable and the mapping between original and new values.

For example, reverse coding a five-point item requires the correct scale endpoints and confirmation that the instrument actually requires that item to be reversed.

Similarly, standardizing "BSIT" and "BS IT" may be justified when both represent the same academic program. Combining Information Technology and Computer Science into one category requires a separate substantive rationale.

The distinction between representation and meaning is central to AI-suggested data recoding rules.

Altman and Royston (2006) also demonstrate why grouping continuous variables can create information loss and statistical problems. Category boundaries should not be chosen simply because AI finds them convenient.

Must Researchers Approve Every Technical Formatting Change?

Not necessarily. Requiring individual approval for every harmless whitespace correction could make data preparation unnecessarily burdensome.

A more proportionate approach is to approve a rule such as trimming leading and trailing spaces from a specified categorical variable, then verify its execution.

However, even seemingly minor operations require attention to context. Removing spaces from free-text narratives or identifiers may alter meaning or record identity.

The appropriate distinction is between transformations whose effects are predictable and well-defined and those requiring interpretation of the underlying evidence.

Should Cleaning Rules Be Established Before Seeing the Results?

Where feasible, consequential cleaning and exclusion rules should be established before examining the substantive results they could influence.

This reduces the risk that decisions are selected because they produce preferred statistical outcomes.

However, researchers cannot anticipate every data-quality problem. Unexpected errors may emerge during inspection, requiring justified changes to the cleaning plan.

When rules are revised after examining the data, the researcher should document what prompted the revision, when it occurred, and whether it was informed by the substantive findings.

Exploratory decisions are not inherently invalid, but they should not be misrepresented as predetermined.

What Does an Explicit Cleaning Rule Look Like?

A useful rule identifies the relevant variable, condition, authorized action, exceptions, and verification procedure.

This rule can be implemented automatically without allowing AI to guess what an invalid response should have been.

How Should Researchers Record Rule Approval?

A cleaning specification should identify the rule, its rationale, responsible reviewer, approval status, and applicable dataset version.

For substantive decisions, it may also record the supporting instrument, protocol, or methodological reference.

When AI proposes a rule, the record should distinguish the proposal from the researcher's approved decision.

These records become part of the audit trail for AI-assisted data cleaning.

Preparing Researcher-Approved Cleaning Rules for a Student Survey

The AI executes approved detection and standardization procedures but cannot delete records or replace missing responses.

When an unexpected problem appears, the researcher reviews the evidence and documents an additional rule before authorizing further changes.

Interpretation: The researcher establishes methodological boundaries while AI assists with implementation. This arrangement permits efficient automation without allowing computational convenience to determine the meaning or inclusion of observations.

Common Misconceptions About Researcher-Specified Cleaning Rules

Which Decisions Can You Automate, and Which Require Approval?

Classify cleaning tasks according to whether they involve established rules, uncertain interpretation, or substantive analytical consequences.

Researchers should also decide which changes may be applied to a working copy and which operations are prohibited from modifying the master dataset.

After execution, verify that the AI followed the approved rules and that the resulting dataset contains no unintended transformations.

Before Approving AI-Assisted Data Cleaning Rules

Frequently Asked Questions About Researcher-Controlled Cleaning Rules

Researchers Define the Rules; AI Can Execute Them

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.