03 · What You Need to Know
Why AI-Generated Graphs Can Misrepresent Correct Research Data
How Can a Graph Be Visually Correct but Numerically Wrong?
A graph may follow conventional design practices while containing errors in the data it represents.
For example, an AI-generated bar chart may have a properly labeled vertical axis, consistent bar widths, and an appropriate legend. Yet the heights of the bars may correspond to incorrectly calculated means.
The graph is visually coherent because its elements follow recognizable graphical conventions. Its scientific accuracy, however, depends on the numerical information encoded by those elements.
This distinction is particularly important when researchers evaluate figures by asking whether they look reasonable.
An incorrect mean of 76.4 may appear entirely plausible when the verified mean is 74.6. The difference may be difficult to detect without comparing the figure against the underlying analytical output.
What Are the Main Types of Numerical Errors in AI-Generated Graphs?
Errors can arise during data preparation, statistical calculation, graphical mapping, or rendering.
| Error Type |
What Happens |
Example |
| Incorrect variable selection |
The graph uses values from the wrong dataset column. |
Plotting pretest scores instead of posttest scores. |
| Incorrect aggregation |
The graph summarizes observations using the wrong calculation or grouping. |
Plotting a sum instead of a mean. |
| Category mismatch |
Correct values are assigned to incorrect labels. |
Exchanging treatment and control group values. |
| Missing-data error |
Missing or excluded observations are handled incorrectly. |
Including missing scores coded as zero. |
| Incorrect uncertainty |
Error bars represent the wrong quantity or calculation. |
Displaying standard deviations labeled as confidence intervals. |
| Graphical encoding error |
The plotted visual element does not correspond to its intended value. |
A bar labeled 80 visually reaching 85. |
| Invented values |
The system introduces observations or statistics absent from the research data. |
Generating an additional measurement occasion. |
| Incorrect transformation |
Values are transformed improperly or displayed on an unexplained scale. |
Plotting logarithmic values as though they were raw measurements. |
These errors are not exclusive to AI. Conventional statistical software can also produce incorrect figures when researchers provide inappropriate instructions. Generative AI adds another opportunity for errors when it interprets natural-language requests and generates analytical procedures.
Can AI Plot the Wrong Values Even When the Dataset Is Correct?
Yes. An accurate dataset does not guarantee accurate data processing.
Suppose a dataset contains variables for pretest scores, immediate posttest scores, and delayed posttest scores.
The researcher requests a graph comparing pretest and immediate posttest performance, but AI mistakenly selects the delayed posttest column.
The resulting graph may use genuine observations and calculate its summaries correctly. Nevertheless, it represents the wrong outcome.
This is an important distinction: a graph can be computationally correct relative to the selected data while scientifically incorrect relative to the research question.
Researchers should verify variable mappings directly rather than relying on axis labels, which may themselves have been generated independently of the selected columns.
Can AI Assign Correct Numbers to the Wrong Categories?
Yes. Category mapping errors can reverse or distort the apparent findings without changing any numerical values.
For example, suppose the verified mean score is 72 for the control group and 81 for the treatment group.
An AI-generated graph might display the values correctly but reverse their labels.
The graph would then appear to show that the control group outperformed the treatment group.
Such an error can occur when code sorts labels and numerical arrays independently or when the researcher provides values without stable group identifiers.
Using structured data with explicit variable names and group mappings can reduce this risk, although the final figure must still be verified.
How Can Incorrect Aggregation Produce a Misleading Graph?
Aggregation converts multiple observations into summaries such as means, totals, percentages, or medians.
AI may misunderstand which quantity should be calculated or which observations should contribute to it.
For example, a researcher requests average assessment scores by academic program. The generated code may accidentally calculate total scores by program.
Programs with more students would then appear to have higher performance simply because they contribute more observations.
Another error occurs when a dataset contains repeated measurements from the same participants and AI calculates summaries at an inappropriate level.
The correct aggregation depends on the research question, sampling structure, and intended estimand.
Can AI Miscalculate Percentages?
Yes. Percentage calculations require an appropriate numerator and denominator.
For example, the percentage of students who completed an online course could be calculated relative to all enrolled students, students who started the course, or students with complete records.
These denominators answer different questions.
If AI selects the wrong denominator, the percentage may be mathematically correct but substantively misleading.
Researchers should verify how categories were defined, which observations were included, and whether the percentages represent the intended population.
Figures should identify the relevant sample or denominator when it is necessary for interpretation.
Can AI Generate Incorrect Error Bars?
Yes. Error bars are particularly vulnerable to errors because their meaning depends on statistical conventions and study design.
A graph may display error bars representing standard deviations, standard errors, confidence intervals, or another measure of variation or uncertainty.
These quantities should not be treated as interchangeable.
For example, AI may calculate standard deviations correctly but label them as 95% confidence intervals.
Alternatively, it may calculate a conventional standard error without accounting for repeated or clustered observations.
Researchers should inspect the exact calculations used to produce uncertainty indicators and confirm that the caption describes them accurately.
Can AI Misrepresent Regression Results in a Graph?
Yes. Regression visualizations introduce additional opportunities for numerical and interpretive errors.
AI may plot coefficients from the wrong model, omit reference categories, misrepresent confidence intervals, or apply incorrect transformations.
For example, logistic regression coefficients are commonly expressed on a log-odds scale. Exponentiating them produces odds ratios.
A coefficient plot must distinguish these representations because their reference values differ.
An odds-ratio plot generally uses a reference value of one, while an unexponentiated coefficient plot uses zero.
Using the wrong reference line can mislead readers about the direction or interpretation of an association.
Researchers should compare every plotted estimate and interval with the verified model output.
Can AI Plot Incorrect Coordinates in Scatterplots?
Yes. Scatterplots require the correct pairing of observations across variables.
Suppose a dataset contains students' weekly study hours and assessment scores. Each plotted point should represent the two values belonging to the same student.
If AI-generated code sorts the variables independently before plotting, the coordinates may combine values from different students.
The resulting scatterplot could suggest a relationship that does not exist in the original paired observations.
This problem may be difficult to recognize because the plotted values themselves remain within the expected ranges.
Researchers should verify observation identifiers and the alignment of variables before interpreting plotted relationships.
Can AI Produce Incorrect Time-Series Graphs?
Yes. Time-series graphs depend on correct ordering, date interpretation, and treatment of intervals.
AI may sort dates as text, combine measurements from different periods, or treat irregular observation intervals as equally spaced.
For example, a graph showing monthly student engagement might incorrectly place October before February because the date field was interpreted alphabetically.
Another problem occurs when AI fills missing months with invented values to produce a continuous line.
Researchers should verify the time variable, chronological order, missing periods, and whether connecting observations is methodologically appropriate.
Can a Graph Display Correct Labels but Incorrect Bar Heights?
Yes, particularly when a chart is produced as a rendered image rather than through a reproducible numerical plotting system.
A general-purpose image generator may create bars, axes, and numerical annotations that are not geometrically consistent.
For example, two bars labeled 60 and 80 might have nearly identical heights, or the bar labeled 60 might appear taller.
The numerical labels may be correct while the visual encoding is wrong.
Research figures should therefore be generated using methods that map verified numerical values to graphical positions or dimensions through defined procedures.
As discussed in AI-assisted research figure creation, reproducible computational plotting is generally preferable for empirical graphs.
Why Can Visual Inspection Fail to Detect Numerical Errors?
Visual inspection is useful for identifying obvious problems, but it is not a reliable substitute for numerical verification.
Small errors may produce only subtle differences in graphical positions or lengths.
Even larger errors may appear plausible when the viewer does not know the verified values.
Research on graphical perception by Cleveland and McGill (1984) demonstrates that different visual encodings vary in how accurately people can judge quantitative relationships.
This reinforces the need to evaluate numerical values directly rather than relying exclusively on visual estimates.
A figure should be checked against the data supplied to the plotting function, not merely judged by whether its pattern looks reasonable.
Can AI-Generated Plotting Code Produce Wrong Values?
Yes. Executable code can contain logical errors while running successfully.
For example, code may calculate the mean of the wrong variable, use an inappropriate filter, or group observations incorrectly.
The software will then plot the resulting values according to the instructions it receives.
The fact that the code executes without an error message does not establish that the calculations are scientifically correct.
Researchers should inspect the code, verify intermediate analytical outputs, and compare results with independent calculations where appropriate.
Does Reproducibility Guarantee That the Graph Is Accurate?
No. Reproducibility means that the same procedure can generate the same result under specified conditions.
An incorrect plotting script may reproduce the same incorrect graph repeatedly.
Nevertheless, reproducibility supports verification because it allows researchers to identify where a mistake occurred.
For example, a reviewer may discover that a graph used the wrong column by examining the code and plotting data.
Reproducibility and accuracy should therefore be treated as related but distinct requirements.
How Can Researchers Detect Incorrect Plotted Values?
The most reliable approach is to compare the plotting data with independently verified source values.
For a bar chart, this means checking each displayed summary against the original statistical output.
For a scatterplot, it means confirming that each coordinate pair corresponds to the correct observation.
For a regression figure, it means checking coefficients, confidence intervals, transformations, and reference values.
Researchers should also inspect labels, legends, axis limits, and graphical mappings.
| Figure Component |
Verification Question |
| Plotted values |
Do they match the original observations or verified analytical results? |
| Group labels |
Are values assigned to the correct categories? |
| Statistical summaries |
Were the intended calculations and denominators used? |
| Error bars |
Do they represent the stated uncertainty or variation measure? |
| Axes |
Are units, scales, transformations, and limits correct? |
| Data exclusions |
Were missing or excluded observations handled according to the analysis plan? |
| Visual encoding |
Do graphical positions, lengths, and proportions correspond to the numerical values? |
| Interpretation |
Does the figure support the conclusion presented in the manuscript? |
The more consequential the figure is to the study's conclusions, the stronger the justification for independent verification of its underlying calculations.
Can Automated Checks Detect Graphing Errors?
Yes. Researchers can use programmatic checks to identify inconsistencies between verified data and plotted values.
For example, a script can compare calculated group means with the values supplied to a plotting function and flag differences beyond an appropriate numerical tolerance.
Other checks can confirm expected category labels, sample sizes, missing-value counts, and the presence of required observations.
However, automated checks must themselves be designed correctly.
If both the plotting script and verification script use the same incorrect assumption, they may agree while remaining wrong.
Independent calculations or review of the original analytical outputs can provide additional assurance.
What Is the Difference Between Incorrect Values and Invented Data?
Incorrect plotted values may arise from calculation errors, mislabeling, or inappropriate transformations of authentic data.
Invented data involve introducing observations or results that were not obtained through the reported research or a properly identified analytical procedure.
Incorrect plotted value
A value displayed inaccurately because of a calculation, mapping, transformation, or graphical encoding error.
Invented research value
A numerical observation or result introduced without an authentic or appropriately documented analytical basis and potentially misrepresented as empirical evidence.
For example, plotting 76 instead of the verified mean of 74 may result from an error. Creating a fictional third experimental group and presenting its scores as observed results raises a different integrity concern.
The issue of AI inventing data during research visualization should therefore be distinguished from ordinary numerical mistakes, although both can compromise the accuracy of a figure.
Can Incorrect Graphs Affect Research Conclusions?
Yes. Figures influence how readers understand differences, trends, associations, and uncertainty.
An incorrect group label could reverse the apparent direction of an intervention effect. A miscalculated mean could exaggerate a difference. Incorrect confidence intervals could misrepresent the precision of an estimate.
These problems may affect the manuscript's interpretation even when the underlying statistical analysis was performed correctly.
Researchers should therefore verify that the narrative description, statistical tables, and figures consistently represent the same analytical results.
Can an Incorrect AI Graph Constitute Research Misconduct?
An incorrect graph does not automatically establish research misconduct.
Errors may arise from programming mistakes, misunderstanding of the data, or inappropriate analytical decisions.
However, knowingly or recklessly presenting altered or invented results as authentic research evidence may raise fabrication or falsification concerns under applicable research misconduct standards.
The U.S. Office of Research Integrity distinguishes fabrication and falsification from honest error.
Researchers should investigate discrepancies, correct affected figures, and determine whether the conclusions remain supported before making judgments about responsibility.
How Should Researchers Document Graph Verification?
A defensible verification record should connect the final figure to the original data and analytical outputs.
Relevant records may include the source dataset, variable definitions, calculation scripts, plotting data, final figure, and corrections made during review.
For important figures, a separate verification table can document the exact values expected and those displayed.
The process of verifying every value in AI-generated research figures provides a systematic approach to this task.
Researchers should also follow the current requirements of their target journal concerning figure preparation, data availability, and AI assistance.
Watch Out
Do not rely on a graph's visual plausibility as evidence of numerical accuracy. AI can produce a polished figure using the wrong variable, the wrong calculation, or the wrong category mapping. The error may be invisible until the plotted values are compared with the original analytical records.
