03 · What You Need to Know
How to Verify AI-Modified Research Figures Against Their Source Evidence
Why Must the Final Figure Be Verified Rather Than Only the Original?
Scientific figures are representations of research evidence. Their accuracy depends on the relationship between the displayed graphical elements and the data or analytical results they are intended to communicate.
When a figure changes, that relationship may change as well.
A modification can affect numerical content directly by altering plotted values or indirectly by changing how readers perceive those values.
For example, a graph may retain the same means while using a truncated axis that exaggerates their apparent differences. Another figure may retain its axis limits but associate the wrong legend labels with the plotted groups.
Verification must therefore address the final version that readers will encounter.
The original figure's accuracy provides a useful reference, but it does not automatically establish the accuracy of subsequent modifications.
Does Every Modification Require the Same Level of Checking?
No. A proportionate verification strategy considers what changed, how the change was implemented, and the consequences of an undetected error.
For example, changing a font size through a clearly identified plotting parameter generally presents less risk to numerical accuracy than reconstructing an entire graph using an image-generation system.
Nevertheless, even a font change can cause clipping or obscure a mathematical symbol.
Researchers should distinguish between low-risk presentation changes and modifications that affect the data representation.
| Modification |
Primary concern |
Appropriate verification |
| Typography or spacing |
Clipped or unreadable information |
Inspect the final figure and confirm unchanged analytical code. |
| Color or marker changes |
Incorrect category mapping |
Check every legend and graphical series. |
| Axis changes |
Distorted magnitudes or relationships |
Verify limits, transformations, units, and interpretation. |
| New chart type |
Changed statistical or perceptual emphasis |
Check the data encoding and intended analytical purpose. |
| Recalculated estimates |
Incorrect statistical quantities |
Compare values directly with verified analytical outputs. |
| Generative image reconstruction |
Invented, omitted, or altered graphical content |
Reconcile consequential elements with the original data and figure. |
These categories help determine the depth of checking, but they should not be interpreted as guarantees that a particular modification is harmless.
What Should Be Considered the Authoritative Source?
The appropriate reference depends on the figure's content and the research workflow.
For an empirical graph, the authoritative evidence ordinarily includes the dataset and verified analytical outputs from which the visualization was generated.
For a regression coefficient plot, the relevant reference may be a validated table of model estimates and confidence intervals.
For a descriptive chart, the analytical dataset and documented aggregation rules may provide the necessary basis for verification.
For a conceptual diagram, the relevant reference may instead be the approved theoretical framework, protocol, or documented system architecture.
The original image is useful for identifying changes, but it should not automatically be treated as the ultimate source of scientific truth. The original may itself contain an error.
Researchers should distinguish consistency with the earlier figure from accuracy relative to the underlying evidence.
Why Is Comparing Two Images Not Enough?
Visual comparison can identify obvious differences in labels, layout, colors, and graphical elements.
However, two figures may look almost identical while representing different numerical values.
A confidence interval endpoint may move slightly, or a data point may shift within a dense scatterplot. Such differences can be difficult to detect visually.
Conversely, two figures can look substantially different while preserving exactly the same underlying values.
Image comparison therefore provides evidence about visual similarity, not necessarily scientific equivalence.
Researchers should use numerical or code-based checks when the represented quantities matter to the research findings.
How Can Researchers Verify Plotted Values?
The verification process should connect the plotted graphical elements to their corresponding source values.
For example, if a figure displays mean engagement scores for three instructional groups, researchers can calculate those means independently from the analytical dataset and compare them with the plotted values.
For coefficient plots, researchers can compare the point estimates and interval endpoints with the statistical model output.
For scatterplots, the number of displayed observations and the mapping of variables should be checked against the intended analytical sample.
Where plotting code is available, researchers can inspect the arrays or data frames passed to the plotting functions rather than attempting to estimate values from the rendered image.
This is especially important when AI has regenerated or substantially rewritten the plotting code.
Why Must Sample Sizes and Missing-Data Rules Be Checked?
AI-generated plotting code may silently exclude observations with missing values or apply filtering that differs from the intended analysis.
For example, a revised figure may display only participants with complete values for all plotted variables, even though the original analysis used a different inclusion rule.
The resulting graph may appear plausible while representing a different analytical sample.
Researchers should verify group sizes, filtering criteria, missing-data handling, and any transformations applied before plotting.
Where sample sizes vary across analyses, the figure or caption should provide sufficient information to avoid misleading comparisons.
How Should Statistical Estimates and Uncertainty Be Verified?
Statistical figures often display point estimates together with measures of uncertainty.
These may include confidence intervals, credible intervals, prediction intervals, standard errors, or standard deviations.
Researchers should verify both the numerical values and the statistical meaning of the displayed intervals.
For example, an AI-generated revision might replace model-based confidence intervals with intervals calculated using a generic formula inappropriate for the study design.
Even when the intervals appear similar, their interpretation may differ.
Where possible, figures should use verified analytical estimates rather than recalculate them through an unrelated plotting procedure.
AI should not independently select a new uncertainty calculation merely to improve the visual presentation.
Why Do Axis Scales Require Verification Even When the Data Are Unchanged?
Axis scales influence how readers perceive magnitude, variation, and relationships.
A figure may preserve every numerical value while changing the apparent strength of a difference through axis truncation or transformation.
For conventional bar charts, a zero baseline is generally important because bar length represents magnitude.
For scatterplots and line graphs, restricted ranges may be appropriate when clearly labeled and scientifically justified.
Researchers should therefore examine the axes and scales of AI-modified research graphs in relation to their visual encodings.
The goal is not to enforce one axis rule across all charts, but to prevent the final representation from encouraging an unsupported interpretation.
Can AI Change the Meaning of Colors and Legends?
Yes. A change in palette may accidentally alter which graphical elements correspond to particular groups.
For example, a researcher may originally use blue for the control group and orange for the intervention group. An AI-generated revision could exchange those colors without updating the legend consistently.
The plotted values remain unchanged, but their apparent group identities become incorrect.
Heatmaps introduce additional concerns because colors encode numerical magnitudes rather than only categories.
Researchers should verify that the relationship between values, colors, symbols, and labels remains correct after modification.
What If AI Only Adds Labels or Annotations?
Annotations can influence scientific interpretation even when the underlying plotted values are unchanged.
An AI assistant might add a label describing a difference as statistically significant, a trend as causal, or an observation as an outlier.
These descriptions require evidence.
Researchers should verify that annotations accurately represent the analytical results and do not introduce unsupported conclusions.
Similarly, an arrow or highlight may direct attention toward a pattern that is not the primary finding or that depends on a selective interpretation.
Presentation choices should clarify the evidence rather than substitute for scientific justification.
How Should Code-Based AI Modifications Be Checked?
When AI modifies a plotting script, version-control differences can help identify the changes.
Researchers should inspect whether the revision affects only presentation parameters or also changes filtering, grouping, calculations, and statistical transformations.
For example, a request to improve a bar chart's appearance might unexpectedly change the aggregation function from median to mean.
The revised script may still run successfully and produce a convincing figure.
Researchers should compare consequential code changes with the intended analytical specification and rerun relevant tests.
Maintaining version-controlled AI-assisted code revisions can make such changes easier to identify.
How Should Image-Based AI Modifications Be Checked?
Image-based generative editing presents a different verification problem because the revised figure may not retain an explicit connection to the original numerical data.
An image-generation system may reconstruct graphical marks, text, or patterns while attempting to improve appearance.
Researchers should compare the edited image with the original and verify consequential details against the source evidence.
For dense empirical figures, manually reconciling every generated mark may be impractical.
In those cases, regenerating the figure from the original data and plotting code is generally preferable.
The possibility of AI introducing visual errors during figure enhancement is one reason image-based reconstruction should not be treated as an automatically reliable scientific editing method.
Can Automated Tests Help Verify AI-Modified Figures?
Yes. Automated checks can compare the numerical inputs used for plotting, validate expected sample sizes, and identify changes in statistical outputs.
For example, a test may confirm that a coefficient plot contains the expected number of estimates and that its numerical values match a preserved results table.
Other checks may verify category ordering, axis transformations, or the presence of required labels.
However, automated tests cannot fully establish whether a visualization is perceptually appropriate or scientifically clear.
A graph may pass numerical checks while using a misleading visual encoding.
Researchers should combine computational verification with informed visual inspection.
Can AI Verify Its Own Figure Modifications?
AI can assist by identifying visible differences, reviewing plotting code, or suggesting verification checks.
However, its assessment should not be treated as independent certification.
The same model may overlook a change it introduced or incorrectly conclude that a revised graph preserves the original meaning.
Researchers should establish verification criteria independently of the AI-generated revision.
Where numerical accuracy is consequential, direct comparison with verified analytical outputs provides stronger evidence than model reassurance.
What About Scientific Diagrams Without Numerical Data?
Not every research visualization represents a dataset directly.
Conceptual frameworks, process diagrams, methodological flowcharts, and system architectures may communicate relationships rather than numerical measurements.
For these figures, verification should focus on whether components, labels, arrows, sequences, and relationships remain consistent with the intended scientific content.
For example, changing an arrow direction in a conceptual framework may reverse a hypothesized relationship.
Researchers should therefore adapt the verification reference to the type of figure rather than demand an underlying numerical dataset where none exists.
Does Verification Need to Be Documented?
Documentation is particularly valuable when figures are central to the research findings or when modifications are substantial.
Researchers can preserve the original and revised figures, the plotting code, the relevant analytical outputs, and a concise record of accepted changes.
For consequential modifications, the record may identify what was changed, why it was changed, and how the final figure was checked.
This supports transparency and allows collaborators to investigate discrepancies later.
The level of documentation should reflect the significance of the figure and applicable institutional or publication requirements.
Can a Verified Figure Still Be Misleading?
Yes. Numerical verification does not automatically establish that a figure communicates the evidence appropriately.
A chart may use correct values but omit relevant uncertainty, conceal distributional differences, or employ scales that exaggerate apparent effects.
Similarly, an accurate graph may be accompanied by a caption that overstates causality or statistical significance.
Researchers should therefore distinguish numerical accuracy from interpretive validity.
The broader task of identifying misleading research graphs complements direct verification against the underlying data.
