03 · What You Need to Know
What AI Actually Does When It Fills Missing Scientific Image Regions
What Is AI Image Inpainting?
Image inpainting is a computational process that estimates content within a missing, damaged, or deliberately masked image region.
Traditional approaches may interpolate information from neighboring pixels or use mathematical assumptions about image continuity. AI-based approaches can learn patterns from large collections of images and use those patterns to generate plausible content.
For example, if a region of a microscopy image is missing, a generative model may extend nearby cellular boundaries or synthesize textures resembling the surrounding specimen.
The resulting region may look realistic. However, realism does not establish that the generated structures correspond to what was originally present.
Inpainting therefore involves an inferential problem: the algorithm must estimate information that is absent or incomplete in the supplied image.
Does Filling a Missing Region Mean Recovering the Original Information?
Not necessarily. The distinction depends on what information remains available and how the reconstruction is performed.
Suppose a small section of an image was lost during file corruption. If the missing pixel values cannot be recovered from another copy, acquisition record, or redundant measurement, an AI model cannot establish their exact original values merely by examining neighboring pixels.
It may generate an estimate that is statistically plausible under its learned assumptions.
That estimate could resemble the original region, but other possible images might be equally consistent with the available information.
Data recovery
Restoring information from an authentic source, backup, redundant measurement, or recoverable acquisition record.
Model-based image completion
Estimating missing content using mathematical assumptions, learned patterns, or available measurements without necessarily recovering the exact original observation.
The distinction should be explicit whenever reconstructed regions influence scientific interpretation.
How Does Generative Inpainting Differ From Scientific Reconstruction?
General-purpose generative inpainting and scientific image reconstruction can both produce missing or estimated image content, but they may rely on substantially different information.
A general-purpose image editor often generates plausible pixels based on the surrounding image and learned visual patterns.
Scientific reconstruction may incorporate a physical measurement model, instrument characteristics, known acquisition geometry, or other constraints derived from the imaging process.
For example, reconstruction from undersampled magnetic resonance imaging data may use measured frequency-domain information and assumptions about the acquisition system.
Such methods can be scientifically useful, but their outputs remain dependent on the available measurements and reconstruction assumptions.
| Approach |
Information Used |
Principal Limitation |
| Recovery from original files |
Authentic backups or retained acquisition data |
Possible only when recoverable source information exists |
| Conventional interpolation |
Nearby pixel values and mathematical assumptions |
May smooth or inaccurately estimate structures |
| Generative inpainting |
Visible image context and learned visual patterns |
May synthesize plausible but unobserved features |
| Physics-informed reconstruction |
Instrument measurements and acquisition constraints |
May remain underdetermined or sensitive to modeling assumptions |
| Deep-learning reconstruction |
Measured data and learned representations |
May introduce model-dependent features or reconstruction instability |
No approach is automatically valid for every scientific application. The reconstruction method should be evaluated against the imaging modality, available measurements, and intended scientific endpoint.
Can AI Invent Structures That Were Never Present?
Yes. Generative models can produce structures that are plausible under their learned patterns but are not supported by the specific observation.
For example, a model trained on cellular images may generate an apparent membrane boundary within a missing region. The boundary may resemble authentic biological structures while having no independent evidential basis.
In medical imaging, a reconstructed feature may resemble normal anatomy even when the original measurements contain an unusual structure. Conversely, the model might introduce an abnormal-looking feature unsupported by the acquisition data.
These possibilities are especially consequential when reconstructed images influence diagnosis, classification, segmentation, or quantitative measurement.
The related concern that AI upscaling can introduce visual details reflects the same broader problem: visually plausible information may not be uniquely determined by the measured input.
Why Can Missing Scientific Information Be Impossible to Reconstruct Exactly?
Some reconstruction problems are underdetermined. This means the available measurements are compatible with more than one possible underlying image.
Imagine that an image contains a completely missing region. Many different structures could occupy that region while remaining consistent with the visible surrounding pixels.
A model may select one plausible completion based on its training and assumptions, but that selection does not establish which structure was actually present.
The problem becomes particularly important when the missing region contains irregular, rare, or unexpected features.
AI systems may favor patterns commonly represented in their training data, potentially suppressing unusual structures that are scientifically important.
Researchers should distinguish an estimate conditioned on available information from a uniquely recoverable observation.
Can Physics-Informed AI Reconstruction Be Scientifically Valid?
Yes, under appropriate conditions. Scientific imaging frequently relies on reconstruction methods because instruments do not always record the final image directly.
For example, magnetic resonance imaging, computed tomography, and certain microscopy techniques use computational procedures to reconstruct interpretable images from measured signals.
AI may contribute to these procedures by incorporating learned representations or assisting with reconstruction from incomplete measurements.
However, the use of physical constraints does not guarantee that every reconstructed feature is accurate.
Antun and colleagues (2020) demonstrated that certain deep-learning image reconstruction methods can exhibit instability, including substantial changes in reconstructed outputs following relatively small changes in measurements.
Their findings illustrate why reconstruction methods should be validated for the relevant measurement conditions and scientific tasks.
A reconstruction method may be appropriate for a defined analytical workflow while remaining unsuitable for claims about fine details that have not been independently verified.
Can AI Fill Missing Areas of Microscopy Images?
Technically, yes. AI models can estimate missing pixels or structures in microscopy images.
However, the scientific acceptability of the result depends on the reason the information is missing and how the completed image will be used.
If a region is absent because of file corruption, researchers should first investigate whether the original acquisition can be recovered or repeated.
If the missing information results from a defined imaging limitation, an established reconstruction procedure may be appropriate when sufficiently validated.
By contrast, generating cellular structures within an unobserved region and presenting them as authentic microscopy data would misrepresent the evidential record.
The researcher should not infer that an AI-generated structure was present in the specimen merely because it resembles neighboring structures.
Can AI Reconstruct Missing Parts of Medical Images?
AI-assisted reconstruction is an active area of medical imaging research, including applications involving incomplete or accelerated acquisitions.
Some methods seek to improve image quality or recover clinically relevant information from limited measurements.
However, reconstructed medical images must be evaluated according to the intended clinical or research use. Errors may affect the visibility of abnormalities, anatomical boundaries, or quantitative measurements.
For high-stakes applications, validation should include appropriate reference standards, clinically relevant performance measures, and assessment of unusual or difficult cases.
Researchers should not use general-purpose generative image completion as a substitute for validated medical imaging procedures.
Whether a method is suitable for clinical use also depends on applicable regulatory and institutional requirements.
What About Restoring Historical Photographs or Archival Research Images?
Historical and archival images present a different but related problem.
A damaged photograph may contain missing regions that AI can fill with plausible faces, objects, architectural details, or background textures.
Such restoration may be useful for illustration or public engagement, but the generated content should not be mistaken for documentary evidence of what the original photograph contained.
For example, an AI-restored photograph of an early classroom might add furniture or educational equipment that appears historically plausible but was never visible in the surviving source.
If the image is used as historical evidence, researchers should distinguish authentic surviving content from reconstructed or conjectural regions.
Preserving and, where appropriate, displaying the original alongside the restoration can help readers understand the evidential limits.
Can AI Fill Missing Image Data Before Quantitative Analysis?
Potentially, but the method requires validation for the specific analytical task.
Suppose a scientific image contains missing pixels within a region used to measure object size or fluorescence intensity.
Filling the missing region may change the calculated measurement because the generated values contribute directly to the result.
Researchers should evaluate whether the missingness mechanism, reconstruction assumptions, and available reference data justify the procedure.
Validation may involve artificially masking complete reference images, reconstructing the masked regions, and comparing estimated values with the known originals.
However, success on simulated missing regions does not guarantee accuracy when actual missingness arises under different conditions.
Researchers should consider sensitivity analyses and, where appropriate, report how conclusions change when reconstructed regions are excluded or treated as uncertain.
Can AI-Completed Images Be Used for Image Segmentation?
They may be used within a validated analytical workflow, but segmentation results can inherit errors from the reconstructed image.
For example, an AI-completed microscopy region may contain a synthetic cell boundary. A segmentation model could then identify that boundary as though it represented an observed cell.
The resulting cell count or area measurement would partly depend on generated information.
Researchers should distinguish segmentation of authentic observations from segmentation of model-estimated regions.
The separate task of using AI to segment scientific images requires validation of the segmentation method and the evidential quality of its inputs.
How Should Researchers Represent Uncertainty in Reconstructed Regions?
When missing regions are estimated rather than recovered, uncertainty should remain visible in the scientific interpretation.
Depending on the method, researchers may identify reconstructed regions through masks, annotations, supplementary figures, or explicit descriptions of the processing procedure.
Some reconstruction approaches can estimate uncertainty or produce multiple plausible completions. However, model-generated uncertainty estimates also require validation and should not automatically be treated as calibrated probabilities.
Where several completions are consistent with the available data, presenting a single output as definitive may be misleading.
The report should explain which features are directly supported by measurements and which depend on reconstruction assumptions.
Does an AI-Completed Image Become Authentic if It Looks Indistinguishable From the Original?
No. Visual realism does not establish provenance.
A generated region may match the surrounding texture, lighting, or structural pattern so well that readers cannot identify the alteration.
That does not mean the generated content was present in the original specimen or scene.
Indeed, seamlessness can increase the risk of misinterpretation because the boundary between observed and generated information becomes invisible.
Researchers should not use the absence of visible editing artifacts as evidence that a reconstruction is scientifically accurate.
What Do Journal Image Integrity Policies Say About Generative Filling?
Publisher policies commonly require scientific images to represent the underlying data accurately and restrict alterations that introduce or remove evidential content.
Nature Portfolio's image integrity guidance emphasizes faithful representation of original data and imposes restrictions on generative image editing.
Elsevier's generative AI policies distinguish permitted explanatory imagery and validated analytical workflows from prohibited generation or alteration of primary research images.
Researchers should verify the current requirements of their intended journal before submitting images containing reconstructed or generated regions.
A method used legitimately in computational imaging research may still require specific methodological documentation or may be restricted in a particular publication context.
Can Filling Missing Scientific Image Details Constitute Falsification?
Yes, when generated or estimated content is represented as authentic observed data in a way that misrepresents the research record.
For example, replacing a missing region of an experimental image with invented structures and presenting the completed image as an unaltered observation could raise serious falsification concerns.
However, not every use of reconstruction constitutes misconduct. Validated model-based estimation may be part of legitimate scientific methodology when its nature and limitations are accurately represented.
Formal misconduct determinations depend on the applicable definitions, evidence, and required findings concerning intent or recklessness.
The distinction between methodological estimation and scientific image falsification depends on what was changed, how it was reported, and whether the research evidence was misrepresented.
Watch Out
A completed image may contain pixels that no instrument ever recorded. Do not present reconstructed structures as directly observed features unless the relevant scientific claim is supported by appropriate independent evidence.
Should Researchers Preserve the Incomplete Original Image?
Yes. The original image should generally be retained, subject to applicable research-record requirements.
The incomplete source provides evidence of which information was actually available before reconstruction.
Researchers should also preserve the missing-region mask, relevant acquisition metadata, reconstruction method, model version, processing settings, and output files where these are necessary for verification.
The importance of preserving original scientific images is particularly clear when generated details could otherwise become indistinguishable from observed information.
Without the original and processing records, subsequent reviewers may be unable to determine which structures were measured and which were estimated.
