Institutional Repository [SANDBOX]
Technical University of Crete
EN  |  EL

Search

Browse

My Space

Hallucination detection in image inpainting

Zafeirakis Konstantinos

Full record


URI: http://purl.tuc.gr/dl/dias/ADC011A6-AB63-4161-9C5B-F870209E07D3
Year 2024
Type of Item Diploma Work
License
Details
Bibliographic Citation Konstantinos Zafeirakis, "Hallucination detection in image inpainting", Diploma Work, School of Electrical and Computer Engineering, Technical University of Crete, Chania, Greece, 2024 https://doi.org/10.26233/heallink.tuc.100492
Appears in Collections

Summary

This thesis addresses the critical issue of hallucinations in deep learning-based image inpainting within remote sensing applications. Remote sensing images are often degraded due to sensor malfunctions or adverse atmospheric conditions. As such, they require inpainting to restore missing information accurately. This restoration is vital for enabling downstream tasks such as classification, detection, and segmentation. Despite the advancements, deep learning models for inpainting face multiple challenges including hallucinations, where the model incorrectly introduces non-existent elements in the image. This study introduces a novel framework for detecting hallucinations using an image inpainting generator coupled with a two-class discriminator and a class activation mapping (Grad-CAM) model. The experimental setup involves diverse masking techniques and analyzes the inpainting results across different image classes. Our findings reveal significant impacts of mask type and size on hallucination metrics, with rectangular masks generally yielding better results than irregular and random masks. Additionally, each class-specific generator exhibited unique inpainting behaviors, influenced by mask size. The study identifies the in-distribution Dice metric and out-of-distribution prediction value as effective measures for hallucination detection, with the FID metric proving optimal for reconstruction quality.

Available Files

Services

Statistics