Image inpainting through neural networks hallucinations

Abstract

We consider in this paper the problem of image inpainting, where the objective is to reconstruct large continuous regions of missing or deteriorated parts of an image. Traditional in-painting algorithms are unfortunately not well adapted to handle such corruptions as they rely on image processing techniques that cannot properly infer missing information when the corrupted holes are too large. To tackle this problem, we propose a novel approach where we rely on the hallucinations of pre-trained neural networks to fill large holes in images. To generate globally coherent images, we further impose smoothness and consistency regularization, thereby constraining the neural network hallucinations. Through illustrative experiments, we show that pre-trained neural networks contain crucial prior information that can effectively guide the reconstruction process of complex inpainting problems.