Object removal by depth-wise image inpainting

The problem posed in this paper was to fill-up the hole in the digital image left behind after the removal of an object from it. The real challenge is to deal with images of cluttered scenes of objects with complex and infrequent textures. In the past, this problem has been addressed by two classes of algorithms: (i) exemplar-based algorithms to fill the hole using the patches available in a given example and (ii) depth-based algorithms to differentiate the foreground and background before inpainting. This paper presents a novel method that combines advantages of these two approaches. We use depth map of an image to find the order of all the objects in the target image and a database of multi-views of objects to fill the holes. To reduce the size of the database, we propose a method called Keyview Extraction. An object retrieval followed by geometric and photometric registration algorithms is employed to make every object an exact match for inpainting. A number of experiments on both real and synthetic images demonstrate the advantage of depth-wise image inpainting compared with other inpainting methods. In these experiments, we compare three different types of image inpainting methods with our method quantitatively by computing their SSIM values using a ground-truth.