A Method of Semantic Image Inpainting with Generative Adversarial Networks

Semantic image inpainting focuses on the completing task of high-level missing regions at the basis of the uncorrupted image. The classical methods of image inpainting can only deal with low-level or mid-level missing regions due to the lack of representation of the image. In the essay, we conclude a new method of semantic image inpainting. It’s based on the generative model with learning the representation of image database. We propose an architecture of completion model using perceptual loss and contextual loss based on generative adversarial networks after having trained generative model using DCGAN. We qualitatively and quantitatively explore the effect of missing regions of different types and sizes on image inpainting. Our method successfully completes inpainting tasks in large missing regions and results looks realistic with extensive experiments. We conclude that the performance of our model mostly is good when completing image corrupted with the mask with an area of less than 50% as well as with center or random masks.

[1]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2007, SIGGRAPH 2007.

[2]  Gang Hua,et al.  Labeled Faces in the Wild: A Survey , 2016 .

[3]  Minh N. Do,et al.  Semantic Image Inpainting with Deep Generative Models , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  H. Niederreiter,et al.  Quasi-Monte Carlo Methods , 2010 .

[5]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[8]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[9]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[10]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[11]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[12]  Ming-Hsuan Yang,et al.  Generative Face Completion , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[14]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Andrew Zisserman,et al.  Get Out of my Picture! Internet-based Inpainting , 2009, BMVC.

[16]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[17]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.