Edge-guided generative adversarial network for image inpainting

In this paper, we present an edge-guided generative adversarial network (EGGAN) for edge-based image inpainting that can be adopted in image compression and transmission error concealment. Our key idea is to integrate edges into the generative network, and train the generative network to minimize both gradient loss and adversarial loss. Given a corrupted image and the estimated edges of the missing area, the trained generative network is capable in generating the missing area in a visually plausible manner, and meanwhile reproducing the given edges faithfully. Experimental results on the challenging face images have shown the effectiveness of EGGAN.

[1]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2008, Commun. ACM.

[2]  Oscar C. Au,et al.  Edge-Directed Error Concealment , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[4]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[5]  Daniel Cohen-Or,et al.  Fragment-based image completion , 2003, ACM Trans. Graph..

[6]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[7]  Graham W. Taylor,et al.  Deconvolutional networks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Dong Liu,et al.  Image Compression With Edge-Based Inpainting , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[10]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[11]  Minh N. Do,et al.  Semantic Image Inpainting with Perceptual and Contextual Losses , 2016, ArXiv.

[12]  Marcel J. T. Reinders,et al.  Edge-based image restoration , 2005, IEEE Transactions on Image Processing.

[13]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[14]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Guillermo Sapiro,et al.  Structure and texture filling-in of missing image blocks in wireless transmission and compression applications , 2003, IEEE Trans. Image Process..