Image Inpainting Based on Generative Adversarial Networks

Image inpainting aims to fill missing regions of a damaged image with plausibly synthesized content. Existing methods for image inpainting either fill the missing regions by borrowing information from surrounding areas or generating semantically coherent content from region context. They often produce ambiguous or semantically incoherent content when the missing region is large or with complex structures. In this paper, we present an approach for image inpainting. The completion model based on our proposed algorithm contains one generator, one global discriminator, and one local discriminator. The generator is responsible for inpainting the missing area, the global discriminator aims evaluating whether the repair result has global consistency, and the local discriminator is responsible for identifying whether the repair area is correct. The architecture of the generator is an auto-encoder. We use the skip-connection in the generator to improve the prediction power of the model. Also, we use Wasserstein GAN loss to ensure the stability of training. Experiments on CelebA dataset and LFW dataset demonstrate that our proposed model can deal with large-scale missing pixels and generate realistic completion results.

[1]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Junwu Zhu,et al.  On Cloud Resources Consumption Shifting Scheme for Two Different Geographic Areas , 2017, IEEE Systems Journal.

[4]  Bin Li,et al.  A Voting Aggregation Algorithm for Optimal Social Satisfaction , 2018, Mob. Networks Appl..

[5]  Kristen Grauman,et al.  On-demand Learning for Deep Image Restoration , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Gözde B. Ünal,et al.  Patch-Based Image Inpainting with Generative Adversarial Networks , 2018, ArXiv.

[7]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[8]  Bin Li,et al.  On complex tasks scheduling scheme in cloud market based on coalition formation , 2017, Comput. Electr. Eng..

[9]  Pethuru Raj,et al.  Modified Exemplar based Image Inpainting Algorithm , 2016 .

[10]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[11]  Ming-Hsuan Yang,et al.  Generative Face Completion , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Thomas S. Huang,et al.  Generative Image Inpainting with Contextual Attention , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Hiroshi Ishikawa,et al.  Globally and locally consistent image completion , 2017, ACM Trans. Graph..

[14]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Dan Xu,et al.  Exemplar-Based Pixel by Pixel Inpainting Based on Patch Shift , 2017, CCCV.

[16]  Bixin Li,et al.  Static change impact analysis techniques: A comparative study , 2015, J. Syst. Softw..

[17]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[18]  Yuejin Zhao,et al.  Image inpainting using Wasserstein Generative Adversarial Network , 2018, Optical Engineering + Applications.

[19]  Pasi Liljeberg,et al.  LiRCUP: Linear Regression Based CPU Usage Prediction Algorithm for Live Migration of Virtual Machines in Data Centers , 2013, 2013 39th Euromicro Conference on Software Engineering and Advanced Applications.

[20]  Matthew D. Zeiler ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[21]  Erik Learned-Miller,et al.  Labeled Faces in the Wild : Updates and New Reporting Procedures , 2014 .

[22]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[24]  Junwu Zhu,et al.  An auction-based rescue task allocation approach for heterogeneous multi-robot system , 2018, Multimedia Tools and Applications.

[25]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[28]  Bin Li,et al.  Wound intensity correction and segmentation with convolutional neural networks , 2017, Concurr. Comput. Pract. Exp..

[29]  Ting-Chun Wang,et al.  Image Inpainting for Irregular Holes Using Partial Convolutions , 2018, ECCV.

[30]  Eric van Damme,et al.  Non-Cooperative Games , 2000 .

[31]  Minh N. Do,et al.  Semantic Image Inpainting with Deep Generative Models , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Jianfei Cai,et al.  Pluralistic Image Completion , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Junwu Zhu,et al.  Adaptive consensus control of leader-following systems with transmission nonlinearities , 2019, Int. J. Control.

[34]  Huimin Lu,et al.  A pricing method of online group-buying for continuous price function , 2019, Neural Computing and Applications.

[35]  Thomas S. Huang,et al.  Free-Form Image Inpainting With Gated Convolution , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[36]  Mehran Ebrahimi,et al.  EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning , 2019, ArXiv.

[37]  Hao Li,et al.  High-Resolution Image Inpainting Using Multi-scale Neural Patch Synthesis , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[39]  Junwu Zhu,et al.  A Combinatorial Auction-based collaborative cloud services platform , 2015 .

[40]  Bin Li,et al.  A formal method for integrating distributed ontologies and reducing the redundant relations , 2009, Kybernetes.

[41]  Bin Li,et al.  On truthful auction mechanisms for electricity allocation with multiple time slots , 2018, Multimedia Tools and Applications.

[42]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.