Hierarchical Guidance Strategy and Exemplar-Based Image Inpainting

To solve the issue that it is difficult to maintain the consistency of linear structures when filling large regions by the exemplar-based technique, a hierarchical guidance strategy and exemplar-based image inpainting technique is proposed. The inpainting process is as follows: (i) the multi-layer resolution images are firstly acquired through decomposing of the pyramid on the target image; (ii) the top-layer inpainted image, the beginning of the inpainting from the top layer, is generated by the exemplar-based technique; (iii) there is a combined result between the next layer of the top image and the up-sampling output on the top-layer inpainted image, and the target regions are filled with information as guidance data; (iv) this process is repeated until the inpainting of all layers have been completed. Our results were compared to those obtained by existing techniques, and our proposed technique maintained the consistency of linear structures in a visually plausible way. Objectively, we choose SSIM (structural similarity index measurement) and PSNR (peak signal-to-noise ratio) as the measurement indices. Since the values of SSIM are well reflected when compared with other techniques, our technique clearly demonstrated that our approach is better able to maintain the consistency of linear structures. The core of our algorithm is to fill large regions whether they are synthesis images or real-scene photographs. It is easy to apply in practice, with the goal of having plausible inpainted image.

[1]  Rae-Hong Park,et al.  Robust exemplar-based inpainting algorithm using region segmentation , 2012, IEEE Transactions on Consumer Electronics.

[2]  Zhijing Yang,et al.  Blind inpainting using the fully convolutional neural network , 2017, The Visual Computer.

[3]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[4]  Baek-Sop Kim,et al.  Exemplar based inpainting in a multi-scaled space , 2015 .

[5]  Hao Li,et al.  High-Resolution Image Inpainting Using Multi-scale Neural Patch Synthesis , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Arif Mahmood,et al.  Image inpainting based on pyramids , 2010, IEEE 10th INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING PROCEEDINGS.

[7]  Qing Zhang,et al.  Exemplar-Based Image Inpainting Using Color Distribution Analysis , 2012, J. Inf. Sci. Eng..

[8]  Vicent Caselles,et al.  Exemplar-Based Image Inpainting Using Multiscale Graph Cuts , 2013, IEEE Transactions on Image Processing.

[9]  Cláudio Rosito Jung,et al.  Block-based image inpainting in the wavelet domain , 2007, The Visual Computer.

[10]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[11]  Raymond H. Chan,et al.  Alternating Direction Method for Image Inpainting in Wavelet Domains , 2011, SIAM J. Imaging Sci..

[12]  Bernhard Schölkopf,et al.  Mask-Specific Inpainting with Deep Neural Networks , 2014, GCPR.

[13]  Dong Liang,et al.  An new method for image inpainting using wavelets , 2011, 2011 International Conference on Multimedia Technology.

[14]  Pascal Getreuer,et al.  Total Variation Inpainting using Split Bregman , 2012, Image Process. Line.

[15]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[16]  Tony F. Chan,et al.  Euler's Elastica and Curvature-Based Inpainting , 2003, SIAM J. Appl. Math..

[17]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[18]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[19]  Daniel Cohen-Or,et al.  Fragment-based image completion , 2003, ACM Trans. Graph..

[20]  Zongben Xu,et al.  Image Inpainting by Patch Propagation Using Patch Sparsity , 2010, IEEE Transactions on Image Processing.

[21]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[23]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[24]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[25]  S. Padmavathi,et al.  Hirarchical Digital Image Inpainting Using Wavelets , 2012, ArXiv.

[26]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, IEEE Trans. Image Process..

[27]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.