Exemplar-Based Image Inpainting With Multi-Resolution Information and the Graph Cut Technique

Filling holes in an image is achieved in a manner similar to peeling the onion. The order of filling affects the image inpainting results, especially concerning the content of complex images. When high-resolution images are used to extract edge information, they are susceptible to high-frequency information, such as complex textures and noise. Furthermore, edge information is extracted in different resolutions, while the main contour information of the image can be obtained more easily. In this paper, multi-resolution information is used to prioritize which target patches in an image to fill, which helps to elucidate the optimal sequence for image repair. Multi-resolution images provide more information than single-resolution images, and similar patches are computed on multi-resolution images to obtain multiple candidate patches. Similar patch calculations use a variety of information on colors, gradients, and boundaries to more accurately search for similar patches. We chose the most reasonable candidate patch by means of the structural similarity index measure (SSIM). When pasting the patch to fill the target region, we used graph cut technology to eliminate blockiness. Compared with the state-of-the-art repair algorithm, the experimental results prove that the proposed repair algorithm can repair the image very well.

[1]  Zongben Xu,et al.  Image Inpainting by Patch Propagation Using Patch Sparsity , 2010, IEEE Transactions on Image Processing.

[2]  Ting-Zhu Huang,et al.  Exemplar-Based Image Inpainting Using a Modified Priority Definition , 2015, PloS one.

[3]  Wei Hu,et al.  Image inpainting via sparse representation , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  Ce Zhu,et al.  Image Completion Using Low Tensor Tree Rank and Total Variation Minimization , 2019, IEEE Transactions on Multimedia.

[5]  Pascal Getreuer,et al.  Total Variation Inpainting using Split Bregman , 2012, Image Process. Line.

[6]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[7]  Guillermo Sapiro,et al.  A Variational Framework for Non-local Image Inpainting , 2009, EMMCVPR.

[8]  Rae-Hong Park,et al.  Robust exemplar-based inpainting algorithm using region segmentation , 2012, IEEE Transactions on Consumer Electronics.

[9]  Thomas S. Huang,et al.  Generative Image Inpainting with Contextual Attention , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Narendra Ahuja,et al.  Image completion using planar structure guidance , 2014, ACM Trans. Graph..

[11]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[12]  Yilong Yin,et al.  Patch-Based Image Inpainting via Two-Stage Low Rank Approximation , 2018, IEEE Transactions on Visualization and Computer Graphics.

[13]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[14]  Byungjin Chung,et al.  Hybrid error concealment method combining exemplar-based image inpainting and spatial interpolation , 2014, Signal Process. Image Commun..

[15]  Vicent Caselles,et al.  Exemplar-Based Image Inpainting Using Multiscale Graph Cuts , 2013, IEEE Transactions on Image Processing.

[16]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[17]  Xuelong Li,et al.  Matrix completion by Truncated Nuclear Norm Regularization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Qunsheng Peng,et al.  Image completion using structural priority belief propagation , 2009, MM '09.

[19]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[20]  Daniel Cohen-Or,et al.  Fragment-based image completion , 2003, ACM Trans. Graph..

[21]  Ting-Chun Wang,et al.  Image Inpainting for Irregular Holes Using Partial Convolutions , 2018, ECCV.

[22]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[23]  Eli Shechtman,et al.  Image melding , 2012, ACM Trans. Graph..

[24]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[25]  Haixia Wang,et al.  Exemplar-based image inpainting using structure consistent patch matching , 2017, Neurocomputing.

[26]  Bhabatosh Chanda,et al.  Scale-Invariant Image Inpainting Using Gradient-Based Image Composition , 2016, ICVGIP Workshops.

[27]  Joachim Weickert,et al.  Optimising Data for Exemplar-Based Inpainting , 2018, ACIVS.

[28]  Irfan Essa,et al.  Texture optimization for example-based synthesis , 2005, SIGGRAPH 2005.

[29]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[31]  Xinwei Yang,et al.  Effective exemplar-based image inpainting using low-rank matrix completion , 2015, 2015 IEEE 7th International Conference on Awareness Science and Technology (iCAST).

[32]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, IEEE Trans. Image Process..

[33]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[34]  Patrick Pérez,et al.  Non-Local Patch-Based Image Inpainting , 2017, Image Process. Line.

[35]  Christine Guillemot,et al.  Hierarchical Super-Resolution-Based Inpainting , 2013, IEEE Transactions on Image Processing.

[36]  T. Chan,et al.  Image inpainting by correspondence maps: A deterministic approach , 2003 .

[37]  Minghui Zhang,et al.  Gradient-based low rank method and its application in image inpainting , 2018, Multimedia Tools and Applications.

[38]  Carola-Bibiane Schönlieb,et al.  Combined First and Second Order Total Variation Inpainting using Split Bregman , 2013, Image Process. Line.

[39]  Gabriele Facciolo,et al.  Variational Framework for Non-Local Inpainting , 2015, Image Process. Line.

[40]  Weilan Wang,et al.  Thangka Image Inpainting Using Adjacent Information of Broken Area , .

[41]  Steven B. Damelin,et al.  On Surface Completion and Image Inpainting by Biharmonic Functions: Numerical Aspects , 2017, Int. J. Math. Math. Sci..

[42]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[43]  S. Padmavathi,et al.  Hirarchical Digital Image Inpainting Using Wavelets , 2012, ArXiv.

[44]  Nikos Komodakis,et al.  Image Completion Using Efficient Belief Propagation Via Priority Scheduling and Dynamic Pruning , 2007, IEEE Transactions on Image Processing.

[45]  Jeffrey J. Rodríguez,et al.  Image Inpainting Using Nonlocal Texture Matching and Nonlinear Filtering , 2019, IEEE Transactions on Image Processing.

[46]  Anupam,et al.  Fast and Enhanced Algorithm for Exemplar Based Image Inpainting , 2010, 2010 Fourth Pacific-Rim Symposium on Image and Video Technology.

[47]  Sekhar Mandal,et al.  A Group-Based Image Inpainting Using Patch Refinement in MRF Framework , 2018, IEEE Transactions on Image Processing.

[48]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.