Image Inpainting Based on Multi-Patch Match with Adaptive Size

Patch-based image inpainting methods iteratively fill the missing region via searching the best sample patch from the source region. However, most of the existing approaches basically use the fixed size of patch regardless of content features nearby, which may lead to inpainting defects. Also, global match is needed for searching the best sample patch, but only to fill one target patch in each iteration, resulting in low efficiency. To handle the issues above, we first evaluate the nonuniformity in an image, by which the patch size is adaptively determined. Moreover, we divide the source region into multiple non-overlapping subregions with different nonuniformity levels, and the patch match proceeds in every subregion, respectively. This strategy not only saves the match time for single target patch, but also reduces the mismatch, and enables the simultaneous filling of multiple target patches in a single iteration. Experimental results show that in comparison to previous patch-based works, our method has achieved further improvement both in quality and efficiency. We believe our method could provide a new way for patch match with better accuracy and efficiency in image inpainting tasks.

[1]  Patrick Pérez,et al.  Object removal by exemplar-based inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[2]  Jana Reinhard,et al.  Textures A Photographic Album For Artists And Designers , 2016 .

[3]  Fulong Wang,et al.  Improved Criminisi Algorithm Based on a New Priority Function with the Gray Entropy , 2013, 2013 Ninth International Conference on Computational Intelligence and Security.

[4]  Aline Roumy,et al.  Exemplar-based image inpainting: Fast priority and coherent nearest neighbor search , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[5]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[6]  Tony F. Chan,et al.  Euler's Elastica and Curvature-Based Inpainting , 2003, SIAM J. Appl. Math..

[7]  Huaiyu Cai,et al.  Artifact Handling Based on Depth Image for View Synthesis , 2019 .

[8]  Manuel Menezes de Oliveira Neto,et al.  Fast Digital Image Inpainting , 2001, VIIP.

[9]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[10]  Alexander A. Sawchuk,et al.  Computational Models For Texture Analysis And Synthesis , 1981, Other Conferences.

[11]  Ke-wen Xia,et al.  Research on weighted priority of exemplar-based image inpainting , 2012 .

[12]  Hailing Zhou,et al.  Adaptive patch size determination for patch-based image completion , 2010, 2010 IEEE International Conference on Image Processing.

[13]  David Donovan Garber,et al.  Computational models for texture analysis and texture synthesis , 1981 .

[14]  Qian Shen,et al.  A Novel Exemplar-Based Image Inpainting Algorithm , 2015, 2015 International Conference on Intelligent Networking and Collaborative Systems.

[15]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[16]  Baining Guo,et al.  Real-time texture synthesis by patch-based sampling , 2001, TOGS.

[17]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[18]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[19]  Safia Abdelmounaime,et al.  New Brodatz-Based Image Databases for Grayscale Color and Multiband Texture Analysis , 2013 .

[20]  Mehran Ebrahimi,et al.  EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning , 2019, ArXiv.

[21]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[22]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.