A UNIFYING FRAMEWORK FOR IMAGE INPAINTING By Aur élie Bugeau

Inpainting is the art of modifying an image in a form that is not detectable by an ordinary observer. There are numerous and very different approaches to tackle the inpainting problem, but we point out that the most successful inpainting algorithms are based on one or two of the following three basic techniques: copy-and-paste texture synthesis, geometric PDE’s, and coherence among neighboring pixels. We combine these three building blocks in a unifying variational model, and provide a working algorithm for image inpainting trying to approximate the minimum of the proposed energy functional. Our experiments show that the combination of all three terms of the proposed energy works better than taking each term separately, and the results obtained are state-of-the-art. Index Terms Image inpainting, variational models, texture synthesis, PDE’s.

[1]  Patrick Pérez,et al.  Object removal by exemplar-based inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[2]  Stanley Osher,et al.  Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing , 2003, J. Sci. Comput..

[3]  Mohamed-Jalal Fadili,et al.  Inpainting and Zooming Using Sparse Representations , 2009, Comput. J..

[4]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[5]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[6]  T. Chan,et al.  Image inpainting by correspondence maps: A deterministic approach , 2003 .

[7]  E. Shechtman,et al.  Transactions on Pattern Analysis and Machine Intelligence 1 Space-time Video Completion Draft Transactions on Pattern Analysis and Machine Intelligence 2 , 2022 .

[8]  Aurélie Bugeau,et al.  Combining Texture Synthesis and Diffusion for Image Inpainting , 2009, VISAPP.

[9]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[10]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[11]  Guy Gilboa,et al.  Nonlocal Linear Image Regularization and Supervised Segmentation , 2007, Multiscale Model. Simul..

[12]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[13]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2007, SIGGRAPH 2007.

[14]  Saïd Ladjal,et al.  Exemplar-Based Inpainting from a Variational Point of View , 2010, SIAM J. Math. Anal..

[15]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[16]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Assaf Zomet,et al.  Learning how to inpaint from global image statistics , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[18]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[19]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  Gabriel Peyré,et al.  Manifold models for signals and images , 2009, Comput. Vis. Image Underst..

[21]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[22]  Rachid Deriche,et al.  Vector-valued image regularization with PDEs: a common framework for different applications , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  David Tschumperlé,et al.  Fast Anisotropic Smoothing of Multi-Valued Images using Curvature-Preserving PDE's , 2006, International Journal of Computer Vision.

[24]  Folkmar Bornemann,et al.  Fast Image Inpainting Based on Coherence Transport , 2007, Journal of Mathematical Imaging and Vision.

[25]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[26]  Hossein Mobahi,et al.  Data-driven image completion by image patch subspaces , 2009, 2009 Picture Coding Symposium.

[27]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[28]  David Mumford,et al.  Filtering, Segmentation and Depth , 1993, Lecture Notes in Computer Science.

[29]  Nikos Komodakis,et al.  Image Completion Using Global Optimization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[30]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[31]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[32]  William T. Freeman,et al.  The patch transform and its applications to image editing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Marcelo Bertalmío,et al.  Strong-continuation, contrast-invariant inpainting with a third-order optimal PDE , 2006, IEEE Transactions on Image Processing.

[34]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.