Combining anisotropic diffusion, transport equation and texture synthesis for inpainting textured images

In this work we propose a new image inpainting technique that combines texture synthesis, anisotropic diffusion, transport equation and a new sampling mechanism designed to alleviate the computational burden of the inpainting process. Given an image to be inpainted, anisotropic diffusion is initially applied to generate a cartoon image. A block-based inpainting approach is then applied so that to combine the cartoon image and a measure based on transport equation that dictates the priority on which pixels are filled. A sampling region is then defined dynamically so as to hold the propagation of the edges towards image structures while avoiding unnecessary searches during the completion process. Finally, a cartoon-based metric is computed to measure likeness between target and candidate blocks. Experimental results and comparisons against existing techniques attest the good performance and flexibility of our technique when dealing with real and synthetic images.

[1]  Zongben Xu,et al.  Image Inpainting by Patch Propagation Using Patch Sparsity , 2010, IEEE Transactions on Image Processing.

[2]  Lin He,et al.  Cahn--Hilliard Inpainting and a Generalization for Grayvalue Images , 2009, SIAM J. Imaging Sci..

[3]  Rachid Deriche,et al.  Vector-valued image regularization with PDEs: a common framework for different applications , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Maurílio Boaventura,et al.  A Decomposition and Noise Removal Method Combining Diffusion Equation and Wave Atoms for Textured Images , 2010 .

[5]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[6]  Paul Harrison,et al.  A Non-Hierarchical Procedure for Re-Synthesis of Complex Textures , 2001, WSCG.

[7]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[8]  Song Wang,et al.  Image inpainting based on scene transform and color transfer , 2010, Pattern Recognit. Lett..

[9]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.

[10]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[11]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[12]  Mohamed-Jalal Fadili,et al.  Inpainting and Zooming Using Sparse Representations , 2009, Comput. J..

[13]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[14]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[15]  Nikos Komodakis,et al.  Image Completion Using Efficient Belief Propagation Via Priority Scheduling and Dynamic Pruning , 2007, IEEE Transactions on Image Processing.

[16]  Guillermo Sapiro,et al.  A Comprehensive Framework for Image Inpainting , 2010, IEEE Transactions on Image Processing.

[17]  S. Mallat A wavelet tour of signal processing , 1998 .

[18]  Vicent Caselles,et al.  Exemplar-Based Image Inpainting Using Multiscale Graph Cuts , 2013, IEEE Transactions on Image Processing.

[19]  Xin Li,et al.  Image Recovery Via Hybrid Sparse Representations: A Deterministic Annealing Approach , 2011, IEEE Journal of Selected Topics in Signal Processing.

[20]  Saïd Ladjal,et al.  Exemplar-Based Inpainting from a Variational Point of View , 2010, SIAM J. Math. Anal..

[21]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part I: theory , 2006, IEEE Transactions on Image Processing.

[22]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[23]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[24]  M. Levoy,et al.  Texture synthesis by fixed neighborhood searching , 2001 .

[25]  Qiang Chen,et al.  Image Inpainting with Improved Exemplar-Based Approach , 2007, MCAM.

[26]  Charlie C. L. Wang,et al.  Interactive Image Inpainting Using DCT Based Exemplar Matching , 2009, ISVC.

[27]  Zhihui Wei,et al.  Edge-and-corner preserving regularization for image interpolation and reconstruction , 2008, Image Vis. Comput..

[28]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[29]  T. Chan,et al.  Image inpainting by correspondence maps: A deterministic approach , 2003 .

[30]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2007, SIGGRAPH 2007.

[31]  Fiorella Sgallari,et al.  Image inpainting with structural bootstrap priors , 2006, Image Vis. Comput..

[32]  L. Demanet,et al.  Wave atoms and sparsity of oscillatory patterns , 2007 .

[33]  Ming Zhao,et al.  Image inpainting with salient structure completion and texture propagation , 2011, Pattern Recognit. Lett..

[34]  Naokazu Yokoya,et al.  Image Inpainting Considering Brightness Change and Spatial Locality of Textures , 2009, VISAPP.

[35]  Jian-Feng Cai,et al.  A framelet-based image inpainting algorithm , 2008 .

[36]  Eli Shechtman,et al.  Space-Time Completion of Video , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  C. Barcelos,et al.  Edge detection and noise removal by use of a partial differential equation with automatic selection of parameters , 2005 .

[38]  Patrick Pérez,et al.  Geometrically Guided Exemplar-Based Inpainting , 2011, SIAM J. Imaging Sci..

[39]  Stanley Osher,et al.  Color Texture Modeling and Color Image Decomposition in a Variational-PDE Approach , 2006, 2006 Eighth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing.

[40]  Zhou Wang,et al.  On the Mathematical Properties of the Structural Similarity Index , 2012, IEEE Transactions on Image Processing.

[41]  Stanley Osher,et al.  Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing , 2003, J. Sci. Comput..

[42]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[43]  Yves Meyer,et al.  Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures , 2001 .

[44]  Maurílio Boaventura,et al.  A well-balanced flow equation for noise removal and edge detection , 2003, IEEE Trans. Image Process..

[45]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[46]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[47]  Ja-Ling Wu,et al.  Robust Algorithm for Exemplar-based Image Inpainting , 2005 .

[48]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[49]  Harald Grossauer,et al.  A Combined PDE and Texture Synthesis Approach to Inpainting , 2004, ECCV.

[50]  Aurélie Bugeau,et al.  Combining Texture Synthesis and Diffusion for Image Inpainting , 2009, VISAPP.

[51]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part II: adaptive algorithms , 2006, IEEE Transactions on Image Processing.

[52]  Folkmar Bornemann,et al.  Fast Image Inpainting Based on Coherence Transport , 2007, Journal of Mathematical Imaging and Vision.