Exemplar-based Image Inpainting and Applications

Image inpainting consists in recovering the missing or corrupted parts of an image so that the reconstructed image looks natural. The purpose of this talk will be to give an overview of recent techniques in non-local exemplar-based image inpainting and its applications in video and cinema post-production. Non-local methods for image denoising and inpainting have gained considerable attention in recent years. This is due to their superior performance in textured images, a known weakness of purely local methods. Local methods on the other hand have shown to be very appropriate for the recovering of geometric structure such as image edges. The synthesis of both types of methods is a trend in current research. Variational analysis in particular is an appropriate tool for a unified treatment of local and non-local methods. We present a general variational framework for the problem of non-local image inpainting, from which some previous inpainting schemes can be derived, in addition to leading to novel ones. We give an statistical mechanics interpretation of the proposed framework. We also study the properties of the variational formulation of the Efros-Leung copying scheme. We show applications of image inpainting to different problems: interpolation of sparsely sampled images, the replacement of objects in video sequences, and to the post-production of depth-enhanced imagery. AMS Classification: 68U10, 35A15, 65D05, 65C50.

[1]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[3]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[4]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[5]  Vicent Caselles,et al.  A Variational Model for Gradient-Based Video Editing , 2013, International Journal of Computer Vision.

[6]  Simon Masnou,et al.  Disocclusion: a variational approach using level lines , 2002, IEEE Trans. Image Process..

[7]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[8]  Naokazu Yokoya,et al.  Image Inpainting Considering Brightness Change and Spatial Locality of Textures , 2009, VISAPP.

[9]  Eli Shechtman,et al.  Space-Time Completion of Video , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Guillermo Sapiro,et al.  A Variational Framework for Non-local Image Inpainting , 2009, EMMCVPR.

[11]  Aurélie Bugeau,et al.  Temporally Consistent Gradient Domain Video Editing , 2011, EMMCVPR.

[12]  T. Chan,et al.  Image inpainting by correspondence maps: A deterministic approach , 2003 .

[13]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[14]  Gabriel Peyré,et al.  Manifold models for signals and images , 2009, Comput. Vis. Image Underst..

[15]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[16]  Guillermo Sapiro,et al.  Exemplar-Based Interpolation of Sparsely Sampled Images , 2009, EMMCVPR.

[17]  Tony F. Chan,et al.  Euler's Elastica and Curvature-Based Inpainting , 2003, SIAM J. Appl. Math..

[18]  Nikos Komodakis,et al.  Image Completion Using Efficient Belief Propagation Via Priority Scheduling and Dynamic Pruning , 2007, IEEE Transactions on Image Processing.

[19]  Pau Gargallo,et al.  Stereoscopic image inpainting using scene geometry , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[20]  Vicent Caselles,et al.  Analysis of a Variational Framework for Exemplar-Based Image Inpainting , 2012, Multiscale Model. Simul..

[21]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[22]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[23]  Guillermo Sapiro,et al.  A Variational Framework for Exemplar-Based Image Inpainting , 2011, International Journal of Computer Vision.

[24]  Guy Gilboa,et al.  Nonlocal Linear Image Regularization and Supervised Segmentation , 2007, Multiscale Model. Simul..

[25]  Saïd Ladjal,et al.  Exemplar-Based Inpainting from a Variational Point of View , 2010, SIAM J. Math. Anal..

[26]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[27]  Folkmar Bornemann,et al.  Fast Image Inpainting Based on Coherence Transport , 2007, Journal of Mathematical Imaging and Vision.

[28]  Jianhong Shen,et al.  EULER'S ELASTICA AND CURVATURE BASED INPAINTINGS , 2002 .