Rotation Invariance in Exemplar-based Image Inpainting

Due to their good performance on textured images, exemplar-based methods for image recovery have been subject of research in recent years. In this work, the variational framework of exemplar-based inpainting is reviewed and enriched by rotation invariance as an additional degree of freedom in patch searching. For an efficient rotation invariant comparison of image patches we present methods based on sampled Circular Harmonics (CH) expansions, in particular, we also elaborate a method for gradient-based comparisons thanks to the property of CH of being eigenfunctions of the Laplacian. These new pattern matching methods allow for an extremely efficient implementation of the alternating optimization scheme of exemplar-based inpainting, also when rotation invariant patch matching is considered. The patch non-local means algorithm and its performance in the recovery of image structures and textures are described in detail and we demonstrate by numerical examples the significant improvement in recovering smooth edges, which is due to the additional rotation invariance.

[1]  F. Smithies,et al.  Convex Functions and Orlicz Spaces , 1962, The Mathematical Gazette.

[2]  M. Fornasier.  Decompositions of Hilbert spaces: local construction of global frames , 2002 .

[3]  Suyash P. Awate,et al.  Higher-order image statistics for unsupervised, information-theoretic, adaptive, image filtering , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  H. Poppe,et al.  Compactness in general function spaces , 1974 .

[5]  Guillermo Sapiro,et al.  A Variational Framework for Exemplar-Based Image Inpainting , 2011, International Journal of Computer Vision.

[6]  T. Chan,et al.  Image inpainting by correspondence maps: A deterministic approach , 2003 .

[7]  Naokazu Yokoya,et al.  Image Inpainting Considering Brightness Change and Spatial Locality of Textures , 2009, VISAPP.

[8]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[9]  Charles Kervrann,et al.  Local Adaptivity to Variable Smoothness for Exemplar-Based Image Regularization and Representation , 2008, International Journal of Computer Vision.

[10]  Eli Shechtman,et al.  Space-Time Completion of Video , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[12]  Massimo Fornasier,et al.  Inpainting of Ancient Austrian frescoes , 2008 .

[13]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[14]  Massimo Fornasier,et al.  Fast, robust and efficient 2D pattern recognition for re-assembling fragmented images , 2005, Pattern Recognit..

[15]  Zhi Qi,et al.  Theory of Bessel Functions of High Rank , 2015 .

[16]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[17]  Kurt Bernardo Wolf,et al.  Integral transforms in science and engineering , 1979 .

[18]  Rachid Deriche,et al.  Vector-valued image regularization with PDE's: a common framework for different applications , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[19]  L. Ambrosio,et al.  Functions of Bounded Variation and Free Discontinuity Problems , 2000 .

[20]  Arun N. Netravali,et al.  Reconstruction filters in computer-graphics , 1988, SIGGRAPH.

[21]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[22]  Lucas Pereira,et al.  Image replacement through texture synthesis , 1997, Proceedings of International Conference on Image Processing.

[23]  Saïd Ladjal,et al.  Exemplar-Based Inpainting from a Variational Point of View , 2010, SIAM J. Math. Anal..

[24]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[25]  Kim C. Border,et al.  Infinite Dimensional Analysis: A Hitchhiker’s Guide , 1994 .

[26]  H H Arsenault,et al.  Properties of the circular harmonic expansion for rotation-invariant pattern recognition. , 1986, Applied optics.

[27]  Pierre Kornprobst,et al.  Mathematical problems in image processing - partial differential equations and the calculus of variations , 2010, Applied mathematical sciences.

[28]  Z. W. Birnbaum,et al.  Über die Verallgemeinerung des Begriffes der zueinander konjugierten Potenzen , 1931 .

[29]  P. Bickel,et al.  Texture synthesis and nonparametric resampling of random fields , 2006, math/0611258.

[30]  P. Bassanini,et al.  Elliptic Partial Differential Equations of Second Order , 1997 .

[31]  Folkmar Bornemann,et al.  Fast Image Inpainting Based on Coherence Transport , 2007, Journal of Mathematical Imaging and Vision.

[32]  Massimo Fornasier,et al.  Function Spaces Inclusions and Rate of Convergence of Riemann-Type Sums in Numerical Integration , 2003 .

[33]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[34]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[35]  J. Morel,et al.  An axiomatic approach to image interpolation. , 1998, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[36]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[37]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[38]  T. Brox,et al.  Nonlocal texture filtering with efficient tree structures and invariant patch similarity measures , 2008 .

[39]  Jacques Blanc-Talon,et al.  Simultaneous Inpainting and Motion Estimation of Highly Degraded Video-Sequences , 2003, SCIA.

[40]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[41]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[42]  Li Zhang,et al.  X-ray CT metal artifacts reduction through curvature based sinogram inpainting , 2006 .

[43]  Gilberte Émile-Mâle.  The restorer's handbook of easel painting , 1976 .

[44]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[45]  Anil C. Kokaram,et al.  Interpolation of missing data in image sequences , 1995, IEEE Trans. Image Process..