Simultaneous cartoon and texture inpainting

Real images usually have two layers, namely, cartoons (the piecewise smooth part of the image) and textures (the oscillating pattern part of the image). Both these two layers have sparse approximations under some tight frame systems such as framelet, translation invariant wavelet, curvelet, and local DCTs. In this paper, we solve image inpainting problems by using two separate tight frame systems which can sparsely represent cartoons and textures respectively. Different from existing schemes in the literature which are either analysis-based or synthesis-based sparsity priors, our minimization formulation balances these two priors. We also derive iterative algorithms to find their solutions and prove their convergence. Numerical simulation examples are given to demonstrate the applicability and usefulness of our proposed algorithms in image inpainting.

[1]  P. Tseng.  Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[2]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[3]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[4]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[5]  D. Donoho,et al.  Translation-Invariant De-Noising , 1995 .

[6]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[7]  Zuowei Shen.  Affine systems in L 2 ( IR d ) : the analysis of the analysis operator , 1995 .

[8]  A. Ron,et al.  Affine Systems inL2(Rd): The Analysis of the Analysis Operator , 1997 .

[9]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[10]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[11]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[12]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .

[13]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[14]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[15]  Yves Meyer,et al.  Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures , 2001 .

[16]  Non-texture inpainting by curvature-driven diffusions (CDD) , 2001 .

[17]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[18]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[19]  S. Osher,et al.  IMAGE DECOMPOSITION AND RESTORATION USING TOTAL VARIATION MINIMIZATION AND THE H−1 NORM∗ , 2002 .

[20]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[21]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[22]  I. Daubechies,et al.  Framelets: MRA-based constructions of wavelet frames☆☆☆ , 2003 .

[23]  Stanley Osher,et al.  Image Decomposition and Restoration Using Total Variation Minimization and the H1 , 2003, Multiscale Model. Simul..

[24]  Tony F. Chan,et al.  Euler's Elastica and Curvature-Based Inpainting , 2003, SIAM J. Appl. Math..

[25]  R. Gribonval,et al.  Bi-framelet systems with few vanishing moments characterize Besov spaces , 2004 .

[26]  Ivan W. Selesnick,et al.  Pixel recovery via /spl lscr//sub 1/ minimization in the wavelet domain , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[27]  E. Candès,et al.  New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities , 2004 .

[28]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[29]  I. Daubechiesa,et al.  Variational image restoration by means of wavelets : Simultaneous decomposition , deblurring , and denoising , 2005 .

[30]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[31]  Michael Elad,et al.  Submitted to Ieee Transactions on Image Processing Image Decomposition via the Combination of Sparse Representations and a Variational Approach , 2022 .

[32]  R. Chan,et al.  A Framelet-Based Approach for Image Inpainting , 2005 .

[33]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[34]  Tony F. Chan,et al.  Image processing and analysis , 2005 .

[35]  Jean-Luc Starck,et al.  Sparse representations and bayesian image inpainting , 2005 .

[36]  Tony F. Chan,et al.  Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection , 2006, International Journal of Computer Vision.

[37]  D. Donoho.  For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[38]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[39]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part II: adaptive algorithms , 2006, IEEE Transactions on Image Processing.

[40]  Jian-Feng Cai,et al.  A framelet-based image inpainting algorithm , 2008 .

[41]  採編典藏組.  Society for Industrial and Applied Mathematics(SIAM) , 2008 .

[42]  Zuowei Shen,et al.  Dual Wavelet Frames and Riesz Bases in Sobolev Spaces , 2009 .

[43]  Jian-Feng Cai,et al.  Split Bregman Methods and Frame Based Image Restoration , 2009, Multiscale Model. Simul..