Cartoon-Texture Image Decomposition Using Blockwise Low-Rank Texture Characterization

Using a novel characterization of texture, we propose an image decomposition technique that can effectively decomposes an image into its cartoon and texture components. The characterization rests on our observation that the texture component enjoys a blockwise low-rank nature with possible overlap and shear, because texture, in general, is globally dissimilar but locally well patterned. More specifically, one can observe that any local block of the texture component consists of only a few individual patterns. Based on this premise, we first introduce a new convex prior, named the block nuclear norm (BNN), leading to a suitable characterization of the texture component. We then formulate a cartoon-texture decomposition model as a convex optimization problem, where the simultaneous estimation of the cartoon and texture components from a given image or degraded observation is executed by minimizing the total variation and BNN. In addition, patterns of texture extending in different directions are extracted separately, which is a special feature of the proposed model and of benefit to texture analysis and other applications. Furthermore, the model can handle various types of degradation occurring in image processing, including blur+missing pixels with several types of noise. By rewriting the problem via variable splitting, the so-called alternating direction method of multipliers becomes applicable, resulting in an efficient algorithmic solution to the problem. Numerical examples illustrate that the proposed model is very selective to patterns of texture, which makes it produce better results than state-of-the-art decomposition models.

[1]  J. Moreau Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[2]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[3]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[4]  M. Fortin,et al.  Augmented Lagrangian methods : applications to the numerical solution of boundary-value problems , 1983 .

[5]  電子情報通信学会 IEICE transactions on fundamentals of electronics, communications and computer sciences , 1992 .

[6]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[7]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[8]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[9]  Yves Meyer,et al.  Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures , 2001 .

[10]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[11]  S. Osher,et al.  IMAGE DECOMPOSITION AND RESTORATION USING TOTAL VARIATION MINIMIZATION AND THE H−1 NORM∗ , 2002 .

[12]  Stanley Osher,et al.  Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing , 2003, J. Sci. Comput..

[13]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[14]  Stanley Osher,et al.  Image Decomposition and Restoration Using Total Variation Minimization and the H1 , 2003, Multiscale Model. Simul..

[15]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[16]  Antonin Chambolle,et al.  Dual Norms and Image Decomposition Models , 2005, International Journal of Computer Vision.

[17]  I. Daubechiesa,et al.  Variational image restoration by means of wavelets : Simultaneous decomposition , deblurring , and denoising , 2005 .

[18]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[19]  Michael Elad,et al.  Submitted to Ieee Transactions on Image Processing Image Decomposition via the Combination of Sparse Representations and a Variational Approach , 2022 .

[20]  Antonin Chambolle,et al.  Image Decomposition into a Bounded Variation Component and an Oscillating Component , 2005, Journal of Mathematical Imaging and Vision.

[21]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[22]  Tony F. Chan,et al.  Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection , 2006, International Journal of Computer Vision.

[23]  Guy Gilboa,et al.  Nonlocal Linear Image Regularization and Supervised Segmentation , 2007, Multiscale Model. Simul..

[24]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[25]  J.-C. Pesquet,et al.  A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery , 2007, IEEE Journal of Selected Topics in Signal Processing.

[26]  P. L. Combettes,et al.  A proximal decomposition method for solving convex variational inverse problems , 2008, 0807.2617.

[27]  T. Chan,et al.  Fast dual minimization of the vectorial total variation norm and applications to color image processing , 2008 .

[28]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[29]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[30]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[31]  John Wright,et al.  RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[33]  Xavier Bresson,et al.  Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction , 2010, SIAM J. Imaging Sci..

[34]  Karl Kunisch,et al.  Total Generalized Variation , 2010, SIAM J. Imaging Sci..

[35]  Jean-François Aujol,et al.  Mathematical Modeling of Textures: Application to Color Image Decomposition with a Projected Gradient Algorithm , 2010, Journal of Mathematical Imaging and Vision.

[36]  Mohamed-Jalal Fadili,et al.  Image Decomposition and Separation Using Sparse Representations: An Overview , 2010, Proceedings of the IEEE.

[37]  Silvia Gandy,et al.  Convex optimization techniques for the efficient recovery of a sparsely corrupted low-rank matrix , 2010 .

[38]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[39]  Simon Setzer,et al.  Operator Splittings, Bregman Methods and Frame Shrinkage in Image Processing , 2011, International Journal of Computer Vision.

[40]  Yves Meyer,et al.  Properties of $BV-G$ Structures $+$ Textures Decomposition Models. Application to Road Detection in Satellite Images , 2010, IEEE Transactions on Image Processing.

[41]  Jian-Feng Cai,et al.  Simultaneous cartoon and texture inpainting , 2010 .

[42]  Patrick L. Combettes,et al.  Proximal Algorithms for Multicomponent Image Recovery Problems , 2011, Journal of Mathematical Imaging and Vision.

[43]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[44]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[45]  Gabriel Peyré,et al.  Locally Parallel Texture Modeling , 2011, SIAM J. Imaging Sci..

[46]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[47]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[48]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[49]  S. Setzer,et al.  Infimal convolution regularizations with discrete ℓ1-type functionals , 2011 .

[50]  Sergios Theodoridis,et al.  Online Sparse System Identification and Signal Reconstruction Using Projections Onto Weighted $\ell_{1}$ Balls , 2010, IEEE Transactions on Signal Processing.

[51]  Yi Ma,et al.  TILT: Transform Invariant Low-Rank Textures , 2010, ACCV 2010.

[52]  Shunsuke Ono,et al.  A hierarchical convex optimization approach for high fidelity solution selection in image recovery , 2012, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference.

[53]  Shunsuke Ono,et al.  Missing region recovery by promoting blockwise low-rankness , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[54]  Nelly Pustelnik,et al.  A proximal approach for constrained cosparse modelling , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[55]  Hai Yen Le,et al.  Convexifying the set of matrices of bounded rank: applications to the quasiconvexification and convexification of the rank function , 2012, Optim. Lett..

[56]  Shunsuke Ono,et al.  Image Recovery by Decomposition with Component-Wise Regularization , 2012, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[57]  I. Yamada,et al.  Three variants of alternating direction method of multipliers without certain inner iterations and their application to image super-resolution via sparse representation , 2012 .

[58]  Shunsuke Ono,et al.  A sparse system identification by using adaptively-weighted total variation via a primal-dual splitting approach , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[59]  Stanley Osher,et al.  A Low Patch-Rank Interpretation of Texture , 2013, SIAM J. Imaging Sci..

[60]  Michael K. Ng,et al.  Coupled Variational Image Decomposition and Restoration Model for Blurred Cartoon-Plus-Texture Images With Missing Pixels , 2013, IEEE Transactions on Image Processing.

[61]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[62]  Shunsuke Ono,et al.  Poisson image restoration with likelihood constraint via hybrid steepest descent method , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[63]  S. Osher,et al.  Seismic data reconstruction via matrix completion , 2013 .

[64]  Shunsuke Ono,et al.  A Convex Regularizer for Reducing Color Artifact in Color Image Recovery , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Shunsuke Ono,et al.  Blockwise Low-Rank Prior for Cartoon-Texture Image Decomposition (信号処理) , 2013 .

[66]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.