New explicit thresholding/shrinkage formulas for one class of regularization problems with overlapping group sparsity and their applications

The least-square regression problems or inverse problems have been widely studied in many fields such as compressive sensing, signal processing, and image processing. To solve this kind of ill-posed problems, a regularization term (i.e., regularizer) should be introduced, under the assumption that the solutions have some specific properties, such as sparsity and group sparsity. Widely used regularizers include the $\ell_1$ norm, total variation (TV) semi-norm, and so on. Recently, a new regularization term with overlapping group sparsity has been considered. Majorization minimization iteration method or variable duplication methods are often applied to solve them. However, there have been no direct methods for solve the relevant problems because of the difficulty of overlapping. In this paper, we proposed new explicit shrinkage formulas for one class of these relevant problems, whose regularization terms have translation invariant overlapping groups. Moreover, we apply our results in TV deblurring and denoising with overlapping group sparsity. We use alternating direction method of multipliers to iterate solve it. Numerical results also verify the validity and effectiveness of our new explicit shrinkage formulas.

[1]  Xavier Bresson,et al.  Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction , 2010, SIAM J. Imaging Sci..

[2]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[3]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[4]  Junzhou Huang,et al.  The Benefit of Group Sparsity , 2009 .

[5]  T. Hastie,et al.  SparseNet: Coordinate Descent With Nonconvex Penalties , 2011, Journal of the American Statistical Association.

[6]  Raymond H. Chan,et al.  Constrained Total Variation Deblurring Models and Fast Algorithms Based on Alternating Direction Method of Multipliers , 2013, SIAM J. Imaging Sci..

[7]  Junfeng Yang,et al.  A Fast Alternating Direction Method for TVL1-L2 Signal Reconstruction From Partial Fourier Data , 2010, IEEE Journal of Selected Topics in Signal Processing.

[8]  Junfeng Yang,et al.  A Fast Algorithm for Edge-Preserving Variational Multichannel Image Restoration , 2009, SIAM J. Imaging Sci..

[9]  René Vidal,et al.  Robust classification using structured sparse representation , 2011, CVPR 2011.

[10]  Rabab Kreidieh Ward,et al.  Classification via group sparsity promoting regularization , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  Zongben Xu,et al.  Fast image deconvolution using closed-form thresholding formulas of regularization , 2013, J. Vis. Commun. Image Represent..

[12]  Ting-Zhu Huang,et al.  Image restoration using total variation with overlapping group sparsity , 2013, Inf. Sci..

[13]  M. Kowalski Sparse regression using mixed norms , 2009 .

[14]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[15]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[16]  Francis R. Bach,et al.  Structured Variable Selection with Sparsity-Inducing Norms , 2009, J. Mach. Learn. Res..

[17]  Ivan W. Selesnick,et al.  Total variation denoising with overlapping group sparsity , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  Wotao Yin,et al.  Group sparse optimization by alternating direction method , 2013, Optics & Photonics - Optical Engineering + Applications.

[19]  James Theiler,et al.  Local principal component pursuit for nonlinear datasets , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[21]  Xue-Cheng Tai,et al.  Augmented Lagrangian Method, Dual Methods, and Split Bregman Iteration for ROF, Vectorial TV, and High Order Models , 2010, SIAM J. Imaging Sci..

[22]  Ivan W. Selesnick,et al.  Translation-invariant shrinkage/thresholding of group sparse signals , 2013, Signal Process..

[23]  Jean Charles Gilbert,et al.  Numerical Optimization: Theoretical and Practical Aspects , 2003 .

[24]  Ilker Bayram,et al.  Mixed norms with overlapping groups as signal priors , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  R. Glowinski,et al.  Numerical Methods for Nonlinear Variational Problems , 1985 .

[26]  Bingsheng He,et al.  Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities , 1998, Oper. Res. Lett..

[27]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[28]  Babak Hassibi,et al.  On the reconstruction of block-sparse signals with an optimal number of measurements , 2009, IEEE Trans. Signal Process..

[29]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[30]  Junbin Gao,et al.  Dimensionality reduction via compressive sensing , 2012, Pattern Recognit. Lett..

[31]  Yonina C. Eldar,et al.  C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework , 2010, IEEE Transactions on Signal Processing.

[32]  Yueting Zhuang,et al.  Group sparse representation for image categorization and semantic video retrieval , 2011, Science China Information Sciences.

[33]  Mohamed-Jalal Fadili,et al.  Group sparsity with overlapping partition functions , 2011, 2011 19th European Signal Processing Conference.

[34]  Dianne P. O'Leary,et al.  Deblurring Images: Matrices, Spectra and Filtering , 2006, J. Electronic Imaging.

[35]  R. Glowinski Lectures on Numerical Methods for Non-Linear Variational Problems , 1981 .

[36]  Brendt Wohlberg,et al.  A nonconvex ADMM algorithm for group sparsity with sparse groups , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[37]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[38]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[39]  Ivan W. Selesnick,et al.  Translation-invariant shrinkage/thresholding of group sparse signals , 2014, Signal Process..

[40]  Julien Mairal,et al.  Structured sparsity through convex optimization , 2011, ArXiv.

[41]  Gang Liu,et al.  Total Variation with Overlapping Group Sparsity for Image Deblurring under Impulse Noise , 2013, PloS one.

[42]  M. Nikolova An Algorithm for Total Variation Minimization and Applications , 2004 .

[43]  Jean-Philippe Vert,et al.  Group lasso with overlap and graph lasso , 2009, ICML '09.