Elastic-net regularization: error estimates and active set methods

This paper investigates theoretical properties and efficient numerical algorithms for the so-called elastic-net regularization originating from statistics, which enforces simultaneously ℓ1 and ℓ2 regularization. The stability of the minimizer and its consistency are studied, and convergence rates for both a priori and a posteriori parameter choice rules are established. Two iterative numerical algorithms of active set type are proposed, and their convergence properties are discussed. Numerical results are presented to illustrate the features of the functional and algorithms.

[1]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[2]  Lorenzo Rosasco,et al.  Elastic-net regularization in learning theory , 2008, J. Complex..

[3]  O. Scherzer,et al.  Sparse regularization with lq penalty term , 2008, 0806.3222.

[4]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[5]  Dirk A. Lorenz,et al.  Iterated Hard Shrinkage for Minimization Problems with Sparsity Constraints , 2008, SIAM J. Sci. Comput..

[6]  D. Lorenz,et al.  Convergence rates and source conditions for Tikhonov regularization with sparsity constraints , 2008, 0801.1774.

[7]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[8]  Per Christian Hansen,et al.  Regularization Tools version 4.0 for Matlab 7.3 , 2007, Numerical Algorithms.

[9]  I. Loris.  On the performance of algorithms for the minimization of ℓ1-penalized functionals , 2007, 0710.4082.

[10]  D. Lorenz,et al.  A semismooth Newton method for Tikhonov functionals with sparsity constraints , 2007, 0709.3186.

[11]  K. Bredies,et al.  Linear Convergence of Iterative Soft-Thresholding , 2007, 0709.1598.

[12]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[13]  A. Ramm.  A symmetry problem , 2004, math/0411175.

[14]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[15]  S. Osher,et al.  Convergence rates of convex variational regularization , 2004 .

[16]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[17]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[18]  Michael Ulbrich,et al.  Semismooth Newton Methods for Operator Equations in Function Spaces , 2002, SIAM J. Optim..

[19]  Kazufumi Ito,et al.  The Primal-Dual Active Set Strategy as a Semismooth Newton Method , 2002, SIAM J. Optim..

[20]  Xiaojun Chen,et al.  Smoothing Methods and Semismooth Methods for Nondifferentiable Operator Equations , 2000, SIAM J. Numer. Anal..

[21]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[22]  Lea Fleischer,et al.  Regularization of Inverse Problems , 1996 .

[23]  Kazufumi Ito,et al.  On the Choice of the Regularization Parameter in Nonlinear Inverse Problems , 1992, SIAM J. Optim..

[24]  S. Levy,et al.  Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution , 1981 .

[25]  Thomas Bonesky.  Morozov's discrepancy principle and Tikhonov-type functionals , 2008 .

[26]  O. Scherzer,et al.  FA ] 2 9 Ju l 2 00 8 Sparse Regularization with l q Penalty Term , 2008 .

[27]  Ivan P. Gavrilyuk,et al.  Variational analysis in Sobolev and BV spaces , 2007, Math. Comput..

[28]  S. Osher,et al.  Convergence rates of convex variational regularization , 2004 .

[29]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[30]  H. L. Taylor,et al.  Deconvolution with the l 1 norm , 1979 .