Fast and exact unidimensional L2-L1 optimization as an accelerator for iterative reconstruction algorithms

This paper studies the use of fast and exact unidimensional L2-L1 minimization as a line search for accelerating iterative reconstruction algorithms. In L2-L1 minimization reconstruction problems, the squared Euclidean, or L2 norm, measures signal-data discrepancy and the L1 norm stands for a sparsity preserving regularization term. Functionals as these arise in important applications such as compressed sensing and deconvolution. Optimal unidimensional L2-L1 minimization has only recently been studied by Li and Osher for denoising problems and by Wen et al. for line search. A fast L2-L1 optimization procedure can be adapted for line search and used in iterative algorithms, improving convergence speed with little increase in computational cost. This paper proposes a new method for exact L2-L1 line search and compares it with the Li and Osher's, Wen et al.'s, as well as with a standard line search algorithm, the method of false position. The use of the proposed line search improves convergence speed of different iterative algorithms for L2-L1 reconstruction such as iterative shrinkage, iteratively reweighted least squares, and nonlinear conjugate gradient. This assertion is validated experimentally in applications to signal reconstruction in compressed sensing and sparse signal deblurring.

[1]  Roger Fletcher,et al.  New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds , 2006, Math. Program..

[2]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.

[3]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[4]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[5]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[6]  Jeffrey A. Fessler,et al.  Model-Based Image Reconstruction for MRI , 2010, IEEE Signal Processing Magazine.

[7]  Michael Elad,et al.  L1-L2 Optimization in Signal and Image Processing , 2010, IEEE Signal Processing Magazine.

[8]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[9]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[10]  Jerry D. Gibson,et al.  Handbook of Image and Video Processing , 2000 .

[11]  Massimo Fornasier,et al.  Iteratively Re-weighted Least Squares minimization: Proof of faster than linear rate for sparse recovery , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[12]  S. Osher,et al.  A new median formula with applications to PDE based denoising , 2009 .

[13]  Yuying Li,et al.  A Globally Convergent Method for lp Problems , 1991, SIAM J. Optim..

[14]  Anthony G. Constantinides,et al.  A modified Armijo rule for the online selection of learning rate of the LMS algorithm , 2010, Digit. Signal Process..

[15]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[16]  Åke Björck,et al.  Numerical methods for least square problems , 1996 .

[17]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006 .

[18]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[19]  C. Vogel Computational Methods for Inverse Problems , 1987 .

[20]  J. Romberg,et al.  Imaging via Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[21]  R. Tyrrell Rockafellar,et al.  Subderivatives and Subgradients , 1998 .

[22]  Yu-Hong Dai,et al.  A Nonlinear Conjugate Gradient Algorithm with an Optimal Property and an Improved Wolfe Line Search , 2013, SIAM J. Optim..

[23]  Gonzalo R. Arce,et al.  A Maximum Likelihood Approach to Least Absolute Deviation Regression , 2004, EURASIP J. Adv. Signal Process..

[24]  Gonzalo R. Arce,et al.  Compressive Sensing Signal Reconstruction by Weighted Median Regression Estimates , 2011, IEEE Transactions on Signal Processing.

[25]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[26]  Kyle J. Myers,et al.  Foundations of Image Science , 2003, J. Electronic Imaging.

[27]  Yin Zhang,et al.  A Fast Algorithm for Sparse Reconstruction Based on Shrinkage, Subspace Optimization, and Continuation , 2010, SIAM J. Sci. Comput..

[28]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.