Reconstruction of Nonnegative Sparse Signals Using Accelerated Proximal-Gradient Algorithms

We develop an accelerated proximal-gradient scheme for reconstructing nonnegative signals that are sparse in a transform domain from underdetermined measurements. This signal model is motivated by tomographic applications where the signal of interest is known to be nonnegative because it represents a tissue or material density. It is also applicable to optical and hyperspectral imaging, where energy within certain spectral band is nonnegative. We adopt the unconstrained regularization framework where the objective function to be minimized is a sum of a convex data fidelity (negative log-likelihood (NLL)) term and a convex regularization term that imposes signal nonnegativity and sparsity by using indicator-function and l1-norm constraints on the signal and its transform coefficients, respectively. We apply the Nesterov's proximal-gradient (NPG) method with function restart to minimize this objective function and the alternating direction method of multipliers (ADMM) to compute the proximal mapping. To accelerate convergence of the NPG iteration, we apply a step-size selection scheme that accounts for varying local Lipschitz constant of the NLL. We also apply adaptive continuation, which provides numerical stability and can accelerate the convergence of the NPG iteration. We construct compressed-sensing and tomographic reconstruction experiments with Gaussian linear and Poisson generalized linear measurement models, where we compare the proposed reconstruction approach with existing signal reconstruction methods. By exploiting both the nonnegativity of the underlying signal and sparsity of its wavelet coefficients, we can achieve significantly better reconstruction performance than the existing methods.

[1]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[2]  H Stark,et al.  Image restoration by convex projections in the presence of noise. , 1983, Applied optics.

[3]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[4]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[5]  Mohamed-Jalal Fadili,et al.  A Generalized Forward-Backward Splitting , 2011, SIAM J. Imaging Sci..

[6]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Federica Porta,et al.  Variable Metric Inexact Line-Search-Based Methods for Nonsmooth Optimization , 2015, SIAM J. Optim..

[8]  Richard G. Baraniuk,et al.  Sparsity and Structure in Hyperspectral Imaging : Sensing, Reconstruction, and Target Detection , 2014, IEEE Signal Processing Magazine.

[9]  J. M. Ollinger,et al.  Positron Emission Tomography , 2018, Handbook of Small Animal Imaging.

[10]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[11]  D. L. Donoho,et al.  Compressed sensing , 2006, IEEE Trans. Inf. Theory.

[12]  Luca Baldassarre,et al.  Accelerated and Inexact Forward-Backward Algorithms , 2013, SIAM J. Optim..

[13]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[14]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[15]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[16]  Alexandros G. Dimakis,et al.  Sparse Recovery of Nonnegative Signals With Minimal Expansion , 2011, IEEE Transactions on Signal Processing.

[17]  R. Tibshirani,et al.  PATHWISE COORDINATE OPTIMIZATION , 2007, 0708.1485.

[18]  José M. Bioucas-Dias,et al.  Algorithms for imaging inverse problems under sparsity regularization , 2012, 2012 3rd International Workshop on Cognitive Information Processing (CIP).

[19]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[20]  Dimitri P. Bertsekas,et al.  Convex Analysis and Optimization , 2003 .

[21]  Rebecca Willett,et al.  This is SPIRAL-TAP: Sparse Poisson Intensity Reconstruction ALgorithms—Theory and Practice , 2010, IEEE Transactions on Image Processing.

[22]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[23]  Jean-Luc Starck,et al.  Astronomical image and data analysis , 2002 .

[24]  Avinash C. Kak,et al.  Principles of computerized tomographic imaging , 2001, Classics in applied mathematics.

[25]  Hakan Erdogan,et al.  Ordered subsets algorithms for transmission tomography. , 1999, Physics in medicine and biology.

[26]  L. Shepp,et al.  Maximum Likelihood Reconstruction for Emission Tomography , 1983, IEEE Transactions on Medical Imaging.

[27]  D. Harville Matrix Algebra From a Statistician's Perspective , 1998 .

[28]  Yin Zhang,et al.  A Fast Algorithm for Sparse Reconstruction Based on Shrinkage, Subspace Optimization, and Continuation , 2010, SIAM J. Sci. Comput..

[29]  Mohamed-Jalal Fadili,et al.  Convergence rates with inexact non-expansive operators , 2014, Mathematical Programming.

[30]  R. White,et al.  Image recovery from data acquired with a charge-coupled-device camera. , 1993, Journal of the Optical Society of America. A, Optics and image science.

[31]  Yonina C. Eldar,et al.  Smoothing and Decomposition for Analysis Sparse Recovery , 2013, IEEE Transactions on Signal Processing.

[32]  J. Marsden,et al.  Elementary classical analysis , 1974 .

[33]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[34]  Valeria Ruggiero,et al.  Numerical Methods for Parameter Estimation in Poisson Data Inversion , 2014, Journal of Mathematical Imaging and Vision.

[35]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[36]  David L. Donoho,et al.  Precise Undersampling Theorems , 2010, Proceedings of the IEEE.

[37]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[38]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[39]  Mark W. Schmidt,et al.  Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization , 2011, NIPS.

[40]  Justin K. Romberg,et al.  Compressive Sensing by Random Convolution , 2009, SIAM J. Imaging Sci..

[41]  Concha Bielza,et al.  A Survey of L1 Regression , 2013 .

[42]  Damek Davis,et al.  Convergence Rate Analysis of Primal-Dual Splitting Schemes , 2014, SIAM J. Optim..

[43]  Rebecca Willett,et al.  Gradient projection for linearly constrained convex optimization in sparse signal recovery , 2010, 2010 IEEE International Conference on Image Processing.

[44]  A. Robert Calderbank,et al.  Performance Bounds for Expander-Based Compressed Sensing in Poisson Noise , 2010, IEEE Transactions on Signal Processing.

[45]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[46]  Aleksandar Dogandzic,et al.  A fast proximal gradient algorithm for reconstructing nonnegative signals with sparse transform coefficients , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[47]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[48]  Jean-François Aujol,et al.  Stability of Over-Relaxations for the Forward-Backward Algorithm, Application to FISTA , 2015, SIAM J. Optim..

[49]  A. Chambolle,et al.  On the Convergence of the Iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm” , 2015, J. Optim. Theory Appl..

[50]  D. Youla,et al.  Image Restoration by the Method of Convex Projections: Part 1ߞTheory , 1982, IEEE Transactions on Medical Imaging.

[51]  Aleksandar Dogandzic,et al.  Mask iterative hard thresholding algorithms for sparse image reconstruction of objects with known contour , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[52]  Emmanuel J. Candès,et al.  Adaptive Restart for Accelerated Gradient Schemes , 2012, Foundations of Computational Mathematics.

[53]  John A. Nelder,et al.  Generalized linear models. 2nd ed. , 1993 .

[54]  Aleksandar Dogandžić,et al.  Polychromatic sparse image reconstruction and mass attenuation spectrum estimation via B-spline basis function expansion , 2015 .

[55]  Marc Teboulle,et al.  Interior Gradient and Proximal Methods for Convex and Conic Optimization , 2006, SIAM J. Optim..

[56]  Michael Unser,et al.  Poisson Image Reconstruction With Hessian Schatten-Norm Regularization , 2013, IEEE Transactions on Image Processing.

[57]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[58]  Volkan Cevher,et al.  Composite self-concordant minimization , 2013, J. Mach. Learn. Res..

[59]  Aleksandar Dogandzic,et al.  Beam hardening correction via mass attenuation discretization , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[60]  Jiang Hsieh,et al.  Computed Tomography: Principles, Design, Artifacts, and Recent Advances, Fourth Edition , 2022 .

[61]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[62]  Valeria Ruggiero,et al.  A Variable Metric Forward-Backward Method with Extrapolation , 2015, SIAM J. Sci. Comput..

[63]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[64]  Aleksandar Dogandzic,et al.  Nonnegative signal reconstruction from compressive samples via a difference map ECME algorithm , 2011, 2011 IEEE Statistical Signal Processing Workshop (SSP).

[65]  Trevor C. Bailey,et al.  Statistical Methods in Spatial Epidemiology, 2nd edition by A. B. LAWSON , 2007 .