THE LASSO UNDER POISSON-LIKE HETEROSCEDASTICITY

The performance of the Lasso is well understood under the assumptions of the standard sparse linear model with homoscedastic noise. However, in several ap- plications, the standard model does not describe the important features of the data. This paper examines how the Lasso performs on a non-standard model that is mo- tivated by medical imaging applications. In these applications, the variance of the noise scales linearly with the expectation of the observation. Like all heteroscedas- tic models, the noise terms in this Poisson-like model are not independent of the design matrix. Under a sparse Poisson-like model for the high-dimension regime that allows the number of predictors (p) ≫ sample size (n), we give necessary and sufficient conditions for the sign consistency of the Lasso estimate. Simulations re- veal that the Lasso performs equally well in terms of model selection performance on both Poisson-like data and homoscedastic data (with properly scaled noise vari- ance), across a range of parameterizations.

[1]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[2]  S. Chatterjee An error bound in the Sudakov-Fernique inequality , 2005, math/0510424.

[3]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[4]  L. Shepp,et al.  A Statistical Model for Positron Emission Tomography , 1985 .

[5]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[6]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[7]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[8]  S. Szarek,et al.  Chapter 8 - Local Operator Theory, Random Matrices and Banach Spaces , 2001 .

[9]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[10]  Martin J. Wainwright,et al.  Sharp thresholds for high-dimensional and noisy recovery of sparsity , 2006, ArXiv.

[11]  Peng Zhao,et al.  Stagewise Lasso , 2007, J. Mach. Learn. Res..

[12]  J. Fessler Statistical Image Reconstruction Methods for Transmission Tomography , 2000 .

[13]  H. Zou The Adaptive Lasso and Its Oracle Properties , 2006 .

[14]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[15]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[16]  Arkadi Nemirovski,et al.  On sparse representation in pairs of bases , 2003, IEEE Trans. Inf. Theory.

[17]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[18]  M. R. Osborne,et al.  On the LASSO and its Dual , 2000 .

[19]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[20]  Jianqing Fan,et al.  A Selective Overview of Variable Selection in High Dimensional Feature Space. , 2009, Statistica Sinica.

[21]  Jean-Jacques Fuchs,et al.  Recovery of exact sparse representations in the presence of bounded noise , 2005, IEEE Transactions on Information Theory.

[22]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[23]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[24]  Saharon Rosset,et al.  Tracking Curved Regularized Optimization Solution Paths , 2004, NIPS 2004.

[25]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[26]  David A. Freedman,et al.  Statistical Models: Theory and Practice: References , 2005 .

[27]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[28]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[29]  Michael I. Jordan,et al.  Union support recovery in high-dimensional multivariate regression , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[30]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.