The L1L1 penalized LAD estimator for high dimensional linear regression

In this paper, the high-dimensional sparse linear regression model is considered, where the overall number of variables is larger than the number of observations. We investigate the L"1 penalized least absolute deviation method. Different from most of the other methods, the L"1 penalized LAD method does not need any knowledge of standard deviation of the noises or any moment assumptions of the noises. Our analysis shows that the method achieves near oracle performance, i.e. with large probability, the L"2 norm of the estimation error is of order O(klogp/n). The result is true for a wide range of noise distributions, even for the Cauchy distribution. Numerical results are also presented.

[1]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[2]  F. Vega-Redondo.  Complex Social Networks: Econometric Society Monographs , 2007 .

[3]  N. Meinshausen,et al.  LASSO-TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH-DIMENSIONAL DATA , 2008, 0806.0145.

[4]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[5]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[6]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[7]  V. Milman,et al.  New volume ratio properties for convex symmetric bodies in ℝn , 1987 .

[8]  Laurent Zwald,et al.  Robust regression through the Huber’s criterion and adaptive lasso penalty , 2011 .

[9]  M. Rudelson,et al.  Non-asymptotic theory of random matrices: extreme singular values , 2010, 1003.2990.

[10]  Marco Righero,et al.  An introduction to compressive sensing , 2009 .

[11]  Lie Wang,et al.  Shifting Inequality and Recovery of Sparse Signals , 2010, IEEE Transactions on Signal Processing.

[12]  Lie Wang,et al.  Orthogonal Matching Pursuit for Sparse Signal Recovery , 2010 .

[13]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[14]  Lie Wang,et al.  Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise , 2011, IEEE Transactions on Information Theory.

[15]  Tong Zhang,et al.  On the Consistency of Feature Selection using Greedy Least Squares Regression , 2009, J. Mach. Learn. Res..

[16]  R. Koenker,et al.  Asymptotic Theory of Least Absolute Error Regression , 1978 .

[17]  R. Koenker,et al.  The Gaussian hare and the Laplacian tortoise: computability of squared-error versus absolute-error estimators , 1997 .

[18]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[19]  Hansheng Wang,et al.  Robust Regression Shrinkage and Consistent Variable Selection Through the LAD-Lasso , 2007 .

[20]  A. Belloni,et al.  Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming , 2010, 1009.5689.

[21]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[22]  A. Belloni,et al.  L1-Penalized Quantile Regression in High Dimensional Sparse Models , 2009, 0904.2931.

[23]  Lie Wang,et al.  New Bounds for Restricted Isometry Constants , 2009, IEEE Transactions on Information Theory.

[24]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[25]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[26]  A. D. Slastnikov,et al.  Limit Theorems for Moderate Deviation Probabilities , 1979 .

[27]  Xiaohui Chen,et al.  Asymptotic Analysis of Robust LASSOs in the Presence of Noise With Large Variance , 2010, IEEE Transactions on Information Theory.