Sparse recovery under matrix uncertainty

We consider the model y - Y0 +, Z- X + 3? where the random vector y ∈ ℝ n and the random n × p matrix Z are observed, the n x p matrix X is unknown, Θ is an n x p random noise matrix, ξ ∈ ℝ n is a noise independent of Θ, and θ * is a vector of unknown parameters to be estimated. The matrix uncertainty is in the fact that X is observed with additive error. For dimensions p that can be much larger than the sample size n, we consider the estimation of sparse vectors θ * . Under matrix uncertainty, the Lasso and Dantzig selector turn out to be extremely unstable in recovering the sparsity pattern (i.e., of the set of nonzero components of θ * ), even if the noise level is very small. We suggest new estimators called matrix uncertainty selectors (or, shortly, the MU-selectors) which are close to θ * in different norms and in the prediction risk if the restricted eigenvalue assumption on X is satisfied. We also show that under somewhat stronger assumptions, these estimators recover correctly the sparsity pattern.

[1]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[2]  Vladimir Koltchinskii,et al.  On inverse problems with unknown operators , 2001, IEEE Trans. Inf. Theory.

[3]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[4]  N. Hengartner,et al.  Adaptive estimation for inverse problems with noisy operators , 2005 .

[5]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[6]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[7]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[8]  H. Zou The Adaptive Lasso and Its Oracle Properties , 2006 .

[9]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[10]  A. Tsybakov,et al.  Aggregation for Gaussian regression , 2007, 0710.3654.

[11]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[12]  Marc Raimondo,et al.  Wavelet Deconvolution With Noisy Eigenvalues , 2007, IEEE Transactions on Signal Processing.

[13]  C. Marteau REGULARIZATION OF INVERSE PROBLEMS WITH UNKNOWN OPERATOR , 2007 .

[14]  A. Tsybakov,et al.  Sparsity oracle inequalities for the Lasso , 2007, 0705.3308.

[15]  S. Geer HIGH-DIMENSIONAL GENERALIZED LINEAR MODELS AND THE LASSO , 2008, 0804.0703.

[16]  Karim Lounici Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators , 2008, 0801.4610.

[17]  Karim Lounici,et al.  High-dimensional stochastic optimization with the generalized Dantzig estimator , 2008, 0811.2281.

[18]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[19]  Cun-Hui Zhang,et al.  The sparsity and bias of the Lasso selection in high-dimensional linear regression , 2008, 0808.0967.

[20]  Arnak S. Dalalyan,et al.  Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity , 2008, Machine Learning.

[21]  Marc Hoffmann,et al.  Nonlinear estimation for linear inverse problems with error in the operator , 2008, 0803.1956.

[22]  V. Koltchinskii The Dantzig selector and sparsity oracle inequalities , 2009, 0909.0861.

[23]  N. Meinshausen,et al.  LASSO-TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH-DIMENSIONAL DATA , 2008, 0806.0145.

[24]  I. Daubechies,et al.  Sparse and stable Markowitz portfolios , 2007, Proceedings of the National Academy of Sciences.

[25]  Tong Zhang Some sharp performance bounds for least squares regression with L1 regularization , 2009, 0908.2869.

[26]  Arnak S. Dalalyan,et al.  Sparse Regression Learning by Aggregation and Langevin Monte-Carlo , 2009, COLT.

[27]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[28]  Peter J. Bickel,et al.  Hierarchical selection of variables in sparse high-dimensional regression , 2008, 0801.1158.

[29]  V. Koltchinskii,et al.  Oracle inequalities in empirical risk minimization and sparse recovery problems , 2011 .

[30]  Alexander Kukush,et al.  Measurement Error Models , 2011, International Encyclopedia of Statistical Science.