Sparsity and the Possibility of Inference

We discuss the importance of sparsity in the context of nonparametric regression and covariance matrix estimation. We point to low manifold dimension of the covariate vector as a possible important feature of sparsity, recall an estimate of dimension due to Levina and Bickel (2005) and establish some conjectures made in that paper.

[1]  G. Box.  Robustness in the Strategy of Scientific Model Building. , 1979 .

[2]  A. Atkinson.  A note on the generalized information criterion for choice of a model , 1980 .

[3]  J. Friedman,et al.  Estimating Optimal Transformations for Multiple Regression and Correlation. , 1985 .

[4]  Kenneth Falconer,et al.  Fractal Geometry: Mathematical Foundations and Applications , 1990 .

[5]  B. M. Fulk.  MATH , 1992 .

[6]  I. Johnstone,et al.  Minimax risk overlp-balls forlp-error , 1994 .

[7]  I. Johnstone,et al.  Minimax Risk over l p-Balls for l q-error , 1994 .

[8]  I. Johnstone,et al.  Ideal denoising in an orthonormal basis chosen from a library of bases , 1994 .

[9]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[10]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[11]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Yuhong Yang.  Can the Strengths of AIC and BIC Be Shared , 2005 .

[13]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[14]  E. Candès,et al.  New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities , 2004 .

[15]  Peter J. Bickel,et al.  Maximum Likelihood Estimation of Intrinsic Dimension , 2004, NIPS.

[16]  P. Bickel,et al.  Some theory for Fisher''s linear discriminant function , 2004 .

[17]  I. Johnstone,et al.  Adapting to unknown sparsity by controlling the false discovery rate , 2005, math/0505374.

[18]  S. Chatterjee.  A NEW METHOD OF NORMAL APPROXIMATION , 2006, math/0611213.

[19]  Jianqing Fan,et al.  Sure independence screening for ultrahigh dimensional feature space , 2006, math/0612857.

[20]  Noureddine El Karoui.  Spectrum estimation for large dimensional covariance matrices using random matrix theory , 2006, math/0609418.

[21]  S. Geer,et al.  Regularization in statistics , 2006 .

[22]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, SIAM Rev..

[23]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[24]  P. Bickel,et al.  Local polynomial regression on unknown manifolds , 2007, 0708.0983.

[25]  Noureddine El Karoui,et al.  Operator norm consistent estimation of large-dimensional sparse covariance matrices , 2008, 0901.3220.

[26]  Point process stabilization methods and dimension estimation , 2008 .

[27]  P. Bickel,et al.  Regularized estimation of large covariance matrices , 2008, 0803.1909.

[28]  Jeffrey S. Morris,et al.  Sure independence screening for ultrahigh dimensional feature space Discussion , 2008 .

[29]  I. Johnstone,et al.  Sparse Principal Components Analysis , 2009, 0901.4392.

[30]  P. Bickel,et al.  Covariance regularization by thresholding , 2009, 0901.3079.