Confidence sets in sparse regression

The problem of constructing confidence sets in the high-dimensional linear model with $n$ response variables and $p$ parameters, possibly $p\ge n$, is considered. Full honest adaptive inference is possible if the rate of sparse estimation does not exceed $n^{-1/4}$, otherwise sparse adaptive confidence sets exist only over strict subsets of the parameter spaces for which sparse estimators exist. Necessary and sufficient conditions for the existence of confidence sets that adapt to a fixed sparsity level of the parameter vector are given in terms of minimal $\ell^2$-separation conditions on the parameter space. The design conditions cover common coherence assumptions used in models for sparsity, including (possibly correlated) sub-Gaussian designs.

[1]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[2]  Ker-Chau Li,et al.  Honest Confidence Regions for Nonparametric Regression , 1989 .

[3]  A. A. Borokov,et al.  On Asymptotically Optimal Non-Parametric Criteria , 1968 .

[4]  S. Geer.  Empirical Processes in M-Estimation , 2000 .

[5]  J. Robins,et al.  Adaptive nonparametric confidence sets , 2006, math/0605473.

[6]  Sophie Lambert-Lacroix,et al.  On nonparametric confidence set estimation , 2001 .

[7]  Yannick Baraud,et al.  Confidence balls in Gaussian regression , 2004 .

[8]  R. Nickl,et al.  CONFIDENCE BANDS IN DENSITY ESTIMATION , 2010, 1002.4801.

[9]  S. Geer.  Least Squares Estimation , 2005 .

[10]  Cun-Hui Zhang,et al.  Confidence intervals for low dimensional parameters in high dimensional linear models , 2011, 1110.2563.

[11]  Marc Hoffmann,et al.  On adaptive inference and confidence bands , 2011, 1202.5145.

[12]  Adel Javanmard,et al.  Confidence intervals and hypothesis testing for high-dimensional regression , 2013, J. Mach. Learn. Res..

[13]  Sara van de Geer,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2011 .

[14]  S. Geer,et al.  On asymptotically optimal confidence regions and tests for high-dimensional models , 2013, 1303.0518.

[15]  Po-Ling Loh,et al.  High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity , 2011, NIPS.

[16]  T. Tony Cai,et al.  Adaptive Confidence Balls , 2006 .

[17]  R. Dudley.  The Sizes of Compact Subsets of Hilbert Space and Continuity of Gaussian Processes , 1967 .

[18]  R. Beran,et al.  Modulation Estimators and Confidence Sets , 1998 .

[19]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[20]  M. Talagrand.  New concentration inequalities in product spaces , 1996 .

[21]  Adam D. Bull,et al.  Adaptive confidence sets in $$L^2$$ , 2011, 1111.5568.

[22]  Ulrike Schneider,et al.  Distributional results for thresholding estimators in high-dimensional Gaussian regression models , 2011, 1106.6002.

[23]  R. Nickl,et al.  An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive estimation , 2009 .

[24]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[25]  E. Candès,et al.  Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism , 2010, 1007.1434.

[26]  Yu. I. Ingster,et al.  Detection boundary in sparse regression , 2010, 1009.1706.

[27]  Benedikt M. Pötscher.  Confidence Sets Based on Sparse Estimators Are Necessarily Large , 2007 .

[28]  Marc Hoffmann.  Random rates in anisotropic regression , 2002 .