On the Post Selection Inference constant under Restricted Isometry Properties

Uniformly valid confidence intervals post model selection in regression can be constructed based on Post-Selection Inference (PoSI) constants. PoSI constants are minimal for orthogonal design matrices, and can be upper bounded in function of the sparsity of the set of models under consideration, for generic design matrices. In order to improve on these generic sparse upper bounds, we consider design matrices satisfying a Restricted Isometry Property (RIP) condition. We provide a new upper bound on the PoSI constant in this setting. This upper bound is an explicit function of the RIP constant of the design matrix, thereby giving an interpolation between the orthogonal setting and the generic sparse setting. We show that this upper bound is asymptotically optimal in many settings by constructing a matching lower bound.

[1]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[2]  I. Ibragimov,et al.  Norms of Gaussian sample functions , 1976 .

[3]  L. Birge,et al.  An alternative point of view on Lepski's method , 2001 .

[4]  Hannes Leeb,et al.  Performance Limits for Estimators of the Risk or Distribution of Shrinkage-Type Estimators, and Some General Lower Risk-Bound Results , 2002 .

[5]  장윤희,et al.  Y. , 2003, Industrial and Labor Relations Terms.

[6]  B. M. Pötscher,et al.  MODEL SELECTION AND INFERENCE: FACTS AND FICTION , 2005, Econometric Theory.

[7]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[8]  Paul Kabaila,et al.  On the Large-Sample Minimal Coverage Probability of Confidence Intervals After Model Selection , 2006 .

[9]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[10]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[11]  Benedikt M. Pötscher.  Confidence Sets Based on Sparse Estimators Are Necessarily Large , 2007 .

[12]  S. Geer,et al.  On the conditions used to prove oracle results for the Lasso , 2009, 0910.0722.

[13]  J. W. Silverstein,et al.  Spectral Analysis of Large Dimensional Random Matrices , 2009 .

[14]  Walter Zucchini,et al.  Model Selection , 2011, International Encyclopedia of Statistical Science.

[15]  Cun-Hui Zhang,et al.  Confidence intervals for low dimensional parameters in high dimensional linear models , 2011, 1110.2563.

[16]  A. Belloni,et al.  Inference for High-Dimensional Sparse Econometric Models , 2011, 1201.0220.

[17]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[18]  A. Belloni,et al.  Inference on Treatment Effects after Selection Amongst High-Dimensional Controls , 2011, 1201.0224.

[19]  Kengo Kato,et al.  Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors , 2013 .

[20]  Dennis L. Sun,et al.  Exact post-selection inference, with application to the lasso , 2013, 1311.6238.

[21]  A. Buja,et al.  Valid post-selection inference , 2013, 1306.1059.

[22]  R. Tibshirani,et al.  Exact Post-Selection Inference for Sequential Regression Procedures , 2014, 1401.3889.

[23]  C. Giraud.  Introduction to High-Dimensional Statistics , 2014 .

[24]  S. Geer,et al.  On asymptotically optimal confidence regions and tests for high-dimensional models , 2013, 1303.0518.

[25]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[26]  Dennis L. Sun,et al.  Optimal Inference After Model Selection , 2014, 1410.2597.

[27]  Jonathan E. Taylor,et al.  Exact Post Model Selection Inference for Marginal Screening , 2014, NIPS.

[28]  S. Chatterjee.  Superconcentration and Related Topics , 2014 .

[29]  秀俊 松井,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2014 .

[30]  慧 廣瀬.  A Mathematical Introduction to Compressive Sensing , 2015 .

[31]  F. Bachoc,et al.  Uniformly valid confidence intervals post-model-selection , 2016, The Annals of Statistics.

[32]  Kai Zhang,et al.  Spherical Cap Packing Asymptotics and Rank-Extreme Detection , 2015, IEEE Transactions on Information Theory.

[33]  Arun K. Kuchibhotla,et al.  A Model Free Perspective for Linear Regression: Uniform-in-model Bounds for Post Selection Inference , 2018 .

[34]  R. Tibshirani,et al.  Uniform asymptotic inference and the bootstrap after model selection , 2015, The Annals of Statistics.

[35]  F. Bachoc,et al.  Valid confidence intervals for post-model-selection predictors , 2014, The Annals of Statistics.