Confidence Sets for the Optimal Approximating Model - Bridging a Gap between Adaptive Point Estimation and Confidence Regions

In the setting of high-dimensional linear models with Gaussian noise, we investigate the possibility of confidence statements connected to model selection. Although there exist numerous procedures for adaptive point estimation, the construction of adaptive confidence regions is severely limited (cf. Li, 1989). The present paper sheds new light on this gap. We develop exact and adaptive confidence sets for the best approximating model in terms of risk. Our construction is based on a multiscale procedure and a particular coupling argument. Utilizing exponential inequalities for noncentral χ2–distributions, we show that the risk and quadratic loss of all models within our confidence region are uniformly bounded by the minimal risk times a factor close to one.

[1]  C. J. Stone,et al.  An Asymptotically Optimal Window Selection Rule for Kernel Density Estimates , 1984 .

[2]  Ker-Chau Li,et al.  Honest Confidence Regions for Nonparametric Regression , 1989 .

[3]  Boris Polyak,et al.  Asymptotic Optimality of the $C_p$-Test for the Orthogonal Series Estimation of Regression , 1991 .

[4]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[5]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[6]  Philip B. Stark,et al.  Finite-Sample Confidence Envelopes for Shape-Restricted Densities , 1995 .

[7]  R. Beran.  Confidence sets centered at Cp-estimators , 1996 .

[8]  E. Mammen,et al.  Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors , 1997 .

[9]  I. Johnstone,et al.  Minimax estimation via wavelet shrinkage , 1998 .

[10]  R. Beran,et al.  Modulation Estimators and Confidence Sets , 1998 .

[11]  S. Efromovich.  Simultaneous sharp estimation of functions and their derivatives , 1998 .

[12]  A. Futschik.  Confidence regions for the set of global maximizers of nonparametrically estimated curves , 1999 .

[13]  T. Cai.  Adaptive wavelet estimation : A block thresholding and oracle inequality approach , 1999 .

[14]  Rudolf Beran,et al.  React Scatterplot Smoothers: Superefficiency through Basis Economy , 2000 .

[15]  V. Spokoiny,et al.  Multiscale Testing of Qualitative Hypotheses 1 , 2001 .

[16]  V. Spokoiny,et al.  Multiscale testing of qualitative hypotheses , 2001 .

[17]  O. Lepski,et al.  Random rates in anisotropic regression (with a discussion and a rejoinder by the authors) , 2002 .

[18]  Lutz Dümbgen,et al.  Application of local rank tests to nonparametric regression , 2002 .

[19]  T. Tony Cai,et al.  ON BLOCK THRESHOLDING IN WAVELET REGRESSION: ADAPTIVITY, BLOCK SIZE, AND THRESHOLD LEVEL , 2002 .

[20]  L. Duembgen.  Optimal Confidence Bands for Shape-Restricted Curves , 2003, 1312.6466.

[21]  Yannick Baraud,et al.  Confidence balls in Gaussian regression , 2004 .

[22]  Christopher R. Genovese,et al.  Confidence sets for nonparametric wavelet regression , 2005, math/0505632.

[23]  R. Dahlhaus,et al.  Nonparametric quasi-maximum likelihood estimation for Gaussian locally stationary processes , 2006, 0708.0143.

[24]  J. Robins,et al.  Adaptive nonparametric confidence sets , 2006, math/0605473.

[25]  T. Tony Cai,et al.  Adaptive Confidence Balls , 2006 .

[26]  L. Duembgen,et al.  Multiscale inference about a density , 2007, 0706.3968.

[27]  Random Rates in Anisotropic Regression : Discussion , 2010 .