Anti-concentration and honest, adaptive confidence bands

Modern construction of uniform confidence bands for non-parametric densities (and other functions) often relies on the classical Smirnov-Bickel-Rosenblatt (SBR) condition; see, for example, Gine and Nickl (2010). This condition requires the existence of a limit distribution of an extreme value type for the supremum of a studentized empirical process (equivalently, for the supremum of a Gaussian process with the same covariance function as that of the studentized empirical process). The principal contribution of this paper is to remove the need for this classical condition. We show that a considerably weaker sufficient condition is derived from an anti-concentration property of the supremum of the approximating Gaussian process, and we derive an inequality leading to such a property for separable Gaussian processes. We refer to the new condition as a generalized SBR condition. Our new result shows that the supremum does not concentrate too fast around any value. We then apply this result to derive a Gaussian multiplier bootstrap procedure for constructing honest confidence bands for nonparametric density estimators (this result can be applied in other nonparametetric problems as well). An essential advantage of our approach is that it applies generically even in those cases where the limit distribution of the supremum of the studentized empirical process does not exist (or is unknown). This is of particular importance in problems where resolution levels or other tuning parameters have been chosen in a data-driven fashion, which is needed for adaptive constructions of the confidence bands. Furthermore, our approach is asymptotically honest at a polynomial rate - namely, the error in coverage level converges to zero at a fast, polynomial speed (with respect to the sample size). In sharp contrast, the approach based on extreme value theory is asymptotically honest only at a logarithmic rate - the error converges to zero at a slow, logarithmic speed. Finally, of independent interest is our introduction of a new, practical version of Lepski's method, which computes the optimal, non-conservative resolution levels via a Gaussian multiplier bootstrap method.

[1]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[2]  A. Bull A Smirnov–Bickel–Rosenblatt Theorem for Compactly-Supported Wavelets , 2011, 1110.4961.

[3]  D. Picard,et al.  Adaptive confidence interval for pointwise curve estimation , 2000 .

[4]  A. Munk,et al.  Non‐parametric confidence bands in deconvolution density estimation , 2007 .

[5]  A. Tsybakov,et al.  Wavelets, approximation, and statistical applications , 1998 .

[6]  R. Nickl,et al.  An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive estimation , 2009 .

[7]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[8]  M. Talagrand New concentration inequalities in product spaces , 1996 .

[9]  Sokbae Lee,et al.  Intersection bounds: estimation and inference , 2009 .

[10]  Marc Hoffmann,et al.  On adaptive inference and confidence bands , 2011, 1202.5145.

[11]  R. Dudley,et al.  Uniform Central Limit Theorems: Notation Index , 2014 .

[12]  Wolfgang Heardle et al. Wavelets, approximation, and statistical applications , 2013 .

[13]  R. Nickl,et al.  CONFIDENCE BANDS IN DENSITY ESTIMATION , 2010, 1002.4801.

[14]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[15]  R. Nickl,et al.  GLOBAL UNIFORM RISK BOUNDS FOR WAVELET DECONVOLUTION ESTIMATORS , 2011, 1103.1489.

[16]  B. Silverman,et al.  Weak and Strong Uniform Consistency of the Kernel Estimate of a Density and its Derivatives , 1978 .

[17]  M. Rudelson,et al.  The smallest singular value of a random rectangular matrix , 2008, 0802.3956.

[18]  E. Giné,et al.  Some Limit Theorems for Empirical Processes , 1984 .

[19]  Kengo Kato,et al.  Gaussian approximation of suprema of empirical processes , 2014 .

[20]  P. Major,et al.  An approximation of partial sums of independent RV'-s, and the sample DF. I , 1975 .

[21]  Evarist Giné,et al.  On the Periodized Square of L 2 Cardinal Splines , 2011, Exp. Math..

[22]  Kengo Kato,et al.  Comparison and anti-concentration bounds for maxima of Gaussian random vectors , 2013, 1301.4807.

[23]  Peter Hall,et al.  A simple bootstrap method for constructing nonparametric confidence bands for functions , 2013, 1309.4864.

[24]  W. Newey,et al.  Convergence rates and asymptotic normality for series estimators , 1997 .

[25]  Mark G. Low On nonparametric confidence intervals , 1997 .

[26]  T. Cai,et al.  An adaptation theory for nonparametric confidence intervals , 2004, math/0503662.

[27]  Yingcun Xia,et al.  Bias‐corrected confidence bands in nonparametric regression , 1998 .

[28]  R. Nickl,et al.  Uniform limit theorems for wavelet density estimators , 2008, 0805.1406.

[29]  O. Lepskii Asymptotically Minimax Adaptive Estimation. I: Upper Bounds. Optimally Adaptive Estimates , 1992 .

[30]  M. Ledoux,et al.  Un critère sur les petites boules dans le théorème limite central , 1988 .

[31]  P. Hall On convergence rates of suprema , 1991 .

[32]  Ker-Chau Li,et al.  Honest Confidence Regions for Nonparametric Regression , 1989 .

[33]  D. Chetverikov TESTING REGRESSION MONOTONICITY IN ECONOMETRIC MODELS , 2012, Econometric Theory.

[34]  Adam D. Bull,et al.  Honest adaptive confidence bands and self-similar functions , 2011, 1110.4985.

[35]  L. Birge,et al.  An alternative point of view on Lepski's method , 2001 .

[36]  P. Massart,et al.  About the constants in Talagrand's concentration inequalities for empirical processes , 2000 .

[37]  L. Wasserman All of Nonparametric Statistics , 2005 .

[38]  J. Wellner,et al.  Exchangeably Weighted Bootstraps of the General Empirical Process , 1993 .

[39]  Emmanuel Rio,et al.  Local invariance principles and their application to density estimation , 1994 .

[40]  P. Bickel,et al.  On Some Global Measures of the Deviations of Density Function Estimates , 1973 .

[41]  Vladimir Koltchinskii,et al.  Kernel density estimators: convergence in distribution for weighted sup-norms , 2004 .

[42]  E. Giné,et al.  Bootstrapping General Empirical Measures , 1990 .

[43]  Gerda Claeskens,et al.  Bootstrap confidence bands for regression curves and their derivatives , 2003 .

[44]  Larry Wasserman,et al.  All of Nonparametric Statistics (Springer Texts in Statistics) , 2006 .

[45]  R. Nickl,et al.  Adaptive estimation of a distribution function and its density in sup-norm loss by wavelet and spline projections , 2008, 0805.1404.

[46]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[47]  E. Giné,et al.  Rates of strong uniform consistency for multivariate kernel density estimators , 2002 .