Adapting to Unknown Smoothness by Aggregation of Thresholded Wavelet Estimators

We study the performances of an adaptive procedure based on a convex combination, with data-driven weights, of term-by-term thresholded wavelet estimators. For the bounded regression model, with random uniform design, and the nonparametric density model, we show that the resulting estimator is optimal in the minimax sense over all Besov balls under the $L^2$ risk, without any logarithm factor.

[1]  A. Gardner.  Methods of Statistics , 1941 .

[2]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.

[3]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[4]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[5]  A. Cohen.  Ten Lectures on Wavelets, CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 61, I. Daubechies, SIAM, 1992, xix + 357 pp. , 1994 .

[6]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[7]  G. Nason.  Choice of the Threshold Parameter in Wavelet Function Estimation , 1995 .

[8]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[9]  G. Kerkyacharian,et al.  Adaptive Density Estimation , 1996 .

[10]  Y. Benjamini,et al.  Adaptive thresholding of wavelet coefficients , 1996 .

[11]  A. Juditsky,et al.  On Minimax Wavelet Estimators , 1996 .

[12]  I. Johnstone,et al.  Density estimation by wavelet thresholding , 1996 .

[13]  K. Burnham,et al.  Model selection: An integral part of inference , 1997 .

[14]  T. Cai.  On Adaptivity Of BlockShrink Wavelet Estimator Over Besov Spaces , 1997 .

[15]  H. Chipman,et al.  Adaptive Bayesian Wavelet Shrinkage , 1997 .

[16]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[17]  B. Silverman,et al.  Wavelet thresholding via a Bayesian approach , 1998 .

[18]  P. Hall,et al.  Block threshold rules for curve estimation using kernel and wavelet methods , 1998 .

[19]  Hong-Ye Gao,et al.  Wavelet Shrinkage Denoising Using the Non-Negative Garrote , 1998 .

[20]  Norman Weyrich,et al.  Wavelet shrinkage and generalized cross validation for image denoising , 1998, IEEE Trans. Image Process..

[21]  P. Hall,et al.  ON THE MINIMAX OPTIMALITY OF BLOCK THRESHOLDED WAVELET ESTIMATORS , 1999 .

[22]  T. Cai.  Adaptive wavelet estimation : A block thresholding and oracle inequality approach , 1999 .

[23]  E. Mammen,et al.  Smooth Discrimination Analysis , 1999 .

[24]  S. Efromovich.  Quasi-Linear Wavelet Estimation , 1999 .

[25]  Yuhong Yang.  Mixing Strategies for Density Estimation , 2000 .

[26]  A. Juditsky,et al.  Functional aggregation for nonparametric regression , 2000 .

[27]  Sam Efromovich,et al.  Sharp linear and block shrinkage wavelet estimation , 2000 .

[28]  D. Picard,et al.  Adaptive confidence interval for pointwise curve estimation , 2000 .

[29]  Anestis Antoniadis,et al.  Wavelet Estimators in Nonparametric Regression: A Comparative Simulation Study , 2001 .

[30]  Maarten Jansen,et al.  Noise Reduction by Wavelet Thresholding , 2001 .

[31]  B. Silverman,et al.  Incorporating Information on Neighboring Coefficients Into Wavelet Estimation , 2001 .

[32]  P. Massart,et al.  Gaussian model selection , 2001 .

[33]  B. Silverman,et al.  Some new methods for wavelet density estimation , 2001 .

[34]  Yuhong Yang.  Minimax rate adaptive estimation over continuous hyper-parameters , 2001, IEEE Trans. Inf. Theory.

[35]  Tong Zhang.  Statistical behavior and consistency of classification methods based on convex risk minimization , 2003 .

[36]  A. Tsybakov,et al.  Optimal aggregation of classifiers in statistical learning , 2003 .

[37]  Alexandre B. Tsybakov,et al.  Optimal Rates of Aggregation , 2003, COLT.

[38]  Olivier Catoni,et al.  Statistical learning theory and stochastic optimization , 2004 .

[39]  J. Picard,et al.  Statistical learning theory and stochastic optimization : École d'eté de probabilités de Saint-Flour XXXI - 2001 , 2004 .

[40]  I. Johnstone,et al.  Adapting to unknown sparsity by controlling the false discovery rate , 2005, math/0505374.

[41]  Alexander V. Nazin,et al.  Recursive Aggregation of Estimators by the Mirror Descent Algorithm with Averaging , 2005, Probl. Inf. Transm..

[42]  G. Lecu'e.  Simultaneous adaptation to the margin and to complexity in classification , 2005, math/0509696.

[43]  I. Johnstone,et al.  Empirical Bayes selection of wavelet thresholds , 2005, math/0508281.

[44]  T. Cai,et al.  Block thresholding for density estimation: local and global adaptivity , 2005 .

[45]  Cun-Hui Zhang.  General empirical Bayes wavelet methods and exactly adaptive minimax estimation , 2005, math/0504501.

[46]  Guillaume Lecué.  Optimal rates of aggregation in classification , 2006 .

[47]  Guillaume Lecué.  Lower Bounds and Aggregation in Density Estimation , 2006, J. Mach. Learn. Res..

[48]  V. Koltchinskii.  Local Rademacher complexities and oracle inequalities in risk minimization , 2006, 0708.0083.

[49]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[50]  Guillaume Lecué,et al.  Optimal Oracle Inequality for Aggregation of Classifiers Under Low Noise Condition , 2006, COLT.

[51]  Andrew R. Barron,et al.  Information Theory and Mixing Least-Squares Regressions , 2006, IEEE Transactions on Information Theory.

[52]  L. Birge,et al.  Model selection via testing: an alternative to (penalized) maximum likelihood estimators , 2006 .

[53]  Guillaume Lecué,et al.  Suboptimality of Penalized Empirical Risk Minimization in Classification , 2007, COLT.

[54]  Ingo Steinwart,et al.  Fast rates for support vector machines using Gaussian kernels , 2007, 0708.1838.

[55]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[56]  Guillaume Lecu 'e.  Suboptimality of Penalized Empirical Risk Minimization in Classification , 2007 .

[57]  Andrew B. Nobel,et al.  Sequential Procedures for Aggregating Arbitrary Estimators of a Conditional Mean , 2008, IEEE Transactions on Information Theory.

[58]  A. Juditsky,et al.  Learning by mirror averaging , 2005, math/0511468.