Structural adaptation via Lp-norm oracle inequalities

In this paper we study the problem of adaptive estimation of a multivari- ate function satisfying some structural assumption. We propose a novel estimation procedure that adapts simultaneously to unknown structure and smoothness of the underlying function. The problem of structural adaptation is stated as the problem of selection from a given collection of estimators. We develop a general selection rule and establish for it global oracle inequalities under arbitrary L p-losses. These results are applied for adaptive estimation in the additive multi-index model.

[1]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[2]  V. Spokoiny,et al.  Optimal pointwise adaptive methods in nonparametric estimation , 1997 .

[3]  Arkadi Nemirovski,et al.  Topics in Non-Parametric Statistics , 2000 .

[4]  O. Lepski,et al.  Adaptive non-parametric estimation of smooth multivariate functions , 1999 .

[5]  C. J. Stone,et al.  Additive Regression and Other Nonparametric Models , 1985 .

[6]  A. Tsybakov,et al.  Oracle inequalities for inverse problems , 2002 .

[7]  G. Kerkyacharian,et al.  Nonlinear estimation in anisotropic multi-index denoising , 2001 .

[8]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[9]  J. Polzehl,et al.  Structure adaptive approach for dimension reduction , 2001 .

[10]  P. Massart,et al.  Risk bounds for model selection via penalization , 1999 .

[11]  Kellen Petersen August.  Real Analysis , 2009 .

[12]  M. Nyssbaum.  Nonparametric Estimation of a Regression Function that is Smooth in a Domain in $R^k$ , 1987 .

[13]  E. Mammen,et al.  Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors , 1997 .

[14]  Y. Yatracos,et al.  Rates of convergence of estimates, Kolmogorov's entropy and the dimensionality reduction principle in regression , 1997 .

[15]  C. J. Stone,et al.  Optimal Global Rates of Convergence for Nonparametric Regression , 1982 .

[16]  Hung Chen,et al.  ESTIMATION OF A PROJECTION-PURSUIT TYPE REGRESSION MODEL' , 1991 .

[17]  Karine Bertin,et al.  Asymptotically exact minimax estimation in sup-norm for anisotropic Hölder classes , 2004 .

[18]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[19]  Luc Devroye,et al.  Combinatorial methods in density estimation , 2001, Springer series in statistics.

[20]  M. Talagrand.  Sharper Bounds for Gaussian and Empirical Processes , 1994 .

[21]  D. Picard,et al.  Nonlinear Estimation in Anisotropic Multi-Index Denoising. Sparse Case , 2008 .

[22]  R. Jennrich.  Asymptotic Properties of Non-Linear Least Squares Estimators , 1969 .

[23]  A. Juditsky,et al.  Direct estimation of the index coefficient in a single-index model , 2001 .

[24]  A. Goldenshluger.  On Spatial Adaptive Estimation of Nonparametric Regression , 2004 .

[25]  M. Lifshits.  Gaussian Random Functions , 1995 .

[26]  Alexandre B. Tsybakov,et al.  Optimal Rates of Aggregation , 2003, COLT.