Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions

This paper discusses a nonparametric regression model that naturally generalizes neural network models. The model is based on a finite number of one-dimensional transformations and can be estimated with a one-dimensional rate of convergence. The model contains the generalized additive model with unknown link function as a special case. For this case, it is shown that the additive components and link function can be estimated with the optimal rate by a smoothing spline that is the solution of a penalized least squares criterion.

[1]  M. Birman,et al.  PIECEWISE-POLYNOMIAL APPROXIMATIONS OF FUNCTIONS OF THE CLASSES $ W_{p}^{\alpha}$ , 1967 .

[2]  C. J. Stone,et al.  The Dimensionality Reduction Principle for Generalized Additive Models , 1986 .

[3]  Sara van de Geer,et al.  Penalized quasi-likelihood estimation in partial linear models , 1997 .

[4]  Christine Thomas-Agnan,et al.  Smoothing Splines and Shape Restrictions , 1999 .

[5]  Enno Mammen,et al.  The Existence and Asymptotic Properties of a Backfitting Projection Algorithm Under Weak Conditions , 1999 .

[6]  Mark Coppejans,et al.  On Kolmogorov's representation of functions of several variables by functions of one variable , 2004 .

[7]  Joel L. Horowitz,et al.  NONPARAMETRIC ESTIMATION OF A GENERALIZED ADDITIVE MODEL WITH AN UNKNOWN LINK FUNCTION , 2001 .

[8]  O. Linton,et al.  A kernel method of estimating structured nonparametric regression based on marginal integration , 1995 .

[9]  Göran Kauermann,et al.  Local Likelihood Estimation in Generalized Additive Models , 2003 .

[10]  C. J. Stone,et al.  The Use of Polynomial Splines and Their Tensor Products in Multivariate Function Estimation , 1994 .

[11]  Jean D. Opsomer,et al.  Asymptotic Properties of Backfitting Estimators , 2000 .

[12]  E. Mammen,et al.  Generalised structured models , 2003 .

[13]  W. Härdle,et al.  Estimation of additive regression models with known links , 1996 .

[14]  Joel L. Horowitz,et al.  Nonparametric estimation of an additive model with a link function , 2002, math/0508595.

[15]  B. Yandell.  Spline smoothing and nonparametric regression , 1989 .

[16]  David Ruppert,et al.  Fitting a Bivariate Additive Model by Local Polynomial Regression , 1997 .

[17]  J. Friedman,et al.  Estimating Optimal Transformations for Multiple Regression and Correlation. , 1985 .

[18]  E. Mammen,et al.  A simple smooth backfitting method for additive models , 2006, math/0702657.

[19]  Enno Mammen,et al.  Smooth backfitting in generalized additive models , 2008, 0803.1922.

[20]  S. Agmon.  Lectures on Elliptic Boundary Value Problems , 1965 .

[21]  John A. Nelder,et al.  Generalized linear models. 2nd ed. , 1993 .

[22]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[23]  M. C. Jones,et al.  Spline Smoothing and Nonparametric Regression. , 1989 .

[24]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[25]  R. Tibshirani,et al.  Linear Smoothers and Additive Models , 1989 .

[26]  Jens Perch Nielsen,et al.  Smooth backfitting in practice , 2005 .

[27]  Dag Tjøstheim,et al.  Nonparametric Identification of Nonlinear Time Series: Projections , 1994 .

[28]  Anatoli B. Juditsky,et al.  NONPARAMETRIC ESTIMATION OF COMPOSITE FUNCTIONS , 2009, 0906.0865.

[29]  Wolfgang Härdle,et al.  Direct estimation of low-dimensional components in additive models , 1998 .

[30]  H. Ichimura,et al.  SEMIPARAMETRIC LEAST SQUARES (SLS) AND WEIGHTED SLS ESTIMATION OF SINGLE-INDEX MODELS , 1993 .

[31]  S. R. Jammalamadaka,et al.  Empirical Processes in M-Estimation , 2001 .

[32]  S. Geer.  Empirical Processes in M-Estimation , 2000 .

[33]  Joel L. Horowitz,et al.  Optimal estimation in additive regression models , 2006 .

[34]  S. Geer.  Estimating a Regression Function , 1990 .

[35]  W. Newey,et al.  Convergence rates and asymptotic normality for series estimators , 1997 .

[36]  Oliver Linton,et al.  EFFICIENT ESTIMATION OF GENERALIZED ADDITIVE NONPARAMETRIC REGRESSION MODELS , 2000, Econometric Theory.

[37]  C. J. Stone,et al.  Additive Regression and Other Nonparametric Models , 1985 .

[38]  Wolfgang Härdle,et al.  BOOTSTRAP INFERENCE IN SEMIPARAMETRIC GENERALIZED ADDITIVE MODELS , 2004, Econometric Theory.

[39]  Enno Mammen,et al.  Bandwidth selection for smooth backfitting in additive models , 2005, math/0507425.

[40]  Roland Glowinski,et al.  An introduction to the mathematical theory of finite elements , 1976 .

[41]  R. W. Wedderburn.  Quasi-likelihood functions, generalized linear models, and the Gauss-Newton method , 1974 .

[42]  Stefan Hoderlein,et al.  Local Partitioned Regression , 2006 .