Projection estimation in multiple regression with application to functional ANOVA models

A general theory on rates of convergence of the least-squares projection estimate in multiple regression is developed. The theory is applied to the functional ANOVA model, where the multivariate regression function Ž is modeled as a specified sum of a constant term, main effects functions of .Ž one variable and selected interaction terms functions of two or more . variables . The least-squares projection is onto an approximating space constructed from arbitrary linear spaces of functions and their tensor products respecting the assumed ANOVA structure of the regression function. The linear spaces that serve as building blocks can be any of the ones commonly used in practice: polynomials, trigonometric polynomials, splines, wavelets and finite elements. The rate of convergence result that is obtained reinforces the intuition that low-order ANOVA modeling can achieve dimension reduction and thus overcome the curse of dimensionality. Moreover, the components of the projection estimate in an appropriately defined ANOVA decomposition provide consistent estimates of the corresponding components of the regression function. When the regression function does not satisfy the assumed ANOVA form, the projection estimate converges to its best approximation of that form.

[1]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[2]  A. Brown.  THEORY OF APPROXIMATION OF FUNCTIONS OF A REAL VARIABLE , 1966 .

[3]  Carl de Boor,et al.  A bound on the _{∞}-norm of ₂-approximation by splines in terms of a global mesh ratio , 1976 .

[4]  C. R. Deboor,et al.  A practical guide to splines , 1978 .

[5]  L. Schumaker.  Spline Functions: Basic Theory , 1981 .

[6]  C. J. Stone,et al.  Optimal Global Rates of Convergence for Nonparametric Regression , 1982 .

[7]  C. Micchelli,et al.  Recent Progress in multivariate splines , 1983 .

[8]  A. Takemura.  Tensor Analysis of ANOVA Decomposition , 1983 .

[9]  C. J. Stone,et al.  Additive Regression and Other Nonparametric Models , 1985 .

[10]  C. J. Stone,et al.  The Dimensionality Reduction Principle for Generalized Additive Models , 1986 .

[11]  J. Tinsley Oden,et al.  Finite Elements, Mathematical Aspects. , 1986 .

[12]  R. DeVore,et al.  Interpolation of Besov-Spaces , 1988 .

[13]  C. Micchelli,et al.  On multivariate -splines , 1989 .

[14]  Trevor Hastie,et al.  [Flexible Parsimonious Smoothing and Additive Modeling]: Discussion , 1989 .

[15]  J. Friedman,et al.  FLEXIBLE PARSIMONIOUS SMOOTHING AND ADDITIVE MODELING , 1989 .

[16]  Prabir Burman,et al.  Estimation of generalized additive models , 1990 .

[17]  G. Wahba.  Spline models for observational data , 1990 .

[18]  D. Pollard.  Empirical Processes: Theory and Applications , 1990 .

[19]  Zehua Chen.  Interaction Spline Models and Their Convergence Rates , 1991 .

[20]  A. Barron,et al.  APPROXIMATION OF DENSITY FUNCTIONS BY SEQUENCES OF EXPONENTIAL FAMILIES , 1991 .

[21]  G. Wahba,et al.  Smoothing Spline ANOVA with Component-Wise Bayesian “Confidence Intervals” , 1993 .

[22]  Zehua Chen.  Fitting Multivariate Regression Functions by Interaction Spline Models , 1993 .

[23]  Leo Breiman,et al.  Fitting additive models to regression data , 1993, Computational Statistics & Data Analysis.

[24]  Y. Meyer.  Wavelets and Operators , 1993 .

[25]  George G. Lorentz,et al.  Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[26]  A. Timan.  Theory of Approximation of Functions of a Real Variable , 1994 .

[27]  C. J. Stone,et al.  The Use of Polynomial Splines and Their Tensor Products in Multivariate Function Estimation , 1994 .

[28]  Young K. Truong,et al.  The L2 rate of convergence for hazard regression , 1995 .

[29]  C. J. Stone,et al.  Polychotomous Regression , 1995 .

[30]  Young K. Truong,et al.  Polynomial splines and their tensor products in extended linear modeling: 1994 Wald memorial lecture , 1997 .

[31]  I. Johnstone,et al.  Minimax estimation via wavelet shrinkage , 1998 .

[32]  Jianhua Z. Huang,et al.  The L2 Rate of Convergence for Event History Regression with Time‐dependent Covariates , 1998 .

[33]  Jianhua Z. Huang.  Functional ANOVA Models for Generalized Regression , 1998 .

[34]  C. J. Stone,et al.  Hazard Regression , 2022 .