Generalized projection pursuit regression

Projection pursuit regression (PPR) can be used to estimate a smooth function of several variables from noisy and scattered data. The estimate is a sum of smoothed one-dimensional projections of the variables. This paper discusses an extension of PPR to exponential family distributions, called generalized projection pursuit regression (GPPR). The proposed model allows multiple responses and nonlinear projections of the variables. Estimators are defined as minimizers of penalized cost functionals, and estimation is related to the local scoring procedure for generalized additive models (GAMs). Smooths are updated using a blockwise Gauss--Seidel (BGS) method, and convergence is shown for this procedure. The use of generalized cross validation (GCV) to estimate smoothing parameters is discussed. Experimental results are shown for two types of data.

[1]  J. R. Koehler,et al.  Modern Applied Statistics with S-Plus. , 1996 .

[2]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[3]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[4]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[5]  Charles B. Roosen.  Logistic Response Projection Pursuit , 1993 .

[6]  Simon Haykin,et al.  Neural networks , 1994 .

[7]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[8]  R. Tibshirani,et al.  Linear Smoothers and Additive Models , 1989 .

[9]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[10]  P. Diaconis,et al.  On Nonlinear Functions of Linear Combinations , 1984 .

[11]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[12]  Trevor Hastie,et al.  Automatic Smoothing Spline Projection Pursuit , 1994 .

[13]  B. Yandell,et al.  Automatic Smoothing of Regression Functions in Generalized Linear Models , 1986 .

[14]  K. Liestøl,et al.  Lactation and post-partum amenorrhoea: a study based on data from three Norwegian cities 1860–1964 , 1988, Journal of Biosocial Science.

[15]  Yuedong Wang.  Grkpack fitting smoothing spline anova models for exponential families , 1997 .

[16]  G. Wahba.  Smoothing noisy data with spline functions , 1975 .

[17]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[18]  G. Wahba,et al.  Smoothing spline ANOVA for exponential families, with application to the Wisconsin Epidemiological Study of Diabetic Retinopathy : the 1994 Neyman Memorial Lecture , 1995 .

[19]  K. Liestøl,et al.  Survival analysis and neural nets. , 1994, Statistics in medicine.

[20]  D. Hosmer,et al.  Applied Logistic Regression , 1991 .

[21]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[22]  Chong Gu,et al.  Cross-Validating Non-Gaussian Data , 1992 .

[23]  Chong Gu.  Adaptive Spline Smoothing in Non-Gaussian Regression Models , 1990 .

[24]  B. Silverman,et al.  Nonparametric regression and generalized linear models , 1994 .

[25]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[26]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[27]  Howell Tong,et al.  Nonparametric function estimation in noisy chaos , 1993 .

[28]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[29]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[30]  G. Wahba,et al.  A GENERALIZED APPROXIMATE CROSS VALIDATION FOR SMOOTHING SPLINES WITH NON-GAUSSIAN DATA , 1996 .

[31]  R. Tapia,et al.  Nonparametric Function Estimation, Modeling, and Simulation , 1987 .