Boosting additive models using component-wise P-Splines

An efficient approximation of L"2 Boosting with component-wise smoothing splines is considered. Smoothing spline base-learners are replaced by P-spline base-learners, which yield similar prediction errors but are more advantageous from a computational point of view. A detailed analysis of the effect of various P-spline hyper-parameters on the boosting fit is given. In addition, a new theoretical result on the relationship between the boosting stopping iteration and the step length factor used for shrinking the boosting estimates is derived.

[1]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[2]  L. Breiman.  Arcing classifier (with discussion and a rejoinder by the author) , 1998 .

[3]  B. Peter.  BOOSTING FOR HIGH-DIMENSIONAL LINEAR MODELS , 2006 .

[4]  D. Cox.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[5]  B. Silverman,et al.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[6]  G. Tutz,et al.  Generalized Additive Modeling with Implicit Variable Selection by Likelihood‐Based Boosting , 2006, Biometrics.

[7]  Peter Buhlmann,et al.  BOOSTING ALGORITHMS: REGULARIZATION, PREDICTION AND MODEL FITTING , 2007, 0804.2752.

[8]  C. R. Deboor,et al.  A practical guide to splines , 1978 .

[9]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[10]  Gerhard Tutz,et al.  Knot selection by boosting techniques , 2007, Comput. Stat. Data Anal..

[11]  P. Bühlmann,et al.  Boosting With the L2 Loss , 2003 .

[12]  P. Bühlmann,et al.  Boosting with the L2-loss: regression and classification , 2001 .

[13]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[14]  Torsten Hothorn,et al.  Improved prediction of body fat by measuring skinfold thickness, circumferences, and bone breadths. , 2005, Obesity research.

[15]  S. Wood.  Generalized Additive Models: An Introduction with R , 2006 .

[16]  M. Wand,et al.  Semiparametric Regression: Parametric Regression , 2003 .

[17]  D. Ruppert.  Selecting the Number of Knots for Penalized Splines , 2002 .

[18]  Clifford M. Hurvich,et al.  Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion , 1998 .

[19]  Paul H. C. Eilers,et al.  Direct generalized additive modeling with penalized likelihood , 1998 .

[20]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[21]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[22]  Paul H. C. Eilers,et al.  Flexible smoothing with B-splines and penalties , 1996 .

[23]  Leo Breiman,et al.  Prediction Games and Arcing Algorithms , 1999, Neural Computation.

[24]  Paul H. C. Eilers,et al.  Splines, knots, and penalties , 2010 .

[25]  Trevor Hastie,et al.  Additive Logistic Regression : a Statistical , 1998 .