Kernel based partially linear models and nonlinear identification

In this note, we propose partially linear models with least squares support vector machines (LS-SVMs) for nonlinear ARX models. We illustrate how full black-box models can be improved when prior information about model structure is available. A real-life example, based on the Silverbox benchmark data, shows significant improvements in the generalization ability of the structured model with respect to the full black-box model, reflected also by a reduction in the effective number of parameters.

[1]  C. L. Mallows.  Some comments on C_p , 1973 .

[2]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[3]  P. Robinson.  ROOT-N-CONSISTENT SEMIPARAMETRIC REGRESSION , 1988 .

[4]  P. Speckman.  Kernel smoothing in partial linear models , 1988 .

[5]  Siddhartha R. Dalal,et al.  Some Graphical Aids for Deciding When to Stop Testing Software , 1990, IEEE J. Sel. Areas Commun..

[6]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[7]  Simon Haykin,et al.  Neural networks , 1994 .

[8]  C. Mallows.  More comments on C p , 1995 .

[9]  Lennart Ljung,et al.  Nonlinear black-box modeling in system identification: a unified overview , 1995, Autom..

[10]  Joos Vandewalle,et al.  Wavelet Based Modeling of Nonlinear Systems , 1998 .

[11]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[12]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[13]  C. H. Oh,et al.  Some comments on , 1998 .

[14]  Wolfgang Härdle,et al.  Partially Linear Models , 2000 .

[15]  Colin L. Mallows,et al.  Some Comments on Cp , 2000, Technometrics.

[16]  Christopher K. I. Williams,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[17]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[18]  Yves Rolain,et al.  Fast approximate identification of nonlinear systems , 2003, Autom..

[19]  J. Suykens,et al.  Partially linear models and least squares support vector machines , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[20]  K. U. Leuven.  A COMPARATIVE STUDY OF LS-SVM’S APPLIED TO THE SILVER BOX IDENTIFICATION PROBLEM , 2004 .

[21]  H. Tong,et al.  Semiparametric non‐linear time series model selection , 2004 .

[22]  Steve R. Gunn,et al.  Structural Modelling with Sparse Kernels , 2002, Machine Learning.

[23]  D. Lindgren.  Projection Techniques for Classification and Identification , 2005 .

[24]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.