Multi-output regression using a locally regularised orthogonal least-squares algorithm

The paper considers data modelling using multi-output regression models. A locally regularised orthogonal least-squares (LROLS) algorithm is proposed for constructing sparse multi-output regression models that generalise well. By associating each regressor in the regression model with an individual regularisation parameter, the ability of the multi-output orthogonal least-squares (OLS) model selection to produce a parsimonious model with a good generalisation performance is greatly enhanced.

[1]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[2]  Sheng Chen,et al.  Regularized orthogonal least squares algorithm for constructing radial basis function networks , 1996 .

[3]  Peter Grant,et al.  Orthogonal least squares algorithms for training multi-output radial basis function networks , 1991 .

[4]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[5]  Sheng Chen,et al.  Identification of MIMO non-linear systems using a forward-regression orthogonal estimator , 1989 .

[6]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[7]  D. G. Watts,et al.  Spectral analysis and its applications , 1968 .

[8]  Sheng Chen,et al.  Extended model set, global data and threshold model identification of severely non-linear systems , 1989 .

[9]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[10]  S. Chen,et al.  Fast orthogonal least squares algorithm for efficient subset model selection , 1995, IEEE Trans. Signal Process..

[11]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[12]  Chris Bishop,et al.  Improving the Generalization Properties of Radial Basis Function Neural Networks , 1991, Neural Computation.

[13]  T. Kavli.  ASMO—Dan algorithm for adaptive spline modelling of observation data , 1993 .

[14]  Jasvinder S. Kandola,et al.  Interpretable modelling with sparse kernels , 2001 .

[15]  I. J. Leontaritis,et al.  Model selection and validation methods for non-linear systems , 1987 .