Automatic nonlinear predictive model-construction algorithm using forward regression and the PRESS statistic

An automatic nonlinear predictive model-construction algorithm is introduced based on forward regression and the predicted-residual-sums-of-squares (PRESS) statistic. The proposed algorithm is based on the fundamental concept of evaluating a model's generalisation capability through crossvalidation. This is achieved by using the PRESS statistic as a cost function to optimise model structure. In particular, the proposed algorithm is developed with the aim of achieving computational efficiency, such that the computational effort, which would usually be extensive in the computation of the PRESS statistic, is reduced or minimised. The computation of PRESS is simplified by avoiding a matrix inversion through the use of the orthogonalisation procedure inherent in forward regression, and is further reduced significantly by the introduction of a forward-recursive formula. Based on the properties of the PRESS statistic, the proposed algorithm can achieve a fully automated procedure without resort to any other validation data set for iterative model evaluation. Numerical examples are used to demonstrate the efficacy of the algorithm.

[1]  Xia Hong,et al.  Adaptive Modelling, Estimation and Fusion from Data , 2002, Advanced Information Processing.

[2]  Steve A. Billings,et al.  Parameter estimation based on stacked regression and evolutionary algorithms , 1999 .

[3]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[4]  P. Young,et al.  Identification of non-linear stochastic systems by state dependent parameter estimation , 2001 .

[5]  Quanmin Zhu,et al.  Neural network enhanced generalised minimum variance self-tuning controller for nonlinear discrete-time systems , 1999 .

[6]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[7]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[8]  L X Wang,et al.  Fuzzy basis functions, universal approximation, and orthogonal least-squares learning , 1992, IEEE Trans. Neural Networks.

[9]  Xia Hong,et al.  Adaptive Modelling, Estimation and Fusion from Data: A Neurofuzzy Approach , 2002, Advanced information processing.

[10]  Xia Hong,et al.  Nonlinear model structure detection using optimum experimental design and orthogonal least squares , 2001, IEEE Trans. Neural Networks.

[11]  W. Näther.  Optimum experimental designs , 1994 .

[12]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[13]  Xia Hong,et al.  Nonlinear model structure design and construction using orthogonal least squares and D-optimality design , 2002, IEEE Trans. Neural Networks.

[14]  Peter C. Young,et al.  Recursive Estimation and Time Series Analysis , 1984 .

[15]  Sheng Chen,et al.  Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks , 1999, IEEE Trans. Neural Networks.

[16]  Xia Hong,et al.  Generalized neurofuzzy network modeling algorithms using Bezier-Bernstein polynomial functions and additive decomposition , 2000, IEEE Trans. Neural Networks Learn. Syst..

[17]  H. Akaike.  A new look at the statistical model identification , 1974 .

[18]  Anthony J. Jakeman,et al.  An instrumental variable method for model order identification , 1980, Autom..