A generalization error estimate for nonlinear systems

A new estimate (GEN) of the generalization error is presented. The estimator is valid for both incomplete and nonlinear models. An incomplete model is characterized in that it does not model the actual nonlinear relationship perfectly. The GEN estimator has been evaluated by simulating incomplete models of linear and simple neural network systems. Within the linear system GEN is compared to the final prediction error criterion and the leave-one-out cross-validation technique. It was found that the GEN estimate of the true generalization error is less biased on the average. It is concluded that GEN is an applicable alternative in estimating the generalization at the expense of an increased complexity.<<ETX>>

[1]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  John Moody,et al.  Note on generalization, regularization and architecture selection in nonlinear learning systems , 1991, Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop.

[3]  David B. Fogel.  An information criterion for optimal neural network selection , 1991, IEEE Trans. Neural Networks.

[4]  M. Rosenblatt.  Stationary sequences and random fields , 1985 .

[5]  George W. Hart,et al.  Memoryless nonlinear system identification with unknown model order , 1991, IEEE Trans. Inf. Theory.

[6]  H. Akaike.  Fitting autoregressive models for prediction , 1969 .

[7]  N. Draper,et al.  Applied Regression Analysis , 1966 .