Training and evaluation of neural networks for multi-variate time series processing

We study the training and generalization for multi-variate time series processing. It is suggested to used a quasi-maximum likelihood approach rather than the standard sum of squared errors, thus taking dependencies among the errors of the individual time series into account. This may lead to improved generalization performance. Further, we extend the optimal brain damage pruning technique to the multi-variate case. A key ingredient is an algebraic expression for the generalization ability of a multi-variate model. The variability of the suggested techniques are successfully demonstrated in a multi-variate scenario involving the prediction of the cylinder pressure in a marine engine.

[1]  Lars Kai Hansen,et al.  On design and evaluation of tapped-delay neural network architectures , 1993, IEEE International Conference on Neural Networks.

[2]  John J. Moskwa,et al.  Cylinder pressure and combustion heat release estimation for SI engine diagnostics using nonlinear sliding observers , 1995, IEEE Trans. Control. Syst. Technol..

[3]  Stefanos Kollias,et al.  An adaptive least squares algorithm for the efficient training of artificial neural networks , 1989 .

[4]  Shun-ichi Amari,et al.  Network information criterion-determining the number of hidden units for an artificial neural network model , 1994, IEEE Trans. Neural Networks.

[5]  Bernie Mulgrew,et al.  IEEE Workshop on Neural Networks for Signal Processing , 1995 .

[6]  R. G. Cornell,et al.  Simultaneous Nonlinear Estimation , 1966 .

[7]  Halbert White,et al.  Learning in Artificial Neural Networks: A Statistical Perspective , 1989, Neural Computation.

[8]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[9]  Stephen A. Billings,et al.  Non-linear system identification using neural networks , 1990 .

[10]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[11]  Lars Kai Hansen,et al.  Generalization performance of regularized neural network models , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[12]  Lars Kai Hansen,et al.  Designer networks for time series processing , 1993, Neural Networks for Signal Processing III - Proceedings of the 1993 IEEE-SP Workshop.

[13]  J. Schmee.  An Introduction to Multivariate Statistical Analysis , 1986 .

[14]  John E. Moody,et al.  The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems , 1991, NIPS.

[15]  Lars Kai Hansen,et al.  Empirical generalization assessment of neural network models , 1995, Proceedings of 1995 IEEE Workshop on Neural Networks for Signal Processing.

[16]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.