Robustness quantification of recurrent neural network using unscented transform

Artificial recurrent neural network has been proved to be a valuable tool in modeling nonlinear dynamical systems. Robustness study of recurrent neural network is critical to its successful implementation. The goal of robustness study is to reduce the sensitivity of modeling capability to parametric uncertainties or make the network fault tolerant. In this study, an uncertainty propagation analysis is performed to quantify the robustness of a recurrent neural network output due to perturbations in its trained weights. An uncertainty propagation analysis-based robustness measure has been proposed accordingly and further compared with available performance loss-based and sensitivity matrix-based approaches. Results show that the proposed robustness measure approach is more efficient, generic, and flexible to quantify the robustness of a recurrent neural network.

[1]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[2]  Xiaoyu Wang,et al.  OPTIMIZED RECURRENT NEURAL NETWORK-BASED TOOL WEAR MODELING IN HARD TURNING , 2009 .

[3]  Yakov Ben-Haim,et al.  Evaluation of Neural Network Robust Reliability Using Information-Gap Theory , 2006, IEEE Transactions on Neural Networks.

[4]  S. Haykin.  Kalman Filtering and Neural Networks , 2001 .

[5]  Wei-Liem Loh.  On Latin hypercube sampling , 1996 .

[6]  Chilukuri K. Mohan,et al.  Robustness of feedforward neural networks , 1993, IEEE International Conference on Neural Networks.

[7]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[8]  A. N. Michel,et al.  Robustness analysis of a class of discrete-time recurrent neural networks under perturbations , 1999 .

[9]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[10]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  F. Scotti,et al.  A training-time analysis of robustness in feed-forward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[12]  Duc Truong Pham,et al.  Adaptive control of dynamic systems using neural networks , 1993, Proceedings of IEEE Systems Man and Cybernetics Conference - SMC.

[13]  Mo-Yuen Chow,et al.  Robustness of an induction motor incipient fault detector neural network subject to small input perturbations , 1991, IEEE Proceedings of the SOUTHEASTCON '91.

[14]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[15]  A. Assoum,et al.  Influence on ANNs fault tolerance of binary errors introduced during training , 2004, Proceedings. 2004 International Conference on Information and Communication Technologies: From Theory to Applications, 2004..

[16]  Ulrich Rückert,et al.  Robustness of radial basis functions , 2005, Neurocomputing.

[17]  Sharad Singhal,et al.  Training feed-forward networks with the extended Kalman algorithm , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[18]  Derong Liu,et al.  Robustness analysis and design of a class of neural networks with sparse interconnecting structure , 1996, Neurocomputing.

[19]  Günhan Dündar,et al.  The effects of quantization on multilayer neural networks , 1995, IEEE Trans. Neural Networks.

[20]  Kalmanje KrishnaKumar.  Optimization of the neural net connectivity pattern using a backpropagation algorithm , 1993, Neurocomputing.

[21]  Derong Liu,et al.  Robustness analysis of a class of neural networks , 1993, Proceedings of 36th Midwest Symposium on Circuits and Systems.

[22]  Dhananjay S. Phatak,et al.  Complete and partial fault tolerance of feedforward neural nets , 1995, IEEE Trans. Neural Networks.

[23]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.

[24]  Rudolph van der Merwe,et al.  The Unscented Kalman Filter , 2002 .

[25]  Wen Wang,et al.  Design of neural network-based estimator for tool wear modeling in hard turning , 2008, J. Intell. Manuf..

[26]  Cesare Alippi,et al.  A poly-time analysis of robustness in feedforward neural networks , 2001, VIMS 2001. 2001 IEEE International Workshop on Virtual and Intelligent Measurement Systems (IEEE Cat. No.01EX447).

[27]  K. Krishnakumar,et al.  ROBUSTNESS ANALYSIS OF NEURAL NETWORKS WITH AN APPLICATION TO SYSTEM IDENTIFICATION , 1999 .

[28]  P.B. Luh,et al.  Neural network-based market clearing price prediction and confidence interval estimation with an improved extended Kalman filter method , 2005, IEEE Transactions on Power Systems.

[29]  Kai Chang,et al.  Nonlinear statistical modeling and yield estimation technique for use in Monte Carlo simulations [microwave devices and ICs] , 2000 .

[30]  R. Kala,et al.  A Bound for the Euclidean Norm of the Difference Between the Least Squares and the Best Linear Unbiased Estimators , 1978 .

[31]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .