Neural Network Training Using Unscented and Extended Kalman Filter

In the early 1940s, the pioneers of the field, McCulloch & Pitts [1], studied the characterization of neural activity, the neural events and the relationships between them. Such a neural network can be treated by a propositional logic. Others, like Hebb [2], were concerned with the adaptation laws involved in neural systems. Rosenblatt [3] coined the name Perceptron and devised an architecture which has subsequently received much attention. Minsky & Seymour [4] introduced a rigorous analysis of the Perceptron; they proved many properties and pointed out limitations of several models [5]. Multilayer perceptrons are still the most popular artificial neural net structures being used today. Hopfield [6] tried to contextualize the biological model of addressable memory of biological organisms in comparison to the integrated circuits of a computer system, thus generalizing the collective properties that can be used for both the neurons and the computer systems. According to Zhan & Wan [7], neural networks are good tools for working with approximation in some nonlinear systems in order to obtain a desired degree of accuracy in predicting or even in non-stationary processes in which changes occur constantly and fast. As these activities happen in real time, the weights of a neural network are adjusted adaptively. And because of their strong ability to learn, neural networks have been widely used in identifying and modeling nonlinear systems. Artificial Neural Networks (ANNs) are currently an additional tool which the engineer can use for a variety of purposes. Classification and regression are the most common tasks; however, control, modeling, prediction and forecasting are other common tasks undertaken. For more than three decades, the field of ANNs has been a focus for researchers. As a result, one of the outcomes has been the development of many different software tools used to train these kinds of networks, making the selection of an adequate tool difficult for a new user [8]. The conventional algorithm for Multilayered Neural Networks (MNN) is Back propagation (BP), which is an algorithm that uses the Steepest Decent method (SD), an extension of the Laplace method, which tries to approximate, as close as possible, to the points of recurring non-stationary problems, in which you want to predict an event at some future point. BP in neural networks works to adjust the weights of the neural network, minimizing the error between the inputs and expected outputs. Problems related to BP are that it does not

[1]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[2]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.

[3]  Jiri Pospichal,et al.  Neural Network Training with Extended Kalman Filter Using Graphics Processing Unit , 2008, ICANN.

[4]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[5]  Ronald J. Williams,et al.  Training recurrent networks using the extended Kalman filter , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[6]  Jianwei Wan,et al.  Neural network-aided adaptive unscented Kalman filter for nonlinear state estimation , 2006, IEEE Signal Processing Letters.

[7]  Kathleen A. Kramer,et al.  System identification using the neural-extended Kalman filter for state-estimation and controller modification , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[8]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Lee A. Feldkamp,et al.  Decoupled extended Kalman filter training of feedforward layered networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[10]  Fredrik Gustafsson,et al.  Some Relations Between Extended and Unscented Kalman Filters , 2012, IEEE Transactions on Signal Processing.

[11]  Darío Baptista,et al.  A survey of artificial neural network training tools , 2013, Neural Computing and Applications.

[12]  R. Adnan,et al.  New Artificial Neural Network and Extended Kalman Filter hybrid model of flood prediction system , 2013, 2013 IEEE 9th International Colloquium on Signal Processing and its Applications.

[13]  S. Haykin.  Kalman Filtering and Neural Networks , 2001 .

[14]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[15]  Edilson R. R. Kato,et al.  A new comparison of Kalman filtering methods for chaotic series , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[16]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[17]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[18]  Hideaki Sakai,et al.  A real-time learning algorithm for a multilayered neural network based on the extended Kalman filter , 1992, IEEE Trans. Signal Process..