Neural Network Trainer with Second Order Learning Algorithms

Although neural networks have been around for over 20 years, we still have difficulties training them. Training is often difficult and time consuming. The paper describes a software (NNT) developed for neural network training. In addition to the traditional Error Back Propagation (EBP) algorithm, several second order algorithms were implemented. These algorithms are modifications of the Levenberg Marquet algorithm and they are able to train arbitrarily connected feedforward neural networks. In most cases the training process is more than 100 times faster than EBP training. These algorithms can also find solutions for very difficult networks where the EBP algorithm fails.

[1]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[2]  Seul Jung,et al.  Hardware Implementation of a Real-Time Neural Network Controller With a DSP and an FPGA for Nonlinear Systems , 2007, IEEE Transactions on Industrial Electronics.

[3]  G. Dundar,et al.  Merge of Evolutionary Computation with Gradient Based Method for Optimization Problems , 2007, 2007 IEEE International Symposium on Industrial Electronics.

[4]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[5]  Mark J. Embrechts,et al.  Hybrid identification of nuclear power plant transients with artificial neural networks , 2004, IEEE Transactions on Industrial Electronics.

[6]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[7]  Bimal K. Bose,et al.  Recurrent-neural-network-based implementation of a programmable cascaded low-pass filter used in stator flux synthesis of vector-controlled induction motor drive , 1999, IEEE Trans. Ind. Electron..

[8]  T.,et al.  Training Feedforward Networks with the Marquardt Algorithm , 2004 .

[9]  B.M. Wilamowski,et al.  Method of computing gradient vector and Jacobean matrix in arbitrarily connected neural networks , 2007, 2007 IEEE International Symposium on Industrial Electronics.

[10]  Bimal K. Bose,et al.  Neural Network Applications in Power Electronics and Motor Drives—An Introduction and Perspective , 2007, IEEE Transactions on Industrial Electronics.

[11]  Bogdan M. Wilamowski,et al.  Solving parity-N problems with feedforward neural networks , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[12]  B. Karanayil,et al.  Online Stator and Rotor Resistance Estimation Scheme Using Artificial Neural Networks for Vector Controlled Speed Sensorless Induction Motor Drive , 2007, IEEE Transactions on Industrial Electronics.

[13]  Bogdan M. Wilamowski,et al.  Neural Networks and Fuzzy Systems , 2018, Microelectronics.

[14]  Moses Garuba,et al.  Implementation of Artificial Neural Network-Based Tracking Controller for High-Performance Stepper Motor Drives , 2007, IEEE Transactions on Industrial Electronics.