Performance of generalized multilayered perceptons trained using the Levenberg-Marquardt method

The generalized multilayer perceptron (gMLP) generalizes the multilayered perceptron (MLP) architecture to a fully connected feedforward architecture where connections are not restricted to adjacent layers. In this paper the performance of MLP and gMLP networks trained using the Levenberg-Marquardt method are compared. A number of different function approximation tasks were examined. The effect of varying the number of hidden layer neurons was also evaluated.

[1]  B.M. Wilamowski,et al.  Method of computing gradient vector and Jacobean matrix in arbitrarily connected neural networks , 2007, 2007 IEEE International Symposium on Industrial Electronics.

[2]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[3]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[4]  Martin T. Hagan,et al.  Neural network design , 1995 .

[5]  X. Pang,et al.  Neural network design for J function approximation in dynamic programming , 1998, adap-org/9806001.

[6]  Bogdan M. Wilamowski,et al.  Solving parity-N problems with feedforward neural networks , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[7]  Okyay Kaynak,et al.  Computing Gradient Vector and Jacobian Matrix in Arbitrarily Connected Neural Networks , 2008, IEEE Transactions on Industrial Electronics.

[8]  P.J. Werbos,et al.  Efficient Learning in Cellular Simultaneous Recurrent Neural Networks - The Case of Maze Navigation Problem , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[9]  Robert Kozma,et al.  Beyond Feedforward Models Trained by Backpropagation: A Practical Training Tool for a More Efficient Universal Approximator , 2007, IEEE Transactions on Neural Networks.

[10]  P. J. Werbos,et al.  Generalized maze navigation: SRN critics solve what feedforward or Hebbian nets cannot , 1996, 1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929).

[11]  B.M. Wilamowski,et al.  Neural Network Trainer with Second Order Learning Algorithms , 2007, 2007 11th International Conference on Intelligent Engineering Systems.

[12]  Donald C. Wunsch,et al.  The cellular simultaneous recurrent network adaptive critic design for the generalized maze problem has a simple closed-form solution , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[13]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[14]  Bogdan M. Wilamowski.  Neural network architectures and learning , 2003, IEEE International Conference on Industrial Technology, 2003.