Two-level learning algorithm for multilayer neural networks

A two-level learning algorithm that decomposes multilayer neural networks into a set of sub-networks is presented. Many popular optimization methods, such as conjugate-gradient and quasi-Newton methods, can be utilized to train these sub-networks. In addition, if the activation functions are hard-limiting functions, the multilayer neural networks can be trained by the perceptron learning rule in this two-level learning algorithm. Two experimental problems are given as examples for this algorithm.

[1]  F. Williams,et al.  A TWO-LEVEL DECOMPOSITION METHOD FOR SHAPE OPTIMIZATION OF STRUCTURES , 1997 .

[2]  Mukul Agarwal,et al.  Three Methods to Speed up the Training of Feedforward and Feedback Perceptrons , 1997, Neural Networks.

[3]  Farid U. Dowla,et al.  Backpropagation Learning for Multilayer Feed-Forward Neural Networks Using the Conjugate Gradient Method , 1991, Int. J. Neural Syst..

[4]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[5]  Jacek M. Zurada,et al.  Introduction to artificial neural systems , 1992 .

[6]  Singiresu S. Rao.  Engineering Optimization : Theory and Practice , 2010 .

[7]  C. Charalambous,et al.  Conjugate gradient algorithm for efficient training of artifi-cial neural networks , 1990 .

[8]  Patrick van der Smagt.  Minimisation methods for training feedforward neural networks , 1994, Neural Networks.

[9]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[10]  D. J. Toms,et al.  Training binary node feedforward neural networks by back propagation of error , 1990 .

[11]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[12]  Marcus R. Frean,et al.  A "Thermal" Perceptron Learning Rule , 1992, Neural Computation.

[13]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[14]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.

[15]  U. Kirsch,et al.  Two-level optimization of prestressed structures , 1997 .

[16]  Jasbir S. Arora,et al.  Introduction to Optimum Design , 1988 .

[17]  Roberto Battiti,et al.  BFGS Optimization for Faster and Automated Supervised Learning , 1990 .