Generalized net model for parallel optimization of multilayer perceptron with momentum backpropagation algorithm

In this paper we used a generalized net which gives a possibility for parallel optimization of multilayer neural networks. For training the backpropagation algorithm with momentum was considered. We proposed a generalized net model of parallel training of two neural networks with different architectures. The difference between the networks is in the number of neurons in main difference of the neural networks architectures is the numbers of neurons in hidden layers. In result we can obtain optimal neural network architecture.

[1]  M. Krawczak,et al.  Generalized net model for parallel optimization of feed-forward neural network with variable learning rate backpropagation algorithm , 2008, 2008 4th International IEEE Conference Intelligent Systems.

[2]  Danilo P. Mandic,et al.  Towards the Optimal Learning Rate for Backpropagation , 2000, Neural Processing Letters.

[3]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[4]  D. G. Sotiropoulos,et al.  A NEW EFFICIENT VARIABLE LEARNING RATE FOR PERRY’S SPECTRAL CONJUGATE GRADIENT TRAINING METHOD , 2004 .

[5]  A. K. Rigler,et al.  Accelerating the convergence of the back-propagation method , 1988, Biological Cybernetics.

[6]  Hani Hagras,et al.  FPGA implementation of spiking neural networks - an initial step towards building tangible collaborative autonomous agents , 2004, Proceedings. 2004 IEEE International Conference on Field- Programmable Technology (IEEE Cat. No.04EX921).

[7]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[8]  Sotir Sotirov.  A Method of Accelerating Neural Network Learning , 2005, Neural Processing Letters.

[9]  Krassimir T. Atanassov,et al.  Generalized nets , 1991 .

[10]  Vassilis P. Plagianakos,et al.  Automatic Adaptation of Learning Rate for Backpropagation Neural Networks , 1998 .

[11]  Martin T. Hagan,et al.  Neural network design , 1995 .

[12]  D. G. Sotiropoulos,et al.  A SPECTRAL VERSION OF PERRY'S CONJUGATE GRADIENT METHOD FOR NEURAL NETWORK TRAINING , 2002 .

[13]  Yutaka Maeda,et al.  FPGA implementation of a pulse density neural network with learning ability using simultaneous perturbation , 2003, IEEE Trans. Neural Networks.

[14]  Francisco José Ballester-Merelo,et al.  Artificial neural network implementation on a single FPGA of a pipelined on-line backpropagation , 2000, Proceedings 13th International Symposium on System Synthesis.