An Evolved Recurrent Neural Network and Its Application

An evolved recurrent neural network is proposed which automates the design of the network architecture and the connection weights using a new evolutionary learning algorithm. This new algorithm is based on a cooperative system of evolutionary algorithm (EA) and particle swarm optimisation (PSO), and is thus called REAPSO. In REAPSO, the network architecture is adaptively adjusted by PSO, and then EA is employed to evolve the connection weights with this network architecture, and this process is alternated until the best neural network is accepted or the maximum number of generations has been reached. In addition, the strategy of EAC and ET are proposed to maintain the behavioral link between a parent and its offspring, which improves the efficiency of evolving recurrent neural networks. A recurrent neural network is evolved by REAPSO and applied to the state estimation of the CSTR System. The performance of REAPSO is compared to TDRB, GA, PSO and HGAPSO in these recurrent networks design problems, demonstrating its superiority.

[1]  W. M. Jenkins,et al.  Genetic Algorithms and Neural Networks , 1999, Neural Networks in the Analysis and Design of Structures.

[2]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[3]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[4]  Kwang Y. Lee,et al.  Diagonal recurrent neural networks for dynamic systems control , 1995, IEEE Trans. Neural Networks.

[5]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[6]  Chia-Feng Juang,et al.  A hybrid of genetic algorithm and particle swarm optimization for recurrent network design , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Armando Blanco,et al.  A genetic algorithm to obtain the optimal recurrent neural network , 2000, Int. J. Approx. Reason..

[8]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[9]  J. Stephen Judd,et al.  Neural network design and the complexity of learning , 1990, Neural network modeling and connectionism.

[10]  Andrzej Jaszkiewicz.  Comparison of local search-based metaheuristics on the multiple objective knapsack problem , 2001 .

[11]  Xin Yao,et al.  A review of evolutionary artificial neural networks , 1993, Int. J. Intell. Syst..

[12]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[13]  Ah Chung Tsoi,et al.  Rule inference for financial prediction using recurrent neural networks , 1997, Proceedings of the IEEE/IAFE 1997 Computational Intelligence for Financial Engineering (CIFEr).

[14]  Jia Lei,et al.  The state estimation of the CSTR system based on a recurrent neural network trained by HGAs , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[15]  Neil Burgess,et al.  A Constructive Algorithm that Converges for Real-Valued Input Patterns , 1994, Int. J. Neural Syst..

[16]  F. Heimes,et al.  Traditional and evolved dynamic neural networks for aircraft simulation , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[17]  Barak A. Pearlmutter.  Learning State Space Trajectories in Recurrent Neural Networks , 1989, Neural Computation.

[18]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.

[19]  Eros Pasero,et al.  Short term load forecasting using a synchronously operated recurrent neural network , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[20]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[21]  Geoffrey E. Hinton,et al.  A time-delay neural network architecture for isolated word recognition , 1990, Neural Networks.

[22]  Nian Zhang,et al.  Time series prediction with recurrent neural networks trained by a hybrid PSO-EA algorithm , 2004, Neurocomputing.

[23]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.