Tuning of the structure and parameters of a neural network using an improved genetic algorithm

This paper presents the tuning of the structure and parameters of a neural network using an improved genetic algorithm (GA). It is also shown that the improved GA performs better than the standard GA based on some benchmark test functions. A neural network with switches introduced to its links is proposed. By doing this, the proposed neural network can learn both the input-output relationships of an application and the network structure using the improved GA. The number of hidden nodes is chosen manually by increasing it from a small number until the learning performance in terms of fitness value is good enough. Application examples on sunspot forecasting and associative memory are given to show the merits of the improved GA and the proposed neural network.

[1]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[2]  Xiufeng Wang,et al.  Neural network control of induction machines using genetic algorithm training , 1996, IAS '96. Conference Record of the 1996 IEEE Industry Applications Conference Thirty-First IAS Annual Meeting.

[3]  Lin-Ying Lai,et al.  Optimal design for fuzzy controllers by genetic algorithms , 1995, Proceedings IEEE Conference on Industrial Automation and Control Emerging Technology Applications.

[4]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[5]  Jacek M. Zurada,et al.  Sequential network construction for time series prediction , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[6]  Jean-Pierre Martens,et al.  On the initialization and optimization of multilayer perceptrons , 1994, IEEE Trans. Neural Networks.

[7]  Benjamin W. Wah,et al.  Automated learning for reducing the configuration of a feedforward neural network , 1996, IEEE Trans. Neural Networks.

[8]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[9]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (2nd, extended ed.) , 1994 .

[10]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[11]  Kishan G. Mehrotra,et al.  Sunspot numbers forecasting using neural networks , 1990, Proceedings. 5th IEEE International Symposium on Intelligent Control 1990.

[12]  Tamás D. Gedeon,et al.  Exploring constructive cascade networks , 1999, IEEE Trans. Neural Networks.

[13]  S. Okuma,et al.  Accelerated evolutionary computation using fitness estimation , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[14]  Chih-Hong Lin,et al.  Self-constructing fuzzy neural network speed controller for permanent-magnet synchronous motor drive , 2001, IEEE Trans. Fuzzy Syst..

[15]  Yi-Sheng Zhou,et al.  Optimal design for fuzzy controllers by genetic algorithms , 2000 .

[16]  Yamashita,et al.  Backpropagation algorithm which varies the number of hidden units , 1989 .

[17]  Marco P. Schoen,et al.  Intelligent optimization techniques, genetic algorithms, tabu search, simulated annealing, and neural networks, D. T. Pham and D. Karaboga, Springer: Berlin, Heidelberg, New York; Springer London: London, 2000, 302pp, ISBN 1‐85233‐028‐7 , 2005 .

[18]  Somnath Mukhopadhyay,et al.  A polynomial time algorithm for the construction and training of a class of multilayer perceptrons , 1993, Neural Networks.

[19]  H. Youlal,et al.  Fuzzy dynamic path planning using genetic algorithms , 2000 .

[20]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[21]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[22]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[23]  Lalit M. Patnaik,et al.  Genetic algorithms: a survey , 1994, Computer.

[24]  Bin-Da Liu,et al.  Design of adaptive fuzzy logic controller based on linguistic-hedge concepts and genetic algorithms , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[25]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[26]  S. Amin,et al.  Dynamic local search , 1997 .

[27]  Mark S. Nixon,et al.  Generating-shrinking algorithm for learning arbitrary classification , 1994, Neural Networks.

[28]  S. Bornholdt,et al.  General asymmetric neural networks and structure design by genetic algorithms: a learning rule for temporal patterns , 1993, Proceedings of IEEE Systems Man and Cybernetics Conference - SMC.

[29]  Khaled Belarbi,et al.  Genetic algorithm for the design of a class of fuzzy controllers: an alternative approach , 2000, IEEE Trans. Fuzzy Syst..

[30]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[31]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[32]  Chin-Teng Lin,et al.  Genetic Reinforcement Learning through Symbiotic Evolution for Fuzzy Controller Design , 2022 .

[33]  Luigi Fortuna,et al.  Soft computing for greenhouse climate control , 2000, IEEE Trans. Fuzzy Syst..

[34]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[35]  Michael C. Mozer,et al.  Using Relevance to Reduce Network Size Automatically , 1989 .

[36]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[37]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[38]  F.H.F. Leung,et al.  Tuning of the structure and parameters of neural network using an improved genetic algorithm , 2001, IECON'01. 27th Annual Conference of the IEEE Industrial Electronics Society (Cat. No.37243).