GANNET: a genetic algorithm for searching topology and weight spaces in neural network design. The first step in finding a neural network solution

Neural networks have recently generated a great amount of interest. However, designing and training these networks are very difficult tasks, requiring much trial and error, and the standard layered network architectures do not map naturally into hardware. This dissertation presents a novel genetic algorithm which is used to design the topology and find the link weights for a layered, feedforward neural network. Since the algorithm will respect given fan-in and fan-out limits for each node, the networks will easily map into hardware. The topologies are not limited either in the number of layers or in the number of nodes per layer, and two transfer functions are available (sigmoid and Gaussian). A robust, global search is conducted by the genetic algorithm over both the link weight and topology spaces, after which a local search strategy (such as back propagation) can be used to quickly find the desired link weights. Thus, both the genetic algorithm and back propagation can be used to their greatest advantage: the global search of the GA can find the approximate area of a solution, and the back propagation can then quickly find the local optimum.

[1]  Masafumi Hagiwara.  Novel backpropagation algorithm for reduction of hidden units and acceleration of convergence using artificial selection , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[2]  Francesco Palmieri,et al.  Analyses of the genetic algorithms in the continuous space , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[3]  Darrell Whitley,et al.  Optimizing small neural networks using a distributed genetic algorithm , 1990 .

[4]  Yoshio Hirose,et al.  Backpropagation algorithm which varies the number of hidden units , 1989, International 1989 Joint Conference on Neural Networks.

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  Tariq Samad,et al.  Towards the Genetic Synthesisof Neural Networks , 1989, ICGA.

[7]  K. Murase,et al.  A backpropagation algorithm which automatically determines the number of association units , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[8]  N. Dodd,et al.  Optimisation of network structure using genetic techniques , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[9]  Reiko Tanese,et al.  Distributed Genetic Algorithms , 1989, ICGA.

[10]  Sheng-De Wang,et al.  A self growing learning algorithm for determining the appropriate number of hidden units , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[11]  Huan Liu,et al.  Self-generating neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[12]  John R. Koza,et al.  Genetic generation of both the weights and architecture for a neural network , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[13]  L. Darrell Whitley,et al.  Genetic algorithms and neural networks: optimizing connections and connectivity , 1990, Parallel Comput..

[14]  Dit-Yan Yeung.  Automatic determination of network size for supervised learning , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[15]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[16]  Marcus Frean,et al.  The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks , 1990, Neural Computation.

[17]  L. Darrell Whitley,et al.  The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best , 1989, ICGA.

[18]  John M. Zelle,et al.  Growing layers of perceptrons: introducing the Extentron algorithm , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[19]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.

[20]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[21]  Larry J. Eshelman,et al.  Preventing Premature Convergence in Genetic Algorithms by Preventing Incest , 1991, ICGA.

[22]  L. A. Akers,et al.  A limited-interconnect synthetic neural IC , 1988, IEEE 1988 International Conference on Neural Networks.

[23]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .