The evolution of modular artificial neural networks

This thesis describes a novel approach to the evolution of Modular Artificial Neural Networks. Standard Evolutionary Algorithms, used in this application include: Genetic Algorithms, Evolutionary Strategies, Evolutionary Programming and Genetic Programming; however, these often fail in the evolution of complex systems, particularly when such systems involve multi-domain sensory information which interacts in complex ways with system outputs. The aim in this work is to produce an evolutionary method that allows the structure of the network to evolve from simple to complex as it interacts with a dynamic environment. This new algorithm is therefore based on Incremental Evolution. A simulated model of a legged robot was used as a test-bed for the approach. The algorithm starts with a simple robotic body plan. This then grows incrementally in complexity along with its controlling neural network and the environment it reacts with. The network grows by adding modules to its structure - so the technique may also be termed a Growth Algorithm. Experiments are presented showing the successful evolution of multi-legged gaits and a simple vision system. These are then integrated together to form a complete robotic system. The possibility of the evolution of complex systems is one advantage of the algorithm and it is argued that it represents a possible path towards more advanced artificial intelligence. Applications in Electronics, Computer Science, Mechanical Engineering and Aerospace are also discussed.

[1]  Roberto Battiti,et al.  Democracy in neural nets: Voting schemes for classification , 1994, Neural Networks.

[2]  L. Darrell Whitley,et al.  Adding Learning to the Cellular Development of Neural Networks: Evolution and the Baldwin Effect , 1993, Evolutionary Computation.

[3]  Albrecht Schmidt,et al.  MODULARITY - A CONCEPT FOR NEW NEURAL NETWORK ARCHITECTURES , 1998, WWW 1998.

[4]  Takayuki Ito,et al.  Neocognitron: A neural network model for a mechanism of visual pattern recognition , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[5]  John R. Koza,et al.  Genetic generation of both the weights and architecture for a neural network , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[6]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[7]  Cardiff E. M. Smith.  The Central Nervous System: Structure and Function , 2004 .

[8]  Michael I. Jordan,et al.  Modular and hierarchical learning systems , 1998 .

[9]  R. P. J. Perazzo,et al.  Asymptotic Inferential Capabilities of Feed-Forward Neural Networks , 1991 .

[10]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[11]  T. Gomi,et al.  Evolution of gaits of a legged robot , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[12]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[13]  D. Parisi,et al.  Growing neural networks , 1991 .

[14]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[15]  Christopher MacLeod.  The synthesis of artificial neural networks using single string evolutionary techniques , 1999 .

[16]  David McMinn,et al.  Evolution and devolved action: towards the evolution of systems. , 2001 .

[17]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[18]  Robert F. Harrison,et al.  Optimization and training of feedforward neural networks by genetic algorithms , 1991 .

[19]  Claudio A. Perez,et al.  Improvements on handwritten digit recognition by cooperation of modular neural networks , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[20]  Karl Sims,et al.  Evolving 3d morphology and behavior by competition , 1994 .

[21]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[22]  Bernd Fritzke,et al.  Fast learning with incremental RBF networks , 1994, Neural Processing Letters.

[23]  G. M. Maxwell,et al.  An Evolutionary Artificial Nervous System for Animat Locomotion , 2000 .

[24]  Peter J. Bentley,et al.  Digital Biology: The Creation of Life Inside Computers and How It Will Affect Us , 2002 .

[25]  Andy J. Keane.  The design of a satellite boom with enhanced vibration performance using genetic algorithm techniques , 1996 .

[26]  Karl E. Kürten,et al.  Pattern-specific neural network design , 1995 .

[27]  G. Maxwell,et al.  Using Embryology as an Alternative to Genetic Algorithms for Designing Artificial Neural Network Topologies , 1997, ICANNGA.

[28]  Jari Vaario,et al.  From Evolutionary Computation to Computational Evolution , 1994, Informatica.

[29]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[30]  A. Cangelosi,et al.  Cell division and migration in a 'genotype' for neural networks (Cell division and migration in neural networks) , 1993 .

[31]  J. Ewert.  Neuroethology of releasing mechanisms: Prey-catching in toads , 1987, Behavioral and Brain Sciences.

[32]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.

[33]  H. D. Garis.  CAM-BRAIN — The Evolutionary Engineering of a Billion Neuron Artificial Brain , 1999 .

[34]  S. D. Prentice,et al.  Modelling the time-keeping function of the central pattern generator for locomotion using artificial sequential neural network , 1995, Medical and Biological Engineering and Computing.

[35]  William Francis Ganong,et al.  Review of Medical Physiology , 1969 .

[36]  Kevin Warwick,et al.  MUTUAL LEARNING BY AUTONOMOUS MOBILE ROBOTS , 1997 .

[37]  David McMinn.  Using evolutionary artificial neural networks to design hierarchical animat nervous systems , 2001 .

[38]  Richard M. Restak,et al.  The Brain: The Last Frontier , 1980 .

[39]  D. L. Reilly,et al.  A neural model for category learning , 1982, Biological Cybernetics.

[40]  Christopher Macleod,et al.  AN INNOVATIVE NEURAL NETWORK BASED ON THE TOAD'S VISUAL SYSTEM , 2002 .

[41]  Frank Dellaert,et al.  Toward an evolvable model of development for autonomous agent synthesis , 1994 .

[42]  M. Fujita,et al.  Evolution of dynamic gaits for a robot , 2000, 2000 Digest of Technical Papers. International Conference on Consumer Electronics. Nineteenth in the Series (Cat. No.00CH37102).

[43]  Lakhmi C. Jain,et al.  Automatic Generation of Neural Network Architecture Using Evolutionary Computation , 1997, Advances in Fuzzy Systems - Applications and Theory.

[44]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[45]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[46]  Christopher Macleod,et al.  An approach to evolvable neural functionality. , 2003 .