Automatic multi-module neural network evolution in an artificial brain

A major problem in artificial brain building is the automatic construction and training of multi-module systems of neural networks. For example, consider a biological human brain, which has millions of neural nets. If an artificial brain is to have similar complexity, it is unrealistic to require that the training data set for each neural net must be specified explicitly by a human, or that interconnections between evolved nets be performed manually. In this paper we present an original technique to solve this problem. A single large-scale task (too complex to be performed by a single neural net) is automatically split into simpler sub-tasks. A multi-module system of neural nets is then trained so that one of these sub-tasks is performed by each net. We present the results of an experiment using this novel technique for pattern recognition.

[1]  P. Rao Statistical Research Methods in the Life Sciences , 1997 .

[2]  Jim Tørresen,et al.  A Divide-and-Conquer Approach to Evolvable Hardware , 1998, ICES.

[3]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[4]  Tatiana Kalganova,et al.  Bidirectional incremental evolution in extrinsic evolvable hardware , 2000, Proceedings. The Second NASA/DoD Workshop on Evolvable Hardware.

[5]  Nathan Intrator,et al.  Optimal ensemble averaging of neural networks , 1997 .

[6]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[7]  Amanda J. C. Sharkey,et al.  Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems , 1999 .

[8]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[9]  Hugo de Garis,et al.  CoDi-1Bit: A Simplified Cellular Automata Based Neuron Model , 1997, Artificial Evolution.

[10]  Hugo de Garis Special issue on Evolutionary Neural Systems , 2002, Neurocomputing.

[11]  Robert E. Schapire,et al.  The strength of weak learnability , 1990, Mach. Learn..

[12]  Benjamin W. Wah,et al.  Artificial Neural Networks: Concepts and Theory , 1990 .

[13]  Lakhmi C. Jain,et al.  Recurrent Neural Networks: Design and Applications , 1999 .

[14]  Hugo de Garis,et al.  "TiPo" - a "Timed pointer" neural net model with superior evolvabilities for implementation in a second-generation brain-building machine BM2 , 2002, Proceedings 2002 NASA/DoD Conference on Evolvable Hardware.

[15]  J. Chimka Bootstrap Methods: A Practitioner's Guide , 2003 .

[16]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.