Task-dependent evolution of modularity in neural networks

There exist many ideas and assumptions about the development and meaning of modularity in biological and technical neural systems. We empirically study the evolution of connectionist models in the context of modular problems. For this purpose, we define quantitative measures for the degree of modularity and monitor them during evolutionary processes under different constraints. It turns out that the modularity of the problem is reflected by the architecture of adapted systems, although learning can counterbalance some imperfection of the architecture. The demand for fast learning systems increases the selective pressure towards modularity.

[1]  Terry Bossomaier,et al.  MONSTER - the Ghost in the Connection Machine: Modularity of Neural Systems in Theoretical Evolutionary Research , 1995, SC.

[2]  Marc Toussaint,et al.  On model selection and the disability of neural networks to decompose tasks , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[3]  Amanda J. C. Sharkey,et al.  On Combining Artificial Neural Nets , 1996, Connect. Sci..

[4]  S. Kosslyn,et al.  Why are What and Where Processed by Separate Cortical Visual Systems? A Computational Investigation , 1989, Journal of Cognitive Neuroscience.

[5]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[6]  Christian Igel,et al.  Empirical evaluation of the improved Rprop learning algorithms , 2003, Neurocomputing.

[7]  Chi-Hang Lam,et al.  FORMATION AND DYNAMICS OF MODULES IN A DUAL-TASKING MULTILAYER FEED-FORWARD NEURAL NETWORK , 1998 .

[8]  Stefano Nolfi,et al.  Learning and Evolution , 1999, Auton. Robots.

[9]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .

[10]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[11]  Marc Toussaint,et al.  Self-adaptive exploration in evolutionary search , 2001, ArXiv.

[12]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[13]  Frédéric Gruau,et al.  Automatic Definition of Modular Neural Networks , 1994, Adapt. Behav..

[14]  Bernhard Sendhoff,et al.  Variable encoding of modular neural networks for time series prediction , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[15]  J. Fodor,et al.  The Modularity of Mind: An Essay on Faculty Psychology , 1984 .

[16]  C. Igel,et al.  Structure optimization and isomorphisms , 2001 .

[17]  HERBERT A. SIMON,et al.  The Architecture of Complexity , 1991 .

[18]  Amanda J. C. Sharkey,et al.  Modularity, Combining and Artificial Neural Nets , 1997, Connect. Sci..

[19]  Claudio Moraga,et al.  An Evolutionary Method to Find Good Building-Blocks for Architectures of Artificial Neural Networks , 1996 .

[20]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[21]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[22]  J. Elman,et al.  Rethinking Innateness: A Connectionist Perspective on Development , 1996 .

[23]  Michael I. Jordan,et al.  Modular and hierarchical learning systems , 1998 .

[24]  Domenico Parisi,et al.  Evolving Modular Architectures for Neural Networks , 2000, NCPW.

[25]  John A. Bullinaria,et al.  Simulating the Evolution of Modular Neural Systems , 2001 .

[26]  T. Bossomaier,et al.  MONSTER — The Ghost in the Connection Machine: Modularity of Neural Systems in Theoretical Evolutionary Research , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[27]  Christian Igel,et al.  Improving the Rprop Learning Algorithm , 2000 .

[28]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[29]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .

[30]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[31]  Marc Toussaint,et al.  A neural model for multi-expert architectures , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[32]  Bernhard Sendhoff,et al.  Evolution of structures: optimization of artificial neural structures for information processing , 1998 .

[33]  M. Husken,et al.  Optimization for problem classes-neural networks that learn to learn , 2000, 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (Cat. No.00.

[34]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.