Noise and the Evolution of Neural Network Modularity

We study the selective advantage of modularity in artificially evolved networks. Modularity abounds in complex systems in the real world. However, experimental evidence for the selective advantage of network modularity has been elusive unless it has been supported or mandated by the genetic representation. The evolutionary origin of modularity is thus still debated: whether networks are modular because of the process that created them, or the process has evolved to produce modular networks. It is commonly argued that network modularity is beneficial under noisy conditions, but experimental support for this is still very limited. In this article, we evolve nonlinear artificial neural network classifiers for a binary classification task with a modular structure. When noise is added to the edge weights of the networks, modular network topologies evolve, even without representational support.

[1]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[2]  S. Strogatz.  Exploring complex networks , 2001, Nature.

[3]  P. Cluzel,et al.  Effects of topology on network evolution , 2006 .

[4]  G. Wagner,et al.  The road to modularity , 2007, Nature Reviews Genetics.

[5]  M. Newman,et al.  On the uniform generation of random graphs with prescribed degree sequences , 2003, cond-mat/0312028.

[6]  Almut Schüz.  Neuroanatomy in a computational perspective , 1998 .

[7]  J. Bullinaria.  To Modularize or Not To Modularize ? , 2002 .

[8]  Günter P. Wagner,et al.  Complex Adaptations and the Evolution of Evolvability , 2005 .

[9]  Michael W Deem,et al.  Spontaneous emergence of modularity in a model of evolving individuals and in real networks. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  Jordan B. Pollack,et al.  An Endosymbiotic Model for Modular Acquisition in Stochastic Developmental Systems , 1996 .

[11]  S. Shen-Orr,et al.  Network motifs: simple building blocks of complex networks. , 2002, Science.

[12]  Hod Lipson,et al.  ON THE ORIGIN OF MODULAR VARIATION , 2002, Evolution; international journal of organic evolution.

[13]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[14]  N. Packard,et al.  A classification of long-term evolutionary dynamics , 1998 .

[15]  J. Pollack,et al.  Coevolving High-Level Representations , 1993 .

[16]  S. Kosslyn,et al.  Why are What and Where Processed by Separate Cortical Visual Systems? A Computational Investigation , 1989, Journal of Cognitive Neuroscience.

[17]  R. Guimerà,et al.  Functional cartography of complex metabolic networks , 2005, Nature.

[18]  A.-L. Barabasi,et al.  The Architecture of Complexity , 2007, IEEE Control Systems.

[19]  Hod Lipson,et al.  Networks, dynamics, and modularity. , 2004, Physical review letters.

[20]  Michael W. Deem,et al.  Mathematical adventures in biology , 2007 .

[21]  J. Baldwin.  A New Factor in Evolution , 1896, The American Naturalist.

[22]  Michael I. Jordan,et al.  Computational Consequences of a Bias toward Short Connections , 1992, Journal of Cognitive Neuroscience.

[23]  Riccardo Poli,et al.  Evolving the Topology and the Weights of Neural Networks Using a Dual Representation , 2004, Applied Intelligence.

[24]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[25]  Keith L. Downing,et al.  Supplementing evolutionary developmental systems with abstract models of neurogenesis , 2007, GECCO '07.

[26]  HERBERT A. SIMON,et al.  The Architecture of Complexity , 1991 .

[27]  J. Hopfield,et al.  From molecular to modular cell biology , 1999, Nature.

[28]  E. Rolls,et al.  Neural networks and brain function , 1998 .

[29]  Min Shi,et al.  An empirical comparison of evolution and coevolution for designing artificial neural network game players , 2008, GECCO '08.

[30]  Peter Donnelly,et al.  Superfamilies of Evolved and Designed Networks , 2004 .

[31]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[32]  U. Alon,et al.  Spontaneous evolution of modularity and network motifs. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[33]  S. Shen-Orr,et al.  Networks Network Motifs : Simple Building Blocks of Complex , 2002 .

[34]  Josh Bongard,et al.  Evolving modular genetic regulatory networks , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[35]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex Adaptive Systems.

[36]  V. Mountcastle.  The columnar organization of the neocortex. , 1997, Brain : a journal of neurology.

[37]  Neo D. Martinez,et al.  Simple rules yield complex food webs , 2000, Nature.