Cooperative Co-evolution of Multilayer Perceptrons

A theoretical investigation of the dynamics of continuous Hopfield networks, in a modified formulation proposed by Abe, is undertaken. The fixed points are classified according to whether they lie inside or they are vertices of the unit hypercube. It is proved that interior equilibria are saddle points. Besides, a procedure is sketched that determines the basins of attraction of stable vertices. The calculations are completed for the two-neuron network. These results contribute to a solid foundation for these systems, needed for the study of practical problems such as local minima or convergence speed.

[1]  Jihoon Yang,et al.  Constructive Neural-Network Learning Algorithms for Pattern Classification , 2000 .

[2]  Tariq Samad,et al.  High-order Hopfield and Tank optimization networks , 1990, Parallel Comput..

[3]  Reinhold Huber,et al.  Evolving Topologies of Artificial Neural Networks Adapted to Image Processing Tasks , 1996 .

[4]  Ignacio Rojas,et al.  Statistical analysis of the parameters of a neuro-genetic algorithm , 2002, IEEE Trans. Neural Networks.

[5]  Juan Julián Merelo Guervós,et al.  G-Prop: Global optimization of multilayer perceptrons using GAs , 2000, Neurocomputing.

[6]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[7]  Shun-ichi Amari,et al.  New theorems on global convergence of some dynamical systems , 2001, Neural Networks.

[8]  Peter Tiño,et al.  Attractive Periodic Sets in Discrete-Time Recurrent Networks (with Emphasis on Fixed-Point Stability and Bifurcations in Two-Neuron Networks) , 2001, Neural Computation.

[9]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[10]  Lutz Prechelt,et al.  PROBEN 1 - a set of benchmarks and benchmarking rules for neural network training algorithms , 1994 .

[11]  Qiangfu Zhao.  Co-evolutionary learning of neural networks , 1998, J. Intell. Fuzzy Syst..

[12]  S. Abe.  Theories on the Hopfield neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[13]  P. Holmes,et al.  Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields , 1983, Applied Mathematical Sciences.

[14]  Mathukumalli Vidyasagar.  Minimum-seeking properties of analog neural networks with multilinear objective functions , 1995, IEEE Trans. Autom. Control..

[15]  Risto Miikkulainen,et al.  Forming Neural Networks Through Efficient and Adaptive Coevolution , 1997, Evolutionary Computation.

[16]  Juan Julián Merelo Guervós,et al.  Optimisation of Multilayer Perceptrons Using a Distributed Evolutionary Algorithm with SOAP , 2002, PPSN.

[17]  Michael Conrad,et al.  Combining evolution with credit apportionment: A new learning algorithm for neural nets , 1994, Neural Networks.

[18]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[19]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[20]  S. Abe.  Global convergence and suppression of spurious states of the Hopfield neural networks , 1993 .

[21]  Phil Husbands,et al.  Distributed Coevolutionary Genetic Algorithms for Multi-Criteria and Multi-Constraint Optimisation , 1994, Evolutionary Computing, AISB Workshop.

[22]  Paul T. Jackway,et al.  Co-operative Evolution of a Neural Classifier and Feature Subset , 1998, SEAL.

[23]  Yoshiyasu Takefuji,et al.  Artificial neural networks for four-coloring map problems and K-colorability problems , 1991 .

[24]  Gonzalo Joya,et al.  Hopfield neural networks for optimization: study of the different dynamics , 2002 .

[25]  Jan Paredis,et al.  Coevolutionary computation , 1995 .

[26]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.