Effects of static and dynamic disorder on the performance of neural automata.

We report on both analytical and numerical results concerning stochastic Hopfield-like neural automata exhibiting the following (biologically inspired) features: (1) Neurons and synapses evolve in time as in contact with respective baths at different temperatures; (2) the connectivity between neurons may be tuned from full connection to high random dilution, or to the case of networks with the small-world property and/or scale-free architecture; and (3) there is synaptic kinetics simulating repeated scanning of the stored patterns. Although these features may apparently result in additional disorder, the model exhibits, for a wide range of parameter values, an extraordinary computational performance, and some of the qualitative behaviors observed in natural systems. In particular, we illustrate here very efficient and robust associative memory, and jumping between pattern attractors.

[1]  Hilbert J. Kappen,et al.  Associative Memory with Dynamic Synapses , 2002, Neural Computation.

[2]  Joaquín J. Torres,et al.  Switching between memories in neural automata with synaptic noise , 2004, Neurocomputing.

[3]  G. Laurent,et al.  Odor encoding as an active, dynamical process: experiments, computation, and theory. , 2001, Annual review of neuroscience.

[4]  S. N. Dorogovtsev,et al.  Evolution of networks , 2001, cond-mat/0106144.

[5]  R. Dickman,et al.  Nonequilibrium Phase Transitions in Lattice Models , 1999 .

[6]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[7]  Michael Menzinger,et al.  Topology and computational performance of attractor neural networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[8]  A.C.C. Coolen,et al.  Chapter 15 Statistical mechanics of recurrent neural networks II — Dynamics , 2000, cond-mat/0006011.

[9]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[10]  W. Little.  The existence of persistent states in the brain , 1974 .

[11]  S. Shinomoto.  Memory maintenance in neural networks , 1987 .

[12]  Ramón Huerta,et al.  Dynamical encoding by networks of competing neuron groups: winnerless competition. , 2001 .

[13]  Joaquín J. Torres,et al.  Influence of topology on the performance of a neural network , 2004, Neurocomputing.

[14]  Sherrington,et al.  Coupled dynamics of fast spins and slow interactions: An alternative perspective on replicas. , 1993, Physical review. B, Condensed matter.

[15]  Daniel J. Amit,et al.  Modeling brain function: the world of attractor neural networks, 1st Edition , 1989 .

[16]  Frisch,et al.  Lattice gas automata for the Navier-Stokes equations. a new approach to hydrodynamics and turbulence , 1989 .

[17]  Stephen Wolfram,et al.  Cellular automata as models of complexity , 1984, Nature.

[18]  B. M. Fulk.  MATH , 1992 .

[19]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[20]  H. W. Veen,et al.  Handbook of Biological Physics , 1996 .

[21]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[22]  Y. Pomeau,et al.  Lattice-gas automata for the Navier-Stokes equation. , 1986, Physical review letters.

[23]  Joaquín J. Torres,et al.  Neural networks with fast time-variation of synapses , 1997 .

[24]  L. da Fontoura Costa,et al.  Efficient Hopfield pattern recognition on a scale-free neural network , 2002, cond-mat/0212601.

[25]  R. W. Penney,et al.  Coupled dynamics of fast spins and slow interactions in neural networks and spin systems , 1993 .

[26]  J. J. Torres,et al.  Effect of Correlated Fluctuations of Synapses in the Performance of Neural Networks , 1998 .

[27]  Amir Ayali,et al.  Morphological characterization of in vitro neuronal networks. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[28]  Horn,et al.  Neural networks with dynamical thresholds. , 1989, Physical review. A, General physics.

[29]  A. Coolen.  Statistical Mechanics of Recurrent Neural Networks I. Statics , 2000, cond-mat/0006010.

[30]  TamGas Geszti,et al.  Physical Models of Neural Networks , 1990 .