Partial annealing and overfrustration in disordered systems

We study disordered systems with the replica method keeping the number of replicas finite and negative. This is shown to bias the distribution of samples towards overfrustrated ones. General results on the thermodynamics of such a system is presented. The physical situation described by this finite-n approach is one where the usually quenched variables evolve on long timescales, their evolution being driven by the quasi-equilibrium correlations of the thermalized variables. In the case of neural networks this amounts to a coupled dynamics of neurons (on fast timescales) and synapses (on longer timescales). The storage capacity of the Hopfield model is shown to be substantially increased by these coupled dynamics.

[1]  S. Kirkpatrick,et al.  Solvable Model of a Spin-Glass , 1975 .

[2]  Giorgio Parisi,et al.  Toward a mean field theory for spin glasses , 1979 .

[3]  G. Parisi.  A sequence of approximated solutions to the S-K model for spin glasses , 1980 .

[4]  D. Sherrington.  Ising replica magnets , 1980 .

[5]  B. Derrida.  Random-energy model: An exactly solvable model of disordered systems , 1981 .

[6]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[7]  I. Kondor.  Parisi's mean-field solution for spin glasses as an analytic continuation in the replica number , 1983 .

[8]  Giorgio Parisi,et al.  Order parameter for spin-glasses , 1983 .

[9]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[10]  D. Kleinfeld,et al.  "Unlearning" increases the storage capacity of content addressable memories. , 1987, Biophysical journal.

[11]  Kanter,et al.  Associative recall of memory without errors. , 1987, Physical review. A, General physics.

[12]  J. L. van Hemmen,et al.  Increasing the efficiency of a neural network through unlearning , 1990 .

[13]  A. Coolen,et al.  Unsupervised dynamic learning in layered neural networks , 1991 .

[14]  R. W. Penney,et al.  Coupled dynamics of fast spins and slow interactions in neural networks and spin systems , 1993 .

[15]  Complexity in the coupled dynamics of fast neurons and slow synapses , 1993 .