A Generative Model for Attractor Dynamics

Attractor networks, which map an input space to a discrete output space, are useful for pattern completion. However, designing a net to have a given set of attractors is notoriously tricky; training procedures are CPU intensive and often produce spurious afuactors and ill-conditioned attractor basins. These difficulties occur because each connection in the network participates in the encoding of multiple attractors. We describe an alternative formulation of attractor networks in which the encoding of knowledge is local, not distributed. Although localist attractor networks have similar dynamics to their distributed counterparts, they are much easier to work with and interpret. We propose a statistical formulation of localist attract or net dynamics, which yields a convergence proof and a mathematical interpretation of model parameters.

[1]  C. R. Neto,et al.  Multivalley structure of attractor neural networks , 1997 .

[2]  Richard M. Golden,et al.  Probabilistic Characterization of Neural Model Computations , 1987, NIPS.

[3]  Michael I. Jordan,et al.  Mean Field Theory for Sigmoid Belief Networks , 1996, J. Artif. Intell. Res..

[4]  David S. Touretzky,et al.  The Role of the Hippocampus in Solving the Morris Water Maze , 1998, Neural Computation.

[5]  Mark S. Seidenberg,et al.  On the nature and scope of featural representations of word meaning. , 1997, Journal of experimental psychology. General.

[6]  B L McNaughton,et al.  Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model , 1997, The Journal of Neuroscience.

[7]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[8]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[9]  Donald W. Mathis,et al.  A computational theory of consciousness in cognition , 1998 .

[10]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[11]  S. Becker,et al.  Long-term semantic priming: a computational account and empirical evidence. , 1997, Journal of experimental psychology. Learning, memory, and cognition.

[12]  Donald W. Mathis,et al.  Conscious and unconscious perception: A computational theory , 1996 .

[13]  Walter J. Freeman,et al.  Reafference and Attractors in the Olfactory System During Odor Recognition , 1996, Int. J. Neural Syst..