An Internal Mechanism for Detecting Parasite Attractors in a Hopfield Network

This paper presents a built-in mechanism for automatic detection of prototypes (as opposed to parasite attractors) in a Hopfield network. It has a good statistical performance and avoids host computer overhead for this purpose. This mechanism is based on an internal coding of the prototypes during learning, using cyclic redundancy codes, and leads to an efficient implementation in VLSI. The immediate separation of prototypes from parasite attractors can be used to enhance the autonomy of the networks, aiming at hierarchical multinetwork structures. As an example, the use of such an architecture for the classification of handwritten digits is described.

[1]  S. Y. King.  Parallel architectures for artificial neural nets , 1988, [1988] Proceedings. International Conference on Systolic Arrays.

[2]  Michel Weinfeld,et al.  A Fully Digital Integrated CMOS Hopfield Network Including the Learning Algorithm , 1989 .

[3]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[4]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[5]  L. Personnaz,et al.  Collective computational properties of neural networks: New learning mechanisms. , 1986, Physical review. A, General physics.

[6]  P. Gallinari,et al.  Cooperation of neural nets and task decomposition , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[7]  S. Y. Kung,et al.  Parallel architectures for artificial neural nets , 1988, IEEE 1988 International Conference on Neural Networks.

[8]  Gérard Dreyfus,et al.  Handwritten digit recognition by neural networks with single-layer training , 1992, IEEE Trans. Neural Networks.

[9]  Gérard Dreyfus,et al.  Specification and implementation of a digital Hopfield-type associative memory with on-chip training , 1992, IEEE Trans. Neural Networks.

[10]  W. W. Peterson,et al.  Error-Correcting Codes. , 1962 .

[11]  Pierre Peretto,et al.  Stochastic Dynamics of Neural Networks , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[12]  Michel Weinfeld,et al.  Integrated artificial neural networks: components for higher level architectures with new properties , 1989, NATO Neurocomputing.

[13]  Jean-Dominique Gascuel.  Integration vlsi d'un reseau de hopfield de 64 neurones : modele formel, conception et realisation , 1991 .

[14]  Opper,et al.  Learning of correlated patterns in spin-glass networks by local learning rules. , 1987, Physical review letters.

[15]  Teuvo Kohonen,et al.  An introduction to neural computing , 1988, Neural Networks.

[16]  Yann LeCun,et al.  Tangent Prop - A Formalism for Specifying Selected Invariances in an Adaptive Network , 1991, NIPS.

[17]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[19]  M. Weinfeld,et al.  A digital CMOS fully connected neural network with in-circuit learning capability and automatic identification of spurious attractors , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.