Minimum-entropy coding with Hopfield networks

The aim of minimum-entropy coding is to find representative elements that occur as nearly as possible independently of each other, so that the probabilities of all input states can be calculated approximately from the probabilities of these elements. Finding predictive and other associations is an extremely important task for the real nervous system, and a representation of objects and events in the environment based on minimum-entropy coding would greatly improve the versatility and efficiency with which this task could be performed. It is shown in this paper how it can be achieved using Hopfield-type networks.

[1]  F. Attneave.  Some informational aspects of visual perception. , 1954, Psychological review.

[2]  Arne Zettersten,et al.  A word-frequency list based on American English press reportage , 1978 .

[3]  Satosi Watanabe,et al.  Pattern recognition as a quest for minimum entropy , 1981, Pattern Recognit..

[4]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Satosi Watanabe,et al.  Pattern Recognition: Human and Mechanical , 1985 .

[6]  Giorgio Parisi,et al.  Spin Glass Theory , 1987 .

[7]  H. B. Barlow,et al.  Finding Minimum Entropy Codes , 1989, Neural Computation.