Associative memory with occurrence statistics

Distributed associative memory architectures store data in multiple locations redundantly, and are thus robust to circuit irregularities and noise. In this paper we explain ways to store the occurrence statistics of vectors in a distributed memory. Sparse data vectors are used to maximize vector capacity, and binding of sparse vectors is used to combine elementary symbols into symbols that represent larger entities. The use of such statistics is demonstrated with an on-line learning example that uses redundancy reduction. We show that occurrence statistics can be represented with distributed Willshaw-type associative memories sing hardware counters, as well as symbolically using two-bit memory cells.