Sparse binary matrices as efficient associative memories

Associative memories are widely used devices which can be viewed as universal error-correcting decoders. Employing error-correcting code principles in these devices has allowed to greatly enhance their performance. In this paper we reintroduce a neural-based model using the formalism of linear algebra and extend its functionality, originally limited to erasure retrieval, to handle approximate inputs. In order to perform the retrieval, we use an iterative algorithm that provably converges. We then analyze the performance of the associative memory under the assumption of connection independence. We support our theoretical results with numerical simulations.

[1]  Hung-Hsiang Jonathan Chao,et al.  Next generation routers , 2002, Proc. IEEE.

[2]  Vincent Gripon,et al.  Reduced-complexity binary-weight-coded associative memories , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[4]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Amin Karbasi,et al.  Multi-level error-resilient neural networks , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[6]  Norman P. Jouppi,et al.  Improving direct-mapped cache performance by the addition of a small fully-associative cache and prefetch buffers , 1990, [1990] Proceedings. The 17th Annual International Symposium on Computer Architecture.

[7]  Vincent Gripon,et al.  A Massively Parallel Associative Memory Based on Sparse Neural Networks , 2013, ArXiv.

[8]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[9]  Chyuan Shiun Lin,et al.  The design of a rotating associative memory for relational database applications , 1976, TODS.

[10]  Vincent Gripon,et al.  Nearly-optimal associative memories based on distributed constant weight codes , 2012, 2012 Information Theory and Applications Workshop.

[11]  Evangelos P. Markatos,et al.  Improving the accuracy of network intrusion detection systems under load using selective packet discarding , 2010, EUROSEC '10.

[12]  Günther Palm,et al.  Iterative retrieval of sparsely coded associative memory patterns , 1996, Neural Networks.

[13]  Vincent Gripon,et al.  Sparse Neural Networks With Large Learning Diversity , 2011, IEEE Transactions on Neural Networks.

[14]  Vincent Gripon,et al.  A simple and efficient way to store many messages using neural cliques , 2011, 2011 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB).

[15]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.