Regarding the temporal requirements of a hierarchical Willshaw network

In a recent communication, Sacramento and Wichert (2011) proposed a hierarchical retrieval prescription for Willshaw-type associative networks. Through simulation it was shown that one could make use of low resolution descriptor patterns to decrease the total time requirements of recalling a learnt association. However, such a method introduced a dependence on a set of new parameters which define the structure of the hierarchy. In this work we compute the expected retrieval time for the random neural activity regime which maximises the capacity of the Willshaw model and we study the task of finding the optimal hierarchy parametrisation with respect to the derived temporal expectation. Still in regard to this performance measure, we investigate some asymptotic properties of the algorithm.

[1]  J. Krebs,et al.  Optimal learning rules , 1984, Behavioral and Brain Sciences.

[2]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[3]  Andreas Knoblauch,et al.  Neural Associative Memory with Optimal Bayesian Learning , 2011, Neural Computation.

[4]  H. Barlow,et al.  Single Units and Sensation: A Neuron Doctrine for Perceptual Psychology? , 1972, Perception.

[5]  B Kosko,et al.  Adaptive bidirectional associative memories. , 1987, Applied optics.

[6]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[7]  Günther Palm,et al.  On the Information Storage Capacity of Local Learning Rules , 1992, Neural Computation.

[8]  A. R. Gardner-Medwin The recall of events through the learning of associations between their parts , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[9]  Arnold Knopfmacher,et al.  A SURVEY OF FACTORIZATION COUNTING FUNCTIONS , 2005 .

[10]  Karl Steinbuch,et al.  Die Lernmatrix , 2004, Kybernetik.

[11]  Hans-Jörg Schek,et al.  A Quantitative Analysis and Performance Study for Similarity-Search Methods in High-Dimensional Spaces , 1998, VLDB.

[12]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[13]  Sompolinsky,et al.  Willshaw model: Associative memory with sparse coding and low firing rates. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[14]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Jay Buckinghamts On setting unit thresholds in an incompletely connected associative net , 1993 .

[16]  Rafal Bogacz,et al.  Model of Familiarity Discrimination in the Perirhinal Cortex , 2004, Journal of Computational Neuroscience.

[17]  Günther Palm,et al.  Memory Capacities for Synaptic and Structural Plasticity G ¨ Unther Palm , 2022 .

[18]  Andreas Knoblauch,et al.  Zip nets: Efficient associative computation with binary synapses , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[19]  E. Hille,et al.  A problem in "Factorisatio Numerorum" , 1936 .

[20]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[21]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[22]  L. Newberg,et al.  A Lower Bound on the Number of Solutions to the Probed Partial Digest Problem , 1993 .

[23]  Andreas Wichert,et al.  Tree-like hierarchical associative memory structures , 2011, Neural Networks.

[24]  David J. Willshaw,et al.  Improving recall from an associative memory , 1995, Biological Cybernetics.

[25]  Rafal Bogacz,et al.  Comparison of computational models of familiarity discrimination in the perirhinal cortex , 2003, Hippocampus.

[26]  J. Austin Associative memory , 1987 .

[27]  T. Sejnowski,et al.  Storing covariance with nonlinearly interacting neurons , 1977, Journal of mathematical biology.

[28]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[29]  Günther Palm,et al.  Information storage and effective data retrieval in sparse matrices , 1989, Neural Networks.

[30]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[31]  E. Gardner The space of interactions in neural network models , 1988 .

[32]  Friedrich T. Sommer,et al.  Associative Data Storage and Retrieval in Neural Networks , 1996 .

[33]  Mark C. W. van Rossum,et al.  Optimal learning rules for familiarity detection , 2008, Biological Cybernetics.

[34]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[35]  Terrence J Sejnowski,et al.  Communication in Neuronal Networks , 2003, Science.

[36]  David Willshaw,et al.  Performance characteristics of the associative net , 1992 .

[37]  Alexandr Andoni,et al.  Near-Optimal Hashing Algorithms for Approximate Nearest Neighbor in High Dimensions , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[38]  Shun-ichi Amari,et al.  Characteristics of sparsely encoded associative memory , 1989, Neural Networks.

[39]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[40]  P. Dayan,et al.  Optimising synaptic learning rules in linear associative memories , 1991, Biological Cybernetics.

[41]  D Marr,et al.  Simple memory: a theory for archicortex. , 1971, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[42]  J. Buckingham,et al.  Delicate nets, faint recollections : a study of partially connected associative network memories , 1992 .

[43]  William B. Levy,et al.  Energy Efficient Neural Codes , 1996, Neural Computation.

[44]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[45]  P. Lennie The Cost of Cortical Computation , 2003, Current Biology.

[46]  Günther Palm,et al.  Improved bidirectional retrieval of sparse patterns stored by Hebbian learning , 1999, Neural Networks.

[47]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[48]  Andreas Knoblauch,et al.  Neural Associative Memory and the Willshaw--Palm Probability Distribution , 2008, SIAM J. Appl. Math..

[49]  S. Laughlin Energy as a constraint on the coding and processing of sensory information , 2001, Current Opinion in Neurobiology.

[50]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[51]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[52]  Günther Palm,et al.  Information capacity in recurrent McCulloch-Pitts networks with sparsely coded memory states , 1992 .

[53]  G Palm,et al.  Computing with neural networks. , 1987, Science.

[54]  Jean-Pierre Nadal,et al.  Information storage in sparsely coded memory nets , 1990 .

[55]  Benny Chor,et al.  On the number of ordered factorizations of natural numbers , 2000, Discret. Math..

[56]  Peter Dayan,et al.  Optimal Plasticity from Matrix Memories: What Goes Up Must Come Down , 1990, Neural Computation.

[57]  W. Kinzel Physics of Neural Networks , 1990 .