A high-capacity model for one shot association learning in the brain

We present a high-capacity model for one-shot association learning (hetero-associative memory) in sparse networks. We assume that basic patterns are pre-learned in networks and associations between two patterns are presented only once and have to be learned immediately. The model is a combination of an Amit-Fusi like network sparsely connected to a Willshaw type network. The learning procedure is palimpsest and comes from earlier work on one-shot pattern learning. However, in our setup we can enhance the capacity of the network by iterative retrieval. This yields a model for sparse brain-like networks in which populations of a few thousand neurons are capable of learning hundreds of associations even if they are presented only once. The analysis of the model is based on a novel result by Janson et al. on bootstrap percolation in random graphs.

[1]  Günther Palm,et al.  Memory Capacities for Synaptic and Structural Plasticity G ¨ Unther Palm , 2022 .

[2]  Svante Janson,et al.  Majority bootstrap percolation on the random graph G(n,p) , 2010, 1012.3535.

[3]  David Willshaw,et al.  Performance characteristics of the associative net , 1992 .

[4]  M. Colonnier,et al.  The number of neurons in the different laminae of the binocular and monocular regions of area 17 in the cat , 1983, The Journal of comparative neurology.

[5]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[6]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[7]  A. Reyes,et al.  Spatial Profile of Excitatory and Inhibitory Synaptic Connectivity in Mouse Primary Auditory Cortex , 2012, The Journal of Neuroscience.

[8]  G. L. Barrows.  Stochastic Hebbian learning with binary synapses , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[9]  Kenneth A. Dawson,et al.  Bootstrap Percolation , 2009, Encyclopedia of Complexity and Systems Science.

[10]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[11]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[12]  R. H. White,et al.  Competitive Hebbian learning , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[13]  M. Fanselow.  Factors governing one-trial contextual conditioning , 1990 .

[14]  Wolfgang Einhäuser,et al.  Attention in natural scenes: contrast affects rapid visual processing and fixations alike , 2013, Philosophical Transactions of the Royal Society B: Biological Sciences.

[15]  Stanislas Dehaene,et al.  Networks of Formal Neurons and Memory Palimpsests , 1986 .

[16]  Bruno A Olshausen,et al.  Timecourse of neural signatures of object recognition. , 2003, Journal of vision.

[17]  Yali Amit,et al.  Optimizing One-Shot Learning with Binary Synapses , 2008, Neural Computation.

[18]  Daniel J. Amit,et al.  Learning in Neural Networks with Material Synapses , 1994, Neural Computation.

[19]  Günther Palm,et al.  Improved bidirectional retrieval of sparse patterns stored by Hebbian learning , 1999, Neural Networks.

[20]  Yali Amit,et al.  Precise Capacity Analysis in Binary Networks with Multiple Coding Level Inputs , 2010, Neural Computation.

[21]  R. Douglas,et al.  A Quantitative Map of the Circuit of Cat Primary Visual Cortex , 2004, The Journal of Neuroscience.

[22]  J. Hopfield,et al.  All-or-none potentiation at CA3-CA1 synapses. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[24]  Andreas Knoblauch,et al.  Neural Associative Memory and the Willshaw--Palm Probability Distribution , 2008, SIAM J. Appl. Math..

[25]  A. Fine,et al.  Are binary synapses superior to graded weight representations in stochastic attractor networks? , 2009, Cognitive Neurodynamics.

[26]  F. Battaglia,et al.  LEARNING IN NEURAL NETWORKS WITH PARTIALLY STRUCTURED SYNAPTIC TRANSITIONS , 1995 .

[27]  Nicolas Brunel,et al.  Frontiers in Computational Neuroscience Computational Neuroscience , 2022 .

[28]  Günther Palm,et al.  Iterative retrieval of sparsely coded associative memory patterns , 1996, Neural Networks.

[29]  R. Kempter,et al.  Hebbian learning and spiking neurons , 1999 .

[30]  J. Stephen Judd,et al.  Learning in neural networks , 1988, COLT '88.

[31]  N Brunel,et al.  Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network. , 1998, Network.

[32]  Matthieu Gilson,et al.  Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks IV , 2009, Biological Cybernetics.

[33]  N. Kanwisher,et al.  Stages of processing in face perception: an MEG study , 2002, Nature Neuroscience.

[34]  Wulfram Gerstner,et al.  Mathematical formulations of Hebbian learning , 2002, Biological Cybernetics.

[35]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[36]  K. Vairavan,et al.  Performance Characteristics of a , 1995 .

[37]  Simon J. Thorpe,et al.  Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited , 2006, Vision Research.

[38]  Sébastien M. Crouzet,et al.  Fast saccades toward faces: face detection in just 100 ms. , 2010, Journal of vision.

[39]  H. Markram,et al.  The neocortical microcircuit as a tabula rasa. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[40]  H. Markram,et al.  Morphological, Electrophysiological, and Synaptic Properties of Corticocallosal Pyramidal Cells in the Neonatal Rat Neocortex , 2006 .

[41]  S. Wang,et al.  Graded bidirectional synaptic plasticity is composed of switch-like unitary events. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[42]  T. Allison,et al.  Electrophysiological studies of human face perception. I: Potentials generated in occipitotemporal cortex by face and non-face stimuli. , 1999, Cerebral cortex.

[43]  Thomas K. Berger,et al.  A synaptic organizing principle for cortical neuronal groups , 2011, Proceedings of the National Academy of Sciences.

[44]  L. Abbott,et al.  Homeostasis and Learning through Spike-Timing Dependent Plasticity , 2003 .

[45]  Simon J. Thorpe,et al.  BIOLOGICAL CONSTRAINTS ON CONNECTIONIST MODELLING , 2015 .

[46]  Matthieu Gilson,et al.  Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks V: self-organization schemes and weight dependence , 2010, Biological Cybernetics.

[47]  H. S. Meyer,et al.  Cell Type–Specific Thalamic Innervation in a Column of Rat Vibrissal Cortex , 2010, Cerebral cortex.

[48]  J. Nadal,et al.  What can we learn from synaptic weight distributions? , 2007, Trends in Neurosciences.

[49]  Andreas Knoblauch,et al.  Pattern separation and synchronization in spiking associative memories and visual areas , 2001, Neural Networks.