From synapse to network: models of information storage and retrieval in neural circuits

The mechanisms of information storage and retrieval in brain circuits are still the subject of debate. It is widely believed that information is stored at least in part through changes in synaptic connectivity in networks that encode this information and that these changes lead in turn to modifications of network dynamics, such that the stored information can be retrieved at a later time. Here, we review recent progress in deriving synaptic plasticity rules from experimental data and in understanding how plasticity rules affect the dynamics of recurrent networks. We show that the dynamics generated by such networks exhibit a large degree of diversity, depending on parameters, similar to experimental observations in vivo during delayed response tasks.

[1]  Y. Miyashita.  Neuronal correlate of visual associative long-term memory in the primate temporal cortex , 1988, Nature.

[2]  Zoran Tiganj,et al.  Sequential Firing Codes for Time in Rodent Medial Prefrontal Cortex , 2017, Cerebral cortex.

[3]  E. Gardner.  The space of interactions in neural network models , 1988 .

[4]  V. Jayaraman,et al.  Ring attractor dynamics in the Drosophila central brain , 2017, Science.

[5]  Marc W Howard,et al.  Time Cells in Hippocampal Area CA3 , 2016, The Journal of Neuroscience.

[6]  Ulises Pereira,et al.  Attractor Dynamics in Networks with Learning Rules Inferred from In Vivo Data , 2017, Neuron.

[7]  G. Bi,et al.  Gain in sensitivity and loss in temporal contrast of STDP by dopaminergic modulation at hippocampal synapses , 2009, Proceedings of the National Academy of Sciences.

[8]  Christos Constantinidis,et al.  Stable population coding for working memory coexists with heterogeneous neural dynamics in prefrontal cortex , 2016, Proceedings of the National Academy of Sciences.

[9]  Haim Sompolinsky,et al.  Balanced excitation and inhibition are required for high-capacity, noise-robust neuronal selectivity , 2017, Proceedings of the National Academy of Sciences.

[10]  P. Jonas,et al.  Symmetric spike timing-dependent plasticity at CA3–CA3 synapses optimizes storage and recall in autoassociative networks , 2016, Nature Communications.

[11]  Sandro Romani,et al.  Learning in realistic networks of spiking neurons and spike‐driven plastic synapses , 2005, The European journal of neuroscience.

[12]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[13]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[14]  R. Froemke.  Plasticity of cortical excitatory-inhibitory balance. , 2015, Annual review of neuroscience.

[15]  Sho Yagishita,et al.  A critical time window for dopamine actions on the structural plasticity of dendritic spines , 2014, Science.

[16]  Wei Ji Ma,et al.  A diverse range of factors affect the nature of neural representations underlying short-term memory , 2018, Nature Neuroscience.

[17]  Katie C. Bittner,et al.  Behavioral time scale synaptic plasticity underlies CA1 place fields , 2017, Science.

[18]  Terrence J Sejnowski,et al.  Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks , 2020, Nature neuroscience.

[19]  Kenneth D. Miller,et al.  The Role of Constraints in Hebbian Learning , 1994, Neural Computation.

[20]  Nicolas Brunel,et al.  Is cortical connectivity optimized for storing information? , 2016, Nature Neuroscience.

[21]  Aparna Suvrathan,et al.  Beyond STDP—towards diverse and functionally relevant plasticity rules , 2019, Current Opinion in Neurobiology.

[22]  Nicolas Y. Masse,et al.  Circuit mechanisms for the maintenance and manipulation of information in working memory , 2018, Nature Neuroscience.

[23]  G. Mongillo,et al.  Inhibitory connectivity defines the realm of excitatory plasticity , 2018, Nature Neuroscience.

[24]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[25]  Nicolas Brunel,et al.  Synaptic plasticity rules with physiological calcium levels , 2020, Proceedings of the National Academy of Sciences.

[26]  Everton J. Agnes,et al.  Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks , 2015, Nature Communications.

[27]  K. Nakamura,et al.  Mnemonic firing of neurons in the monkey temporal pole during a visual recognition memory task. , 1995, Journal of neurophysiology.

[28]  Sukbin Lim,et al.  Mechanisms underlying sharpening of visual response dynamics with familiarity , 2019, eLife.

[29]  Christopher Summerfield,et al.  Characterizing emergent representations in a space of candidate learning rules for deep networks , 2020, NeurIPS.

[30]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[31]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[32]  D. Amit.  The Hebbian paradigm reintegrated: Local reverberations as internal representations , 1995, Behavioral and Brain Sciences.

[33]  O. Paulsen,et al.  Activity-Dependent Downscaling of Subthreshold Synaptic Inputs during Slow-Wave-Sleep-like Activity In Vivo , 2018, Neuron.

[34]  M. Sur,et al.  Locally coordinated synaptic plasticity of visual cortex neurons in vivo , 2018, Science.

[35]  N. Brunel,et al.  Calcium-based plasticity model explains sensitivity of synaptic changes to spike pattern, rate, and dendritic location , 2012, Proceedings of the National Academy of Sciences.

[36]  Ulises Pereira,et al.  Characteristics of sequential activity in networks with temporally asymmetric Hebbian learning , 2020, Proceedings of the National Academy of Sciences.

[37]  A. Litwin-Kumar,et al.  Formation and maintenance of neuronal assemblies through synaptic plasticity , 2014, Nature Communications.

[38]  Richard Hans Robert Hahnloser,et al.  Spike-Time-Dependent Plasticity and Heterosynaptic Competition Organize Networks to Produce Long Scale-Free Sequences of Neural Activity , 2010, Neuron.

[39]  Naama Brenner,et al.  Stable memory with unstable synapses , 2018, Nature Communications.

[40]  Christos Constantinidis,et al.  Persistent Spiking Activity Underlies Working Memory , 2018, The Journal of Neuroscience.

[41]  Brent Doiron,et al.  Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses , 2015, PLoS Comput. Biol..

[42]  D. Feldman.  Synaptic mechanisms for plasticity in neocortex. , 2009, Annual review of neuroscience.

[43]  J. Hopfield,et al.  All-or-none potentiation at CA3-CA1 synapses. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[44]  H Sompolinsky,et al.  Associative neural network model for the generation of temporal patterns. Theory and application to central pattern generators. , 1988, Biophysical journal.

[45]  Carl Smith,et al.  Fast state-space methods for inferring dendritic synaptic connectivity , 2013, Journal of Computational Neuroscience.

[46]  James M Murray,et al.  Local online learning in recurrent networks with random feedback , 2018, bioRxiv.

[47]  David J. Freedman,et al.  Inferring learning rules from distribution of firing rates in cortical neurons , 2015, Nature Neuroscience.

[48]  L. Cooper,et al.  A unified model of NMDA receptor-dependent bidirectional synaptic plasticity , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[49]  Christopher D. Harvey,et al.  Recurrent Network Models of Sequence Generation and Memory , 2016, Neuron.

[50]  Alex Roxin,et al.  Theta-modulation drives the emergence of connectivity patterns underlying replay in a network model of place cells , 2018, eLife.

[51]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[52]  Thomas Miconi,et al.  Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks , 2016, bioRxiv.

[53]  Alessandro Treves,et al.  Efficiency of local learning rules in threshold-linear associative networks , 2020 .

[54]  Earl K Miller,et al.  Working Memory: Delay Activity, Yes! Persistent Activity? Maybe Not , 2018, The Journal of Neuroscience.

[55]  Lisandro Montangie,et al.  Autonomous emergence of connectivity assemblies via spike triplet interactions , 2020, PLoS computational biology.

[56]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[57]  J. Kotaleski,et al.  Modelling the molecular mechanisms of synaptic plasticity using systems biology approaches , 2010, Nature Reviews Neuroscience.

[58]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[59]  Richard Kempter,et al.  Memory replay in balanced recurrent networks , 2016, bioRxiv.

[60]  Timothy O’Leary,et al.  Causes and consequences of representational drift , 2019, Current Opinion in Neurobiology.

[61]  Misha Tsodyks,et al.  Chaos in Highly Diluted Neural Networks , 1991 .

[62]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[63]  Xiao-Jing Wang,et al.  Task representations in neural networks trained to perform many cognitive tasks , 2019, Nature Neuroscience.

[64]  Jochen Triesch,et al.  Robust development of synfire chains from multiple plasticity mechanisms , 2014, Front. Comput. Neurosci..

[65]  Julio Chapeton,et al.  Efficient associative memory storage in cortical circuits of inhibitory and excitatory neurons , 2012, Proceedings of the National Academy of Sciences.

[66]  W. Gerstner,et al.  Connectivity reflects coding: a model of voltage-based STDP with homeostasis , 2010, Nature Neuroscience.

[67]  W. Gerstner,et al.  Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[68]  Yoram Burak,et al.  Shaping Neural Circuits by High Order Synaptic Interactions , 2016, PLoS Comput. Biol..

[69]  S. Wang,et al.  Malleability of Spike-Timing-Dependent Plasticity at the CA3–CA1 Synapse , 2006, The Journal of Neuroscience.

[70]  Peter A. Appleby,et al.  Triphasic spike-timing-dependent plasticity organizes networks to produce robust sequences of neural activity , 2012, Front. Comput. Neurosci..

[71]  M. Tsodyks,et al.  The Enhanced Storage Capacity in Neural Networks with Low Activity Level , 1988 .

[72]  S. Nelson,et al.  Hebb and homeostasis in neuronal plasticity , 2000, Current Opinion in Neurobiology.

[73]  U. Rutishauser,et al.  Between persistently active and activity‐silent frameworks: novel vistas on the cellular basis of working memory , 2019, Annals of the New York Academy of Sciences.

[74]  Sandro Romani,et al.  Discrete attractor dynamics underlies persistent activity in the frontal cortex , 2019, Nature.

[75]  Zachary P. Kilpatrick,et al.  Networks that learn the precise timing of event sequences , 2014, bioRxiv.

[76]  Daniel L. K. Yamins,et al.  Identifying Learning Rules From Neural Network Observables , 2020, NeurIPS.