The dynamics of memory retrieval in hierarchical networks

Memory retrieval is of central importance to a wide variety of brain functions. To understand the dynamic nature of memory retrieval and its underlying neurophysiological mechanisms, we develop a biologically plausible spiking neural circuit model, and demonstrate that free memory retrieval of sequences of events naturally arises from the model under the condition of excitation-inhibition (E/I) balance. Using the mean-field model of the spiking circuit, we gain further theoretical insights into how such memory retrieval emerges. We show that the spiking neural circuit model quantitatively reproduces several salient features of free memory retrieval, including its semantic proximity effect and log-normal distributions of inter-retrieval intervals. In addition, we demonstrate that our model can serve as a platform to examine memory retrieval deficits observed in neuropsychiatric diseases such as Parkinson’s and Alzheimer’s diseases. Furthermore, our model allows us to make novel and experimentally testable predictions, such as the prediction that there are long-range correlations in the sequences of retrieved items.

[1]  Joshua B. Tenenbaum,et al.  The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth , 2001, Cogn. Sci..

[2]  Hans M Koot,et al.  Verbal fluency in children with autism spectrum disorders: Clustering and switching strategies , 2014, Autism : the international journal of research and practice.

[3]  R. Voss,et al.  Evolution of long-range fractal correlations and 1/f noise in DNA base sequences. , 1992, Physical review letters.

[4]  Alec Solway,et al.  Optimal Behavioral Hierarchy , 2014, PLoS Comput. Biol..

[5]  Xiao-Jing Wang,et al.  Mean-Field Theory of Irregularly Spiking Neuronal Populations and Working Memory in Recurrent Cortical Networks , 2003 .

[6]  O. Sporns Networks of the Brain , 2010 .

[7]  Prof. Dr. Valentino Braitenberg,et al.  Anatomy of the Cortex , 1991, Studies of Brain Function.

[8]  A. Cronin-Golomb,et al.  Parkinson’s Disease as a Disconnection Syndrome , 2010, Neuropsychology Review.

[9]  J. Wixted,et al.  Analyzing the dynamics of free recall: An integrative review of the empirical literature , 1994, Psychonomic bulletin & review.

[10]  D G Gadian,et al.  Hierarchical organization of cognitive memory. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[11]  R. Stein Some models of neuronal variability. , 1967, Biophysical journal.

[12]  Blake S. Porter,et al.  Hippocampal Representation of Related and Opposing Memories Develop within Distinct, Hierarchically Organized Neural Schemas , 2014, Neuron.

[13]  A. Baddeley Human Memory: Theory and Practice, Revised Edition , 1990 .

[14]  P. Dayan,et al.  Supporting Online Material Materials and Methods Som Text Figs. S1 to S9 References the Asynchronous State in Cortical Circuits , 2022 .

[15]  Adam Johnson,et al.  Triple Dissociation of Information Processing in Dorsal Striatum, Ventral Striatum, and Hippocampus on a Learned Spatial Decision Task , 2010, Neuron.

[16]  X. Wang,et al.  Synaptic Basis of Cortical Persistent Activity: the Importance of NMDA Receptors to Working Memory , 1999, The Journal of Neuroscience.

[17]  Alessandro Treves,et al.  Cortical free-association dynamics: distinct phases of a latching network. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[18]  G. Uhlenbeck,et al.  On the Theory of the Brownian Motion , 1930 .

[19]  Michael T. Turvey,et al.  Human memory retrieval as Lévy foraging , 2007 .

[20]  Christof Koch,et al.  Biophysics of Computation: Information Processing in Single Neurons (Computational Neuroscience Series) , 1998 .

[21]  Nicolas Brunel,et al.  Dynamics of the Firing Probability of Noisy Integrate-and-Fire Neurons , 2002, Neural Computation.

[22]  L Schimansky-Geier,et al.  First passage time densities in resonate-and-fire models. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[23]  Thomas T. Hills,et al.  Optimal foraging in semantic memory. , 2012, Psychological review.

[24]  D. McCormick,et al.  Neocortical Network Activity In Vivo Is Generated through a Dynamic Balance of Excitation and Inhibition , 2006, The Journal of Neuroscience.

[25]  Pulin Gong,et al.  Propagating Waves Can Explain Irregular Neural Dynamics , 2015, The Journal of Neuroscience.

[26]  H Sompolinsky,et al.  Dynamics of random neural networks with bistable units. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  Lorena Deuker,et al.  Memory Consolidation by Replay of Stimulus-Specific Neural Activity , 2013, The Journal of Neuroscience.

[28]  Adam Johnson,et al.  Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point , 2007, The Journal of Neuroscience.

[29]  A. Litwin-Kumar,et al.  Slow dynamics and high variability in balanced cortical networks with clustered connections , 2012, Nature Neuroscience.

[30]  Rasmus R. Schröder,et al.  Reversal of Interlaminar Signal Between Sensory and Memory Processing in Monkey Temporal Cortex , 2011 .

[31]  Daniel J. Amit,et al.  Mean Field and Capacity in Realistic Networks of Spiking Neurons Storing Sparsely Coded Random Memories , 2004, Neural Computation.

[32]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[33]  E. Halgren,et al.  Spatiotemporal dynamics of neocortical excitation and inhibition during human sleep , 2012, Proceedings of the National Academy of Sciences.

[34]  H. Risken Fokker-Planck Equation , 1996 .

[35]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[36]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[37]  E. Miller,et al.  Prospective Coding for Objects in Primate Prefrontal Cortex , 1999, The Journal of Neuroscience.

[38]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[39]  D. Amit,et al.  Effective neural response function for collective population states. , 1999, Network.

[40]  Thomas L. Griffiths,et al.  Human memory search as a random walk in a semantic network , 2012, NIPS.

[41]  Henry Kennedy,et al.  Cortical High-Density Counterstream Architectures , 2013, Science.

[42]  P. Hänggi,et al.  Reaction-rate theory: fifty years after Kramers , 1990 .

[43]  M. Botvinick Hierarchical models of behavior and prefrontal function , 2008, Trends in Cognitive Sciences.

[44]  H. Risken Fokker-Planck Equation , 1984 .

[45]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[46]  David J. Foster,et al.  Reverse replay of behavioural sequences in hippocampal place cells during the awake state , 2006, Nature.

[47]  R. Yuste,et al.  Dense Inhibitory Connectivity in Neocortex , 2011, Neuron.

[48]  Sandro Romani,et al.  Scaling Laws of Associative Memory Retrieval , 2013, Neural Computation.

[49]  H. Sompolinsky,et al.  Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity , 1996, Science.

[50]  D Kleinfeld,et al.  Sequential state generation by model neural networks. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[51]  Gustavo Deco,et al.  Neural Network Mechanisms Underlying Stimulus Driven Variability Reduction , 2012, PLoS Comput. Biol..

[52]  A. M. Edwards,et al.  Revisiting Lévy flight search patterns of wandering albatrosses, bumblebees and deer , 2007, Nature.

[53]  Terrence J. Sejnowski,et al.  An Efficient Method for Computing Synaptic Conductances Based on a Kinetic Model of Receptor Binding , 1994, Neural Computation.

[54]  F. Collette,et al.  Alzheimer' Disease as a Disconnection Syndrome? , 2003, Neuropsychology Review.

[55]  Vivien A. Casagrande,et al.  Biophysics of Computation: Information Processing in Single Neurons , 1999 .

[56]  Per B Sederberg,et al.  The temporal contiguity effect predicts episodic memory performance. , 2010, Memory & cognition.

[57]  N. Brunel,et al.  Firing frequency of leaky intergrate-and-fire neurons with synaptic current dynamics. , 1998, Journal of theoretical biology.

[58]  S. Chandrasekhar Stochastic problems in Physics and Astronomy , 1943 .

[59]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[60]  T. Griffiths,et al.  Google and the Mind , 2007, Psychological science.

[61]  Brice A. Kuhl,et al.  Successful Remembering Elicits Event-Specific Activity Patterns in Lateral Parietal Cortex , 2014, The Journal of Neuroscience.

[62]  Henrik Flyvbjerg,et al.  Harmonic oscillator in heat bath: exact simulation of time-lapse-recorded data and exact analytical benchmark statistics. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[63]  Joaquín Goñi,et al.  The semantic organization of the animal category: evidence from semantic verbal fluency and network theory , 2011, Cognitive Processing.

[64]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[65]  Lief E. Fenno,et al.  Neocortical excitation/inhibition balance in information processing and social dysfunction , 2011, Nature.

[66]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[67]  L. Mucke,et al.  Epilepsy and cognitive impairments in Alzheimer disease. , 2009, Archives of neurology.

[68]  M. Scanziani,et al.  Equalizing Excitation-Inhibition Ratios across Visual Cortical Neurons , 2014, Nature.

[69]  G. Winocur,et al.  Clustering and switching as two components of verbal fluency: evidence from younger and older healthy adults. , 1997, Neuropsychology.

[70]  Karl J. Friston The disconnection hypothesis , 1998, Schizophrenia Research.

[71]  W. Wildman,et al.  Theoretical Neuroscience , 2014 .

[72]  M. Freedman,et al.  Clustering and switching on verbal fluency tests in Alzheimer's and Parkinson's disease , 1998, Journal of the International Neuropsychological Society.

[73]  Edward T. Bullmore,et al.  Modular and Hierarchically Modular Organization of Brain Networks , 2010, Front. Neurosci..

[74]  Brad E. Pfeiffer,et al.  Autoassociative dynamics in the generation of sequences of hippocampal place cells , 2015, Science.

[75]  M. Catani,et al.  The rises and falls of disconnection syndromes. , 2005, Brain : a journal of neurology.

[76]  A. Destexhe,et al.  The high-conductance state of neocortical neurons in vivo , 2003, Nature Reviews Neuroscience.

[77]  H. Cline,et al.  Synaptogenesis: A Balancing Act between Excitation and Inhibition , 2005, Current Biology.

[78]  T. M. Mayhew,et al.  Anatomy of the Cortex: Statistics and Geometry. , 1991 .

[79]  M. Crossley,et al.  Clustering and Switching Strategies During Verbal Fluency Performance Differentiate Alzheimer's Disease and Healthy Aging , 2011, Journal of the International Neuropsychological Society.

[80]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[81]  J-P Eckmann,et al.  Hierarchical structures induce long-range dynamical correlations in written texts. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[82]  H. Stanley,et al.  Optimizing the success of random searches , 1999, Nature.

[83]  D. Gillespie Markov Processes: An Introduction for Physical Scientists , 1991 .

[84]  D. Amit,et al.  Retrospective and prospective persistent activity induced by Hebbian learning in a recurrent cortical network , 2003, The European journal of neuroscience.

[85]  Thomas T. Hills,et al.  Exploration versus exploitation in space, mind, and society , 2015, Trends in Cognitive Sciences.

[86]  Nicolas Brunel,et al.  Firing Rate of the Noisy Quadratic Integrate-and-Fire Neuron , 2003, Neural Computation.

[87]  J. Wixted,et al.  An analysis of latency and interresponse time in free recall , 1994, Memory & cognition.

[88]  M. Ross Quillian,et al.  Retrieval time from semantic memory , 1969 .

[89]  R. Yuste,et al.  Dense, Unspecific Connectivity of Neocortical Parvalbumin-Positive Interneurons: A Canonical Microcircuit for Inhibition? , 2011, The Journal of Neuroscience.

[90]  Wulfram Gerstner,et al.  Synaptic Shot Noise and Conductance Fluctuations Affect the Membrane Voltage with Equal Significance , 2005, Neural Computation.

[91]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[92]  R. Shiffrin,et al.  Search of associative memory. , 1981 .

[93]  G. La Camera,et al.  Dynamics of Multistable States during Ongoing and Evoked Cortical Activity , 2015, The Journal of Neuroscience.

[94]  A. Destexhe Kinetic Models of Synaptic Transmission , 1997 .

[95]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[96]  H. Risken The Fokker-Planck equation : methods of solution and applications , 1985 .