Correlations between synapses in pairs of neurons slow down dynamics in randomly connected neural networks.

Networks of randomly connected neurons are among the most popular models in theoretical neuroscience. The connectivity between neurons in the cortex is however not fully random, the simplest and most prominent deviation from randomness found in experimental data being the overrepresentation of bidirectional connections among pyramidal cells. Using numerical and analytical methods, we investigate the effects of partially symmetric connectivity on the dynamics in networks of rate units. We consider the two dynamical regimes exhibited by random neural networks: the weak-coupling regime, where the firing activity decays to a single fixed point unless the network is stimulated, and the strong-coupling or chaotic regime, characterized by internally generated fluctuating firing rates. In the weak-coupling regime, we compute analytically, for an arbitrary degree of symmetry, the autocorrelation of network activity in the presence of external noise. In the chaotic regime, we perform simulations to determine the timescale of the intrinsic fluctuations. In both cases, symmetry increases the characteristic asymptotic decay time of the autocorrelation function and therefore slows down the dynamics in the network.

[1]  G. Wainrib,et al.  Topological and dynamical complexity of random neural networks. , 2012, Physical review letters.

[2]  Mark S. Goldman,et al.  Memory without Feedback in a Neural Network , 2009, Neuron.

[3]  H. Sompolinsky,et al.  Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity , 1996, Science.

[4]  Brent Doiron,et al.  Once upon a (slow) time in the land of recurrent neuronal networks… , 2017, Current Opinion in Neurobiology.

[5]  Leticia F. Cugliandolo,et al.  Out of Equilibrium Dynamics of Complex Systems , 2009 .

[6]  W. Gerstner,et al.  Optimal Control of Transient Dynamics in Balanced Networks Supports Generation of Complex Movements , 2014, Neuron.

[7]  H Sompolinsky,et al.  Dynamics of random neural networks with bistable units. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[8]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Front , 2020, 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4).

[10]  Alex Roxin,et al.  The Role of Degree Distribution in Shaping the Dynamics in Networks of Sparsely Connected Spiking Neurons , 2011, Front. Comput. Neurosci..

[11]  Thomas K. Berger,et al.  A synaptic organizing principle for cortical neuronal groups , 2011, Proceedings of the National Academy of Sciences.

[12]  J. Ginibre Statistical Ensembles of Complex, Quaternion, and Real Matrices , 1965 .

[13]  Theoden I. Netoff,et al.  Synchronization from Second Order Network Connectivity Statistics , 2011, Front. Comput. Neurosci..

[14]  Kenneth D Miller,et al.  Properties of networks with partially structured and partially random connectivity. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  M. Mézard,et al.  Spin Glass Theory And Beyond: An Introduction To The Replica Method And Its Applications , 1986 .

[16]  Ericka Stricklin-Parker,et al.  Ann , 2005 .

[17]  Merav Stern,et al.  Transition to chaos in random networks with cell-type-specific connectivity. , 2014, Physical review letters.

[18]  Huaixing Wang,et al.  A specialized NMDA receptor function in layer 5 recurrent microcircuitry of the adult rat prefrontal cortex , 2008, Proceedings of the National Academy of Sciences.

[19]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[20]  Sompolinsky,et al.  Dynamics of spin systems with randomly asymmetric bonds: Langevin dynamics and a spherical model. , 1987, Physical review. A, General physics.

[21]  Manfred Opper,et al.  Extended Plefka expansion for stochastic dynamics , 2015, 1509.07066.

[22]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[23]  L. Abbott,et al.  Eigenvalue spectra of random matrices for neural networks. , 2006, Physical review letters.

[24]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[25]  B. Mehlig,et al.  Statistical properties of eigenvectors in non-Hermitian Gaussian random matrix ensembles , 2000 .

[26]  L. Abbott,et al.  Stimulus-dependent suppression of chaos in recurrent neural networks. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[28]  K. Miller,et al.  Balanced Amplification: A New Mechanism of Selective Amplification of Neural Activity Patterns , 2016, Neuron.

[29]  H. Sompolinsky,et al.  Relaxational dynamics of the Edwards-Anderson model and the mean-field theory of spin-glasses , 1982 .

[30]  Sommers,et al.  Spectrum of large random asymmetric matrices. , 1988, Physical review letters.

[31]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[32]  C. Dominicis Dynamics as a substitute for replicas in systems with quenched random impurities , 1978 .

[33]  B. Mehlig,et al.  EIGENVECTOR STATISTICS IN NON-HERMITIAN RANDOM MATRIX ENSEMBLES , 1998 .

[34]  K. Harris,et al.  Cortical connectivity and sensory coding , 2013, Nature.

[35]  L. Abbott,et al.  Beyond the edge of chaos: amplification and temporal integration by recurrent networks in the chaotic regime. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[36]  W. Maass,et al.  State-dependent computations: spatiotemporal processing in cortical networks , 2009, Nature Reviews Neuroscience.

[37]  Sean O'Rourke,et al.  The Elliptic Law , 2012, 1208.5883.

[38]  Dieter Fuhrmann,et al.  Spin Glasses And Random Fields , 2016 .

[39]  Tatyana O Sharpee,et al.  Eigenvalue spectra of large correlated random matrices. , 2016, Physical review. E.

[40]  E. Marinari,et al.  On the stability of the mean-field spin glass broken phase under non-Hamiltonian perturbations , 1997 .

[41]  H. Sompolinsky,et al.  Transition to chaos in random neuronal networks , 2015, 1508.06486.

[42]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[43]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[44]  W. Gerstner,et al.  Non-normal amplification in random balanced neuronal networks. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[45]  Francesca Mastrogiuseppe,et al.  Intrinsically-generated fluctuating activity in excitatory-inhibitory networks , 2016, PLoS Comput. Biol..

[46]  Terence Tao,et al.  Random matrices: Universality of ESDs and the circular law , 2008, 0807.4898.

[47]  A. Litwin-Kumar,et al.  Slow dynamics and high variability in balanced cortical networks with clustered connections , 2012, Nature Neuroscience.

[48]  L. Trefethen,et al.  Spectra and Pseudospectra , 2020 .

[49]  Cugliandolo,et al.  Analytical solution of the off-equilibrium dynamics of a long-range spin-glass model. , 1993, Physical review letters.

[50]  Erwan Ledoux,et al.  Instability to a heterogeneous oscillatory state in randomly connected recurrent networks with delayed interactions. , 2016, Physical review. E.

[51]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[52]  H. Markram,et al.  Physiology and anatomy of synaptic connections between thick tufted pyramidal neurones in the developing rat neocortex. , 1997, The Journal of physiology.

[53]  Srdjan Ostojic,et al.  Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons , 2014, Nature Neuroscience.

[54]  T. Wassmer 6 , 1900, EXILE.

[55]  Michael A. Buice,et al.  Path Integral Methods for Stochastic Differential Equations , 2015, Journal of mathematical neuroscience.

[56]  C. Petersen,et al.  The Excitatory Neuronal Network of the C2 Barrel Column in Mouse Primary Somatosensory Cortex , 2009, Neuron.

[57]  Nicolas Brunel,et al.  Is cortical connectivity optimized for storing information? , 2016, Nature Neuroscience.

[58]  Surya Ganguli,et al.  Memory traces in dynamical systems , 2008, Proceedings of the National Academy of Sciences.

[59]  P. J. Sjöström,et al.  Functional specificity of local synaptic connections in neocortical networks , 2011, Nature.

[60]  H. Janssen,et al.  On a Lagrangean for classical field dynamics and renormalization group calculations of dynamical critical properties , 1976 .

[61]  Haim Sompolinsky,et al.  Chaotic Balanced State in a Model of Cortical Circuits , 1998, Neural Computation.

[62]  Paul C. Martin,et al.  Statistical Dynamics of Classical Systems , 1973 .

[63]  Merav Stern,et al.  Eigenvalues of block structured asymmetric random matrices , 2015 .

[64]  Thomas K. Berger,et al.  Heterogeneity in the pyramidal network of the medial prefrontal cortex , 2006, Nature Neuroscience.

[65]  David J. Freedman,et al.  A hierarchy of intrinsic timescales across primate cortex , 2014, Nature Neuroscience.

[66]  Sommers,et al.  Chaos in random neural networks. , 1988, Physical review letters.

[67]  David S. Dean,et al.  FULL DYNAMICAL SOLUTION FOR A SPHERICAL SPIN-GLASS MODEL , 1995 .

[68]  David Hansel,et al.  Asynchronous Rate Chaos in Spiking Neuronal Circuits , 2015, bioRxiv.

[69]  G. Elston Cortex, cognition and the cell: new insights into the pyramidal neuron and prefrontal function. , 2003, Cerebral cortex.

[70]  Joseph Lipka,et al.  A Table of Integrals , 2010 .

[71]  Enzo Marinari,et al.  Off-equilibrium dynamics of a four-dimensional spin glass with asymmetric couplings , 1998 .