Large Deviations, Dynamics and Phase Transitions in Large Stochastic and Disordered Neural Networks

Neuronal networks are characterized by highly heterogeneous connectivity, and this disorder was recently related experimentally to qualitative properties of the network. The motivation of this paper is to mathematically analyze the role of these disordered connectivities on the large-scale properties of neuronal networks. To this end, we analyze here large-scale limit behaviors of neural networks including, for biological relevance, multiple populations, random connectivities and interaction delays. Due to the randomness of the connectivity, usual mean-field methods (e.g. coupling) cannot be applied, but, similarly to studies developed for spin glasses, we will show that the sequences of empirical measures satisfy a large deviation principle, and converge towards a self-consistent non-Markovian process. From a mathematical viewpoint, the proof differs from previous works in that we are working in infinite-dimensional spaces (interaction delays) and consider multiple cell types. The limit obtained formally characterizes the macroscopic behavior of the network. We propose a dynamical systems approach in order to address the qualitative nature of the solutions of these very complex equations, and apply this methodology to three instances in order to show how non-centered coefficients, interaction delays and multiple populations networks are affected by disorder levels. We identify a number of phase transitions in such systems upon changes in delays, connectivity patterns and dispersion, and particularly focus on the emergence of non-equilibrium states involving synchronized oscillations.

[1]  J. Touboul Propagation of chaos in neural fields , 2011, 1108.2414.

[2]  R. Dobrushin Prescribing a System of Random Variables by Conditional Distributions , 1970 .

[3]  S. Amari,et al.  Characteristics of Random Nets of Analog Neuron-Like Elements , 1972, IEEE Trans. Syst. Man Cybern..

[4]  D. Hansel,et al.  Role of delays in shaping spatiotemporal dynamics of neuronal activity in large networks. , 2005, Physical review letters.

[5]  H. Sompolinsky,et al.  Relaxational dynamics of the Edwards-Anderson model and the mean-field theory of spin-glasses , 1982 .

[6]  G. Wainrib,et al.  Topological and dynamical complexity of random neural networks. , 2012, Physical review letters.

[7]  J. Zabczyk,et al.  Stochastic Equations in Infinite Dimensions , 2008 .

[8]  W. Singer,et al.  Neural Synchrony in Brain Disorders: Relevance for Cognitive Dysfunctions and Pathophysiology , 2006, Neuron.

[9]  Alexander S. Ecker,et al.  Decorrelated Neuronal Firing in Cortical Microcircuits , 2010, Science.

[10]  P. Bressloff Spatiotemporal dynamics of continuum neural fields , 2012 .

[11]  A. Guionnet Averaged and quenched propagation of chaos for spin glass dynamics , 1997 .

[12]  Herold Dehling,et al.  Large deviations for some weakly dependent random processes , 1990 .

[13]  Charles J. Wilson,et al.  Up and down states , 2008, Scholarpedia.

[14]  L. Abbott,et al.  Stimulus-dependent suppression of chaos in recurrent neural networks. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Daniel Choquet,et al.  New Concepts in Synaptic Biology Derived from Single-Molecule Imaging , 2008, Neuron.

[16]  Cessac Bruno,et al.  A constructive mean-field analysis of multi population neural networks with random synaptic weights , 2009 .

[17]  Noam E. Ziv,et al.  Long-term Relationships between Cholinergic Tone, Synchronous Bursting and Synaptic Remodeling , 2012, PloS one.

[18]  Sompolinsky,et al.  Dynamics of spin systems with randomly asymmetric bonds: Ising spins and Glauber dynamics. , 1988, Physical review. A, General physics.

[19]  I. Soltesz,et al.  Modulation of network behaviour by changes in variance in interneuronal properties , 2002, The Journal of physiology.

[20]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[21]  S. Coombes,et al.  Delays in activity-based neural networks , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[22]  X. Mao,et al.  Stochastic Differential Equations and Applications , 1998 .

[23]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[24]  Paul C. Bressloff,et al.  Stochastic Neural Field Theory and the System-Size Expansion , 2009, SIAM J. Appl. Math..

[25]  J. Touboul,et al.  Heterogeneous connections induce oscillations in large-scale networks. , 2012, Physical review letters.

[26]  Olivier Pinaud,et al.  Mean-field Theory and Synchronization in Random Recurrent Neural Networks , 2004, Neural Processing Letters.

[27]  E. Marder,et al.  Variability, compensation and homeostasis in neuron and network function , 2006, Nature Reviews Neuroscience.

[28]  J. Touboul Mean-field equations for stochastic firing-rate neural fields with delays: Derivation and noise-induced transitions , 2012 .

[29]  Herbert Jaeger,et al.  Echo state network , 2007, Scholarpedia.

[30]  Michael A. Buice,et al.  Systematic Fluctuation Expansion for Neural Network Activity Equations , 2009, Neural Computation.

[31]  G. Buzsáki Rhythms of the brain , 2006 .

[32]  Olivier Moynot Etude mathematique de la dynamique des reseaux neuronaux aleatoires recurrents , 2000 .

[33]  S. Varadhan,et al.  Large deviations , 2019, Graduate Studies in Mathematics.

[34]  A. Sznitman Topics in propagation of chaos , 1991 .

[35]  Olivier D. Faugeras,et al.  A Constructive Mean-Field Analysis of Multi-Population Neural Networks with Random Synaptic Weights and Stochastic Inputs , 2008, Front. Comput. Neurosci..

[36]  École d'été de probabilités de Saint-Flour,et al.  Ecole d'été de probabilités de Saint-Flour XIX, 1989 , 1991 .

[37]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[38]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[39]  D. Parker Variable Properties in a Single Class of Excitatory Spinal Synapse , 2003, The Journal of Neuroscience.

[40]  B. Cessac,et al.  Mean-field equations, bifurcation map and route to chaos in discrete time neural networks , 1994 .

[41]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[42]  L. Abbott,et al.  Beyond the edge of chaos: amplification and temporal integration by recurrent networks in the chaotic regime. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[43]  J. Cowan,et al.  A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue , 1973, Kybernetik.

[44]  P. Dayan,et al.  Supporting Online Material Materials and Methods Som Text Figs. S1 to S9 References the Asynchronous State in Cortical Circuits , 2022 .

[45]  G. B. Arous,et al.  Large deviations for Langevin spin glass dynamics , 1995 .

[46]  M. Samuelides,et al.  Large deviations and mean-field theory for asymmetric random recurrent neural networks , 2002 .

[47]  Olivier Faugeras,et al.  A large deviation principle for networks of rate neurons with correlated synaptic weights , 2013, BMC Neuroscience.

[48]  Sommers,et al.  Chaos in random neural networks. , 1988, Physical review letters.

[49]  Olivier D. Faugeras,et al.  Noise-Induced Behaviors in Neural Mean Field Dynamics , 2011, SIAM J. Appl. Dyn. Syst..

[50]  A. Sznitman Équations de type de Boltzmann, spatialement homogènes , 1984 .

[51]  V. Mountcastle The columnar organization of the neocortex. , 1997, Brain : a journal of neurology.

[52]  Y. Frégnac,et al.  Orientation dependent modulation of apparent speed: a model based on the dynamics of feed-forward and horizontal connectivity in V1 cortex , 2002, Vision Research.

[53]  Jack K. Hale,et al.  Introduction to Functional Differential Equations , 1993, Applied Mathematical Sciences.

[54]  A. Faisal,et al.  Noise in the nervous system , 2008, Nature Reviews Neuroscience.

[55]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.