From neuron to neural networks dynamics

Abstract.This paper presents an overview of some techniques and concepts coming from dynamical system theory and used for the analysis of dynamical neural networks models. In a first section, we describe the dynamics of the neuron, starting from the Hodgkin-Huxley description, which is somehow the canonical description for the “biological neuron”. We discuss some models reducing the Hodgkin-Huxley model to a two dimensional dynamical system, keeping one of the main feature of the neuron: its excitability. We present then examples of phase diagram and bifurcation analysis for the Hodgin-Huxley equations. Finally, we end this section by a dynamical system analysis for the nervous flux propagation along the axon. We then consider neuron couplings, with a brief description of synapses, synaptic plasticity and learning, in a second section. We also briefly discuss the delicate issue of causal action from one neuron to another when complex feedback effects and non linear dynamics are involved. The third section presents the limit of weak coupling and the use of normal forms technics to handle this situation. We consider then several examples of recurrent models with different type of synaptic interactions (symmetric, cooperative, random). We introduce various techniques coming from statistical physics and dynamical systems theory. A last section is devoted to a detailed example of recurrent model where we go in deep in the analysis of the dynamics and discuss the effect of learning on the neuron dynamics. We also present recent methods allowing the analysis of the non linear effects of the neural dynamics on signal propagation and causal action. An appendix, presenting the main notions of dynamical systems theory useful for the comprehension of the chapter, has been added for the convenience of the reader.

[1]  Shun-ichi Amari,et al.  A method of statistical neurodynamics , 1974, Kybernetik.

[2]  勇一 作村,et al.  Biophysics of Computation , 2001 .

[3]  B. Cessac.  Increase in Complexity in Random Neural Networks , 1995 .

[4]  G. Sell,et al.  The Hopf Bifurcation and Its Applications , 1976 .

[5]  Anton Bovier,et al.  Rigorous bounds on the storage capacity of the dilute Hopfield model , 1992 .

[6]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[7]  Laurent Perrinet,et al.  Dynamical neural networks: Modeling low-level vision at short latencies , 2007 .

[8]  William R. Softky,et al.  Simple codes versus efficient codes , 1995, Current Opinion in Neurobiology.

[9]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[10]  Michel Benaïm.  Dynamiques d'activation et dynamiques d'apprentissage des réseaux de neurones , 1992 .

[11]  J. Rinzel.  Excitation dynamics: insights from simplified membrane models. , 1985, Federation proceedings.

[12]  Vladimir Igorevich Arnolʹd,et al.  Équations différentielles ordinaires , 1974 .

[13]  Frank C. Hoppensteadt,et al.  Bifurcations in brain dynamics , 1996 .

[14]  R. Palmer,et al.  Solution of 'Solvable model of a spin glass' , 1977 .

[15]  C. Soulé.  Graphic Requirements for Multistationarity , 2004, Complexus.

[16]  M. Samuelides,et al.  Random recurrent neural networks dynamics , 2006, math-ph/0612022.

[17]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[18]  A. Babloyantz,et al.  Predictability of human EEG: a dynamical approach , 1991, Biological Cybernetics.

[19]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[20]  Mathias Quoy,et al.  Structure and Dynamics of Random Recurrent Neural Networks , 2006, Adapt. Behav..

[21]  Emmanuel Daucé.  Adaptation dynamique et apprentissage dans des réseaux de neurones récurrents aléatoires. (Dynamic adaptation and learning in random recurrent neural networks) , 2000 .

[22]  J. Rogers.  Chaos , 1876 .

[23]  J. Finnigan.  How Nature Works; The science of self-organized criticality , 2001 .

[24]  Karl J. Friston,et al.  A neural mass model for MEG/EEG: coupling and neuronal dynamics , 2003, NeuroImage.

[25]  E. Adrian,et al.  The impulses produced by sensory nerve‐endings , 1926 .

[27]  Mathias Quoy,et al.  Random Recurrent Neural Networks for Autonomous System Design , 2000 .

[28]  A. Hodgkin,et al.  Currents carried by sodium and potassium ions through the membrane of the giant axon of Loligo , 1952, The Journal of physiology.

[29]  Abbott,et al.  Asynchronous states in networks of pulse-coupled oscillators. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[30]  Mathias Quoy,et al.  Resonant spatiotemporal learning in large random recurrent networks , 2002, Biological Cybernetics.

[31]  W. Singer,et al.  Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties , 1989, Nature.

[32]  R. Jaffard.  [Neurobiology of memory]. , 1991, La Revue du praticien.

[33]  Stanislas Dehaene,et al.  Neuronal models of cognitive functions , 1989, Cognition.

[34]  J. Milnor.  On the concept of attractor , 1985 .

[35]  Ben H. Jansen,et al.  Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns , 1995, Biological Cybernetics.

[36]  B. Cessac.  Occurrence of Chaos and AT Line in Random Neural Networks. , 1994 .

[37]  F. Takens,et al.  On the nature of turbulence , 1971 .

[38]  Sompolinsky,et al.  Dynamics of spin systems with randomly asymmetric bonds: Langevin dynamics and a spherical model. , 1987, Physical review. A, General physics.

[39]  B. Cessac,et al.  On bifurcations and chaos in random neural networks , 1994 .

[40]  B. Cessac,et al.  Stable resonances and signal propagation in a chaotic network of coupled units. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[41]  C. Nicholson.  Electric current flow in excitable cells J. J. B. Jack, D. Noble &R. W. Tsien Clarendon Press, Oxford (1975). 502 pp., £18.00 , 1976, Neuroscience.

[42]  P. Bak,et al.  Learning from mistakes , 1997, Neuroscience.

[43]  A. Babloyantz,et al.  Evidence of Chaotic Dynamics of Brain Activity During the Sleep Cycle , 1985 .

[44]  B. Cessac,et al.  What Can One Learn About Self-Organized Criticality from Dynamical Systems Theory? , 1999, cond-mat/9912081.

[45]  D. Ruelle.  Smooth Dynamics and New Theoretical Ideas in Nonequilibrium Statistical Mechanics , 1998, chao-dyn/9812032.

[46]  Micheal V. Mascagni.  Numerical methods for neuronal modeling , 1989 .

[47]  B. Hassard.  Bifurcation of periodic solutions of Hodgkin-Huxley model for the squid giant axon. , 1978, Journal of theoretical biology.

[48]  Sommers,et al.  Chaos in random neural networks. , 1988, Physical review letters.

[49]  James P. Keener,et al.  Mathematical physiology , 1998 .

[50]  K. Binder,et al.  Spin glasses: Experimental facts, theoretical concepts, and open questions , 1986 .

[51]  Bruno Cessac,et al.  Transmitting a signal by amplitude modulation in a chaotic network , 2005, Chaos.

[52]  Morris W. Hirsch,et al.  Convergent activation dynamics in continuous time networks , 1989, Neural Networks.

[53]  K. Adkins.  Theory of spin glasses , 1974 .

[54]  M. Poo,et al.  Propagation of activity-dependent synaptic depression in simple neural networks , 1997, Nature.

[55]  G. Ermentrout,et al.  Frequency Plateaus in a Chain of Weakly Coupled Oscillators, I. , 1984 .

[56]  S. Amari,et al.  Characteristics of Random Nets of Analog Neuron-Like Elements , 1972, IEEE Trans. Syst. Man Cybern..

[57]  F. H. Lopes da Silva,et al.  Models of neuronal populations: the basic mechanisms of rhythmicity. , 1976, Progress in brain research.

[58]  S. Kirkpatrick,et al.  Solvable Model of a Spin-Glass , 1975 .

[59]  M. Samuelides,et al.  Large deviations and mean-field theory for asymmetric random recurrent neural networks , 2002 .

[60]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[61]  J. Jack,et al.  Electric current flow in excitable cells , 1975 .

[62]  C. Morris,et al.  Voltage oscillations in the barnacle giant muscle fiber. , 1981, Biophysical journal.

[63]  Eve Marder,et al.  Reduction of conductance-based neuron models , 1992, Biological Cybernetics.

[64]  William Bialek,et al.  Reading a Neural Code , 1991, NIPS.

[65]  B. Cessac,et al.  CONTROL OF THE TRANSITION TO CHAOS IN NEURAL NETWORKS WITH RANDOM CONNECTIVITY , 1993 .

[66]  Westervelt,et al.  Dynamics of iterated-map neural networks. , 1989, Physical review. A, General physics.

[67]  S. Smale.  On the differential equations of species in competition , 1976, Journal of mathematical biology.

[68]  Mark Pollicott,et al.  On the rate of mixing of Axiom A flows , 1985 .

[69]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[70]  D. Ruelle,et al.  Ergodic theory of chaos and strange attractors , 1985 .

[71]  R. MacKay,et al.  Transition to topological chaos for circle maps , 1996 .

[72]  J. Rinzel,et al.  Numerical calculation of stable and unstable periodic solutions to the Hodgkin-Huxley equations , 1980 .

[73]  C. Carpenter,et al.  The Display of Enyaliosaurus clarki (Iguanidae: Lacertilia) , 1977 .

[74]  Christof Koch,et al.  Do neurons have a voltage or a current threshold for action potential initiation? , 1995, Journal of Computational Neuroscience.

[75]  R. Bowen.  Equilibrium States and the Ergodic Theory of Anosov Diffeomorphisms , 1975 .

[76]  A. Babloyantz,et al.  Low-dimensional chaos in an instance of epilepsy. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[77]  John Guckenheimer,et al.  Chaos in the Hodgkin-Huxley Model , 2002, SIAM J. Appl. Dyn. Syst..

[78]  E. D. Adrian,et al.  The Basis of Sensation , 1928, The Indian Medical Gazette.

[79]  H. Sullivan.  Ionic Channels of Excitable Membranes, 2nd Ed. , 1992, Neurology.

[80]  A. Bray,et al.  Metastable states in spin glasses , 1980 .

[81]  Isabel S. Labouriau,et al.  Degenerate Hopf bifurcation and nerve impulse, part II , 1989 .

[82]  S. Amari,et al.  A Mathematical Foundation for Statistical Neurodynamics , 1977 .

[83]  B. Cessac,et al.  Spontaneous dynamics and associative learning in an asymmetric recurrent random neural network , 1996 .

[84]  Per Bak,et al.  How Nature Works: The Science of Self‐Organized Criticality , 1997 .

[85]  Hal L. Smith.  Systems of ordinary differential equations which generate an order preserving flow. A survey of results , 1988 .

[86]  D. Thouless,et al.  Stability of the Sherrington-Kirkpatrick solution of a spin glass model , 1978 .

[87]  Mathias Quoy,et al.  Apprentissage dans les réseaux neuromimétiques à dynamique chaotique , 1994 .

[88]  B. Cessac.  Absolute stability criterion for discrete time neural networks , 1994 .

[89]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[90]  Olivier Faugeras,et al.  Analysis of Jansen's model of a single cortical column , 2006 .

[91]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[92]  Wulfram Gerstner,et al.  Spiking Neuron Models , 2002 .

[93]  S. Geman.  A Limit Theorem for the Norm of Random Matrices , 1980 .

[94]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[95]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[96]  Alan Edelman,et al.  The Circular Law and the Probability that a Random Matrix Has k Real Eigenvalues , 1993 .

[97]  B. Cessac,et al.  Mean-field equations, bifurcation map and route to chaos in discrete time neural networks , 1994 .

[98]  Ludwig Boltzmann,et al.  Lectures on Gas Theory , 1964 .

[99]  L. Abbott,et al.  Model neurons: From Hodgkin-Huxley to hopfield , 1990 .

[100]  Michael N. Shadlen,et al.  Noise, neural codes and cortical organization , 1994, Current Opinion in Neurobiology.

[101]  Steve Renals,et al.  Chaos in Neural Networks , 1990, EURASIP Workshop.

[102]  E. Izhikevich,et al.  Weakly connected neural networks , 1997 .

[103]  G. Ermentrout,et al.  Analysis of neural excitability and oscillations , 1989 .

[104]  A. Chenciner,et al.  Bifurcations de tores invariants , 1979 .

[105]  Bruno Cessac,et al.  Self-organization and dynamics reduction in recurrent networks: stimulus presentation and learning , 1998, Neural Networks.

[106]  J. P. Lasalle.  Stability theory for ordinary differential equations. , 1968 .

[107]  P. Holmes,et al.  Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields , 1983, Applied Mathematical Sciences.

[108]  R. Eckhorn,et al.  Coherent oscillations: A mechanism of feature linking in the visual cortex? , 1988, Biological Cybernetics.

[109]  W. Freeman,et al.  How brains make chaos in order to make sense of the world , 1987, Behavioral and Brain Sciences.

[110]  W. Freeman.  Simulation of chaotic EEG patterns with a dynamic model of the olfactory system , 1987, Biological Cybernetics.

[111]  J. Guckenheimer,et al.  Bifurcation of the Hodgkin and Huxley equations: A new twist , 1993 .

[112]  John Guckenheimer,et al.  Dynamical Systems: Some Computational Problems , 1993, chao-dyn/9304010.

[113]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[114]  S. Yoshizawa,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 1962, Proceedings of the IRE.

[115]  N. K. Rozov,et al.  Differential Equations with Small Parameters and Relaxation Oscillations , 1980 .

[116]  Y. Sinai.  GIBBS MEASURES IN ERGODIC THEORY , 1972 .

[117]  Yong Yao,et al.  Central pattern generating and recognizing in olfactory bulb: A correlation learning rule , 1988, Neural Networks.

[118]  B. Cessac,et al.  Self-Organized Criticality and Thermodynamic Formalism , 2002, nlin/0209038.

[119]  R. FitzHugh.  Impulses and Physiological States in Theoretical Models of Nerve Membrane. , 1961, Biophysical journal.

[120]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[121]  R. Keynes.  The ionic channels in excitable membranes. , 1975, Ciba Foundation symposium.

[122]  Schuster,et al.  Suppressing chaos in neural networks by noise. , 1992, Physical review letters.

[123]  J. J. Hopfield,et al.  Pattern recognition computation using action potential timing for stimulus representation , 1995, Nature.

[124]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[125]  J. Zukas.  Introduction to the Modern Theory of Dynamical Systems , 1998 .

[126]  R. Nicoll,et al.  Neurobiology: Long-distance long-term depression , 1997, Nature.

[127]  Daniel J. Amit,et al.  Modeling brain function: the world of attractor neural networks, 1st Edition , 1989 .

[128]  R. F. Williams,et al.  Expanding attractors , 1974 .

[129]  G. Basti,et al.  On the cognitive function of deterministic chaos in neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[130]  Masahiko Yoshioka.  Chaos synchronization in gap-junction-coupled neurons. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[131]  S. Kelso,et al.  Hebbian synapses in hippocampus. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[132]  B. Cessac.  A discrete time neural network model with spiking neurons , 2007, Journal of mathematical biology.

[133]  A Vulpiani,et al.  Relaxation of finite perturbations: beyond the fluctuation-response relation. , 2002, Chaos.

[134]  Vladimir Igorevich Arnolʹd,et al.  Problèmes ergodiques de la mécanique classique , 1967 .

[135]  I. Labouriau.  Degenerate Hopf Bifurcation and Nerve Impulse , 1985 .

[136]  B J Richmond,et al.  Stochastic nature of precisely timed spike patterns in visual system neuronal responses. , 1999, Journal of neurophysiology.

[137]  A. Babloyantz,et al.  The Creutzfeld-Jakob Disease in the Hierarchy of Chaotic Attractors , 1988 .

[138]  J. Gouzé.  Positive and Negative Circuits in Dynamical Systems , 1998 .

[139]  M. Abeles,et al.  Firing Rates and Weil-Timed Events in the Cerebral Cortex , 1994 .

[140]  B. Cessac,et al.  A dynamical system approach to SOC models of Zhang's type , 1997 .

[141]  René Thomas.  On the Relation Between the Logical Structure of Systems and Their Ability to Generate Multiple Steady States or Sustained Oscillations , 1981 .

[142]  Per Bak,et al.  The Discovery of Self-Organized Criticality , 1996 .