Biological learning mechanisms in spiking neuronal networks

IN THE present thesis biological learning mechanisms in spiking neuronal networks are investigated. In the brain, mechanisms at the molecular level modify the strengths (or weights) of the connections (synapses) between neurons, which is hypothesised to develop structure in neuronal networks depending on their activity; this learning mechanism is called ‘synaptic plasticity’. This thesis focuses on a particular physiological model of synaptic plasticity: spike-timing-dependent plasticity (STDP). Using a stochastic model of the spiking neuron, the Poisson neuron, a dynamical system is derived to predict the evolution of the weights and thus of the network activity using mathematical tools from stochastic processes and dynamical systems. The main aim of the study is to gain a better understanding of the weight dynamics induced by such synaptic plasticity in recurrent neuronal networks. The emergence of weight structure in a neuronal network is determined by the interplay between the main players: neuronal mechanisms, network connectivity, stimulating input structure and learning parameters. For a broad range of parameters, STDP can generate at the same time both a stabilisation of the mean incoming weight for each neuron, synonymous to stability of the firing rates in the network, and a diverging behaviour that induces neuronal specialisation to some of its incoming connections (both for input and recurrent weights). The results presented can be linked to cortical self-organisation, for example; STDP can lead to the emergence of neuronal groups sensitive to distinct input pathways. The resulting input selectivity provides a framework for ocular dominance in the primary visual cortex. The study of the learning dynamics also contributes to obtaining insight into the neuronal information processing that occurs in the brain: it indicates a time scale at which variations of the neuronal spiking probabilities are of importance, stresses the importance of the complete

[1]  B. Mandelbrot,et al.  RANDOM WALK MODELS FOR THE SPIKE ACTIVITY OF A SINGLE NEURON. , 1964, Biophysical journal.

[2]  Henry Markram,et al.  An Algorithm for Modifying Neurotransmitter Release Probability Based on Pre- and Postsynaptic Spike Timing , 2001, Neural Computation.

[3]  Terry Elliott,et al.  Multispike Interactions in a Stochastic Model of Spike-Timing-Dependent Plasticity , 2007, Neural Computation.

[4]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[5]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1990 .

[6]  W. Singer,et al.  Different voltage-dependent thresholds for inducing long-term depression and long-term potentiation in slices of rat visual cortex , 1990, Nature.

[7]  David W. Nauen,et al.  Coactivation and timing-dependent integration of synaptic potentiation and depression , 2005, Nature Neuroscience.

[8]  Terry Elliott,et al.  Temporal Dynamics of Rate-Based Synaptic Plasticity Rules in a Stochastic Model of Spike-Timing-Dependent Plasticity , 2008, Neural Computation.

[9]  L. Trussell,et al.  Cell-specific, spike timing–dependent plasticities in the dorsal cochlear nucleus , 2004, Nature Neuroscience.

[10]  Y. Dan,et al.  Spike-timing-dependent synaptic modification induced by natural spike trains , 2002, Nature.

[11]  N. Swindale.  The development of topography in the visual cortex: a review of models. , 1996, Network.

[12]  Robert S. Zucker,et al.  Postsynaptic Levels of [Ca2+]i Needed to Trigger LTD and LTP , 1996, Neuron.

[13]  A. Borovkov.  Ergodicity and stability of stochastic processes , 1998 .

[14]  M. Tsodyks.  Spike-timing-dependent synaptic plasticity – the long road towards understanding neuronal mechanisms of learning and memory , 2002, Trends in Neurosciences.

[15]  Mark C. W. van Rossum,et al.  Correlation based learning from spike timing dependent plasticity , 2001, Neurocomputing.

[16]  Nicolas Brunel,et al.  STDP in a Bistable Synapse Model Based on CaMKII and Associated Signaling Pathways , 2007, PLoS Comput. Biol..

[17]  Wulfram Gerstner,et al.  Intrinsic Stabilization of Output Rates by Spike-Based Hebbian Learning , 2001, Neural Computation.

[18]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[19]  C. Malsburg.  Self-organization of orientation sensitive cells in the striate cortex , 2004, Kybernetik.

[20]  Charlotte A. Boettiger,et al.  Developmentally Restricted Synaptic Plasticity in a Songbird Nucleus Required for Song Learning , 2001, Neuron.

[21]  Yves Frégnac,et al.  Biophysical and Phenomenological Models of Multiple Spike Interactions in Spike-timing Dependent Plasticity , 2006, Int. J. Neural Syst..

[22]  Anthony N. Burkitt,et al.  A Review of the Integrate-and-fire Neuron Model: I. Homogeneous Synaptic Input , 2006, Biological Cybernetics.

[23]  L. Trussell,et al.  Coactivation of Pre- and Postsynaptic Signaling Mechanisms Determines Cell-Specific Spike-Timing-Dependent Plasticity , 2007, Neuron.

[24]  M. Poo,et al.  Coincident Pre- and Postsynaptic Activity Modifies GABAergic Synapses by Postsynaptic Changes in Cl− Transporter Activity , 2003, Neuron.

[25]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[26]  W. Senn,et al.  Neocortical pyramidal cells respond as integrate-and-fire neurons to in vivo-like input currents. , 2003, Journal of neurophysiology.

[27]  Walter Senn,et al.  Beyond spike timing: the role of nonlinear plasticity and unreliable synapses , 2002, Biological Cybernetics.

[28]  Kenneth D. Miller,et al.  The Role of Constraints in Hebbian Learning , 1994, Neural Computation.

[29]  Rajesh P. N. Rao,et al.  Spike-Timing-Dependent Hebbian Plasticity as Temporal Difference Learning , 2001, Neural Computation.

[30]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[31]  R. L. Beurle.  Properties of a mass of cells capable of regenerating pulses , 1956, Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences.

[32]  D. Feldman,et al.  Timing-Based LTP and LTD at Vertical Inputs to Layer II/III Pyramidal Cells in Rat Barrel Cortex , 2000, Neuron.

[33]  P. Brémaud,et al.  STABILITY OF NONLINEAR HAWKES PROCESSES , 1996 .

[34]  M. Mayer,et al.  Voltage-dependent block by Mg2+ of NMDA responses in spinal cord neurones , 1984, Nature.

[35]  S. J. Martin,et al.  Synaptic plasticity and memory: an evaluation of the hypothesis. , 2000, Annual review of neuroscience.

[36]  Katsunori Kitano,et al.  Interplay between a phase response curve and spike-timing-dependent plasticity leading to wireless clustering. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[37]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[38]  K. Svoboda,et al.  The Life Cycle of Ca2+ Ions in Dendritic Spines , 2002, Neuron.

[39]  Laurent Massoulié,et al.  Stability results for a general class of interacting point processes dynamics, and applications , 1998 .

[40]  K. Miller,et al.  Synaptic Economics: Competition and Cooperation in Synaptic Plasticity , 1996, Neuron.

[41]  G. Edelman,et al.  Spike-timing dynamics of neuronal groups. , 2004, Cerebral cortex.

[42]  P. J. Sjöström,et al.  Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity , 2001, Neuron.

[43]  J. Doob.  Stochastic processes , 1953 .

[44]  L. Nowak,et al.  Magnesium gates glutamate-activated channels in mouse central neurones , 1984, Nature.

[45]  R. Stein.  A THEORETICAL ANALYSIS OF NEURONAL VARIABILITY. , 1965, Biophysical journal.

[46]  V. Han,et al.  Reversible Associative Depression and Nonassociative Potentiation at a Parallel Fiber Synapse , 2000, Neuron.

[47]  D. Johnston,et al.  A Synaptically Controlled, Associative Signal for Hebbian Plasticity in Hippocampal Neurons , 1997, Science.

[48]  Sander M. Bohte,et al.  Reducing the Variability of Neural Responses: A Computational Theory of Spike-Timing-Dependent Plasticity , 2007, Neural Computation.

[49]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[50]  Per Jesper Sjöström,et al.  Novel presynaptic mechanisms for coincidence detection in synaptic plasticity , 2006, Current Opinion in Neurobiology.

[51]  T. Sejnowski,et al.  Storing covariance with nonlinearly interacting neurons , 1977, Journal of mathematical biology.

[52]  R. Kempter,et al.  Hebbian learning and spiking neurons , 1999 .

[53]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[54]  F. Crépel,et al.  Homo‐ and heterosynaptic changes in efficacy are expressed in prefrontal neurons: An in vitro study in the rat , 1992, Synapse.

[55]  Lubica Benusková,et al.  STDP rule endowed with the BCM sliding threshold accounts for hippocampal heterosynaptic plasticity , 2007, Journal of Computational Neuroscience.

[56]  J. Leo van Hemmen,et al.  Spontaneously emerging direction selectivity maps in visual cortex through STDP , 2005, Biological Cybernetics.

[57]  John G. Taylor,et al.  Understanding spike-time-dependent plasticity: A biologically motivated computational model , 2006, Neurocomputing.

[58]  David B. Grayden,et al.  Spike-Timing-Dependent Plasticity: The Relationship to Rate-Based Learning for Models with Weight Dynamics Determined by a Stable Fixed Point , 2004, Neural Computation.

[59]  Nicolas Brunel,et al.  Fast Global Oscillations in Networks of Integrate-and-Fire Neurons with Low Firing Rates , 1999, Neural Computation.

[60]  Marco Tomassini,et al.  Dynamics of pruning in simulated large-scale spiking neural networks. , 2005, Bio Systems.

[61]  Terry Elliott,et al.  Synaptic and Temporal Ensemble Interpretation of Spike-Timing-Dependent Plasticity , 2005, Neural Computation.

[62]  Johannes C. Dahmen,et al.  Stimulus-Timing-Dependent Plasticity of Cortical Frequency Representation , 2008, The Journal of Neuroscience.

[63]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[64]  D. Debanne,et al.  Long‐term synaptic plasticity between pairs of individual CA3 pyramidal cells in rat hippocampal slice cultures , 1998, The Journal of physiology.

[65]  安田浩樹.  Long-term depression in rat visual cortex is associated with a lower rise of postsynaptic calcium than long-term potentiation(ラット視覚野における長期抑圧誘発時の長期増強の場合より低いシナプス後部カルシウム増加) , 1997 .

[66]  L. Abbott,et al.  Cortical Development and Remapping through Spike Timing-Dependent Plasticity , 2001, Neuron.

[67]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[68]  Eugene M. Izhikevich,et al.  Relating STDP to BCM , 2003, Neural Computation.

[69]  Luis Carrillo-Reid,et al.  Activation of the cholinergic system endows compositional properties to striatal cell assemblies. , 2009, Journal of neurophysiology.

[70]  M. Paradiso,et al.  Neuroscience: Exploring the Brain , 1996 .

[71]  Markus Diesmann,et al.  Spike-Timing-Dependent Plasticity in Balanced Random Networks , 2007, Neural Computation.

[72]  L. Abbott,et al.  Extending the effects of spike-timing-dependent plasticity to behavioral timescales. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[73]  M. Poo,et al.  Repeated cocaine exposure in vivo facilitates LTP induction in midbrain dopamine neurons , 2005, Nature.

[74]  Mark C. W. van Rossum,et al.  Stable Hebbian Learning from Spike Timing-Dependent Plasticity , 2000, The Journal of Neuroscience.

[75]  J. Lisman,et al.  A mechanism for the Hebb and the anti-Hebb processes underlying learning and memory. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[76]  M. Jacobsen.  Point Process Theory and Applications: Marked Point and Piecewise Deterministic Processes , 2005 .

[77]  Quan Zou,et al.  Kinetic models of spike-timing dependent plasticity and their functional consequences in detecting correlations , 2007, Biological Cybernetics.

[78]  Thomas P. Trappenberg,et al.  Computational consequences of experimentally derived spike-time and weight dependent plasticity rules , 2007, Biological Cybernetics.

[79]  Laurenz Wiskott,et al.  Slowness: An Objective for Spike-Timing–Dependent Plasticity? , 2007, PLoS Comput. Biol..

[80]  Hugues Bersini,et al.  The Road to Chaos by Time-Asymmetric Hebbian Learning in Recurrent Neural Networks , 2007, Neural Computation.

[81]  Terry Elliott,et al.  Stable Competitive Dynamics Emerge from Multispike Interactions in a Stochastic Model of Spike-Timing-Dependent Plasticity , 2006, Neural Computation.

[82]  R. Colbran,et al.  Protein Phosphatases and Calcium/Calmodulin-Dependent Protein Kinase II-Dependent Synaptic Plasticity , 2004, The Journal of Neuroscience.

[83]  Stefano Fusi,et al.  Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates , 2002, Biological Cybernetics.

[84]  Arnaud Delorme,et al.  Networks of integrate-and-fire neurons using Rank Order Coding B: Spike timing dependent plasticity and emergence of orientation selectivity , 2001, Neurocomputing.

[85]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[86]  W. R. Adey,et al.  Firing variability in cat association cortex during sleep and wakefulness. , 1970, Brain research.

[87]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[88]  Jan Karbowski,et al.  Synchrony arising from a balanced synaptic plasticity in a network of heterogeneous neural oscillators. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[89]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[90]  Carson C. Chow,et al.  Calcium time course as a signal for spike-timing-dependent plasticity. , 2005, Journal of neurophysiology.

[91]  Risto Miikkulainen,et al.  Self-organization and segmentation in a laterally connected orientation map of spiking neurons , 1998, Neurocomputing.

[92]  Y. Dan,et al.  Spike timing-dependent plasticity: from synapse to perception. , 2006, Physiological reviews.

[93]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[94]  John M. Beggs,et al.  A Maximum Entropy Model Applied to Spatial and Temporal Correlations from Cortical Networks In Vitro , 2008, The Journal of Neuroscience.

[95]  Nicolas Brunel,et al.  Dynamics of a recurrent network of spiking neurons before and following learning , 1997 .

[96]  P Lánský,et al.  On approximations of Stein's neuronal model. , 1984, Journal of theoretical biology.

[97]  B. Sakmann,et al.  Coincidence detection and changes of synaptic efficacy in spiny stellate neurons in rat barrel cortex , 1999, Nature Neuroscience.

[98]  A. Burkitt,et al.  Learning the structure of correlated synaptic subgroups using stable and competitive spike-timing-dependent plasticity. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[99]  David B. Grayden,et al.  Gain Modulation and Balanced Synaptic Input in a Conductance-Based Neural Model , 2002, Neurocomputing.

[100]  Wulfram Gerstner,et al.  Extracting Oscillations: Neuronal Coincidence Detection with Noisy Periodic Spike Input , 1998, Neural Computation.

[101]  T. Bliss,et al.  Long‐lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path , 1973, The Journal of physiology.

[102]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[103]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[104]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[105]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[106]  H. Markram,et al.  Physiology and anatomy of synaptic connections between thick tufted pyramidal neurones in the developing rat neocortex. , 1997, The Journal of physiology.

[107]  David B. Grayden,et al.  An Analytical Model for the ‘Large, Fluctuating Synaptic Conductance State’ Typical of Neocortical Neurons In Vivo , 2004, Journal of Computational Neuroscience.

[108]  Stefan Rotter,et al.  Correlations and Population Dynamics in Cortical Networks , 2008, Neural Computation.

[109]  W. Levy,et al.  Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus , 1983, Neuroscience.

[110]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[111]  Evgueniy V. Lubenov,et al.  Decoupling through Synchrony in Neuronal Circuits with Propagation Delays , 2008, Neuron.

[112]  N. Shadbolt,et al.  A Neurotrophic Model of the Development of the Retinogeniculocortical Pathway Induced by Spontaneous Retinal Waves , 1999, The Journal of Neuroscience.

[113]  G. Poggio,et al.  TIME SERIES ANALYSIS OF IMPULSE SEQUENCES OF THALAMIC SOMATIC SENSORY NEURONS. , 1964, Journal of neurophysiology.

[114]  R. Kempter,et al.  How spiking neurons give rise to a temporal-feature map: from synaptic plasticity to axonal selection. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[115]  Y. Dan,et al.  Spike-timing-dependent synaptic plasticity depends on dendritic location , 2005, Nature.

[116]  H. W. Veen,et al.  Handbook of Biological Physics , 1996 .

[117]  Anthony N. Burkitt.  Balanced neurons: analysis of leaky integrate-and-fire neurons with reversal potentials , 2001, Biological Cybernetics.

[118]  G. Ermentrout,et al.  Reliability, synchrony and noise , 2008, Trends in Neurosciences.

[119]  Katsunori Kitano,et al.  Structure of Spontaneous UP and DOWN Transitions Self-Organizing in a Cortical Network Model , 2008, PLoS Comput. Biol..

[120]  Andrew Carnell.  An analysis of the use of Hebbian and Anti-Hebbian spike time dependent plasticity learning functions within the context of recurrent spiking neural networks , 2009, Neurocomputing.

[121]  S. Amari.  Dynamics of pattern formation in lateral-inhibition type neural fields , 1977, Biological Cybernetics.

[122]  Matthieu Gilson,et al.  Spike-timing-dependent plasticity for neurons with recurrent connections , 2007, Biological Cybernetics.

[123]  Néstor Parga,et al.  Theory of Input Spike Auto- and Cross-Correlations and Their Effect on the Response of Spiking Neurons , 2007, Neural Computation.

[124]  Stephen Coombes,et al.  Waves, bumps, and patterns in neural field theories , 2005, Biological Cybernetics.

[125]  Mark H. A. Davis.  Piecewise‐Deterministic Markov Processes: A General Class of Non‐Diffusion Stochastic Models , 1984 .

[126]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[127]  C. Schreiner,et al.  Modular organization of frequency integration in primary auditory cortex. , 2000, Annual review of neuroscience.

[128]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[129]  Terrence J. Sejnowski,et al.  Integrate-and-Fire Neurons Driven by Correlated Stochastic Input , 2002, Neural Computation.

[130]  Wulfram Gerstner,et al.  A benchmark test for a quantitative assessment of simple neuron models , 2008, Journal of Neuroscience Methods.

[131]  J. Montgomery,et al.  State-Dependent Heterogeneity in Synaptic Depression between Pyramidal Cell Pairs , 2002, Neuron.

[132]  C. Holmgren,et al.  Coincident Spiking Activity Induces Long-Term Changes in Inhibition of Neocortical Pyramidal Cells , 2001, The Journal of Neuroscience.

[133]  G. Goodhill.  Contributions of Theoretical Modeling to the Understanding of Neural Map Development , 2007, Neuron.

[134]  Haim Sompolinsky,et al.  Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity , 2003, The Journal of Neuroscience.

[135]  A. Hawkes.  Point Spectra of Some Mutually Exciting Point Processes , 1971 .

[136]  Martha White,et al.  Optimal reverse prediction: a unified perspective on supervised, unsupervised and semi-supervised learning , 2009, ICML '09.

[137]  Y. Dan,et al.  Spike timing-dependent plasticity: a Hebbian learning rule. , 2008, Annual review of neuroscience.

[138]  Boris S. Gutkin,et al.  Spike Generating Dynamics and the Conditions for Spike-Time Precision in Cortical Neurons , 2003, Journal of Computational Neuroscience.

[139]  Naoki Masuda,et al.  Formation of feedforward networks and frequency synchrony by spike-timing-dependent plasticity , 2007, Journal of Computational Neuroscience.

[140]  W. Gerstner,et al.  Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[141]  S. Thorpe,et al.  Spike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike Trains , 2008, PloS one.