Learning Universal Computations with Spikes

Providing the neurobiological basis of information processing in higher animals, spiking neural networks must be able to learn a variety of complicated computations, including the generation of appropriate, possibly delayed reactions to inputs and the self-sustained generation of complex activity patterns, e.g. for locomotion. Many such computations require previous building of intrinsic world models. Here we show how spiking neural networks may solve these different tasks. Firstly, we derive constraints under which classes of spiking neural networks lend themselves to substrates of powerful general purpose computing. The networks contain dendritic or synaptic nonlinearities and have a constrained connectivity. We then combine such networks with learning rules for outputs or recurrent connections. We show that this allows to learn even difficult benchmark tasks such as the self-sustained generation of desired low-dimensional chaotic dynamics or memory-dependent computations. Furthermore, we show how spiking networks can build models of external world systems and use the acquired knowledge to control them.

[1]  M. London,et al.  Dendritic computation. , 2005, Annual review of neuroscience.

[2]  Adrienne L Fairhall,et al.  Constructing Precisely Computing Networks with Biophysical Spiking Neurons , 2014, The Journal of Neuroscience.

[3]  Sophie Denève,et al.  Enforcing balance allows local supervised learning in spiking recurrent networks , 2015, NIPS.

[4]  Sven Jahnke,et al.  How Chaotic is the Balanced State? , 2009, Front. Comput. Neurosci..

[5]  Judit K. Makara,et al.  Compartmentalized dendritic plasticity and input feature storage in neurons , 2008, Nature.

[6]  Gordon Pipa,et al.  SORN: A Self-Organizing Recurrent Neural Network , 2009, Front. Comput. Neurosci..

[7]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[8]  Robert A. Legenstein,et al.  2007 Special Issue: Edge of chaos and prediction of computational performance for neural circuit models , 2007 .

[9]  Jon T. Brown,et al.  Activity‐dependent depression of the spike after‐depolarization generates long‐lasting intrinsic plasticity in hippocampal CA3 pyramidal neurons , 2009, The Journal of physiology.

[10]  B. McNaughton,et al.  Hippocampus Leads Ventral Striatum in Replay of Place-Reward Information , 2009, PLoS biology.

[11]  H. Sebastian Seung,et al.  The Autapse: A Simple Illustration of Short-Term Analog Memory Storage by Tuned Synaptic Feedback , 2004, Journal of Computational Neuroscience.

[12]  Raoul-Martin Memmesheimer,et al.  Quantitative prediction of intermittent high-frequency oscillations in neural networks with supralinear dendritic interactions , 2010, Proceedings of the National Academy of Sciences.

[13]  M. Häusser,et al.  Synaptic Integration Gradients in Single Cortical Pyramidal Cell Dendrites , 2011, Neuron.

[14]  L. F. Abbott,et al.  Building functional networks of spiking model neurons , 2016, Nature Neuroscience.

[15]  M. Nicolelis,et al.  Interval timing and the encoding of signal duration by ensembles of cortical and striatal neurons. , 2003, Behavioral neuroscience.

[16]  Morris W. Hirsch,et al.  Convergent activation dynamics in continuous time networks , 1989, Neural Networks.

[17]  Christian K. Machens,et al.  Predictive Coding of Dynamical Variables in Balanced Spiking Networks , 2013, PLoS Comput. Biol..

[18]  Eduardo D. Sontag,et al.  Computational Aspects of Feedback in Neural Circuits , 2006, PLoS Comput. Biol..

[19]  Stefan Schaal,et al.  Skill learning and task outcome prediction for manipulation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[20]  David Sussillo,et al.  Opening the Black Box: Low-Dimensional Dynamics in High-Dimensional Recurrent Neural Networks , 2013, Neural Computation.

[21]  H. Sompolinsky,et al.  Transition to chaos in random neuronal networks , 2015, 1508.06486.

[22]  J. Kurths,et al.  Chaos–order transition in foraging behavior of ants , 2014, Proceedings of the National Academy of Sciences.

[23]  I. Raman,et al.  Resurgent Sodium Current and Action Potential Formation in Dissociated Cerebellar Purkinje Neurons , 1997, The Journal of Neuroscience.

[24]  Peter E. Latham,et al.  Randomly Connected Networks Have Short Temporal Memory , 2013, Neural Computation.

[25]  Wulfram Gerstner,et al.  Spiking Neuron Models: Single Neurons, Populations, Plasticity , 2002 .

[26]  Sophie Denève,et al.  Spike-Based Population Coding and Working Memory , 2011, PLoS Comput. Biol..

[27]  Matthijs A. A. van der Meer,et al.  Frontiers in Integrative Neuroscience Integrative Neuroscience Covert Expectation-of-reward in Rat Ventral Striatum at Decision Points , 2022 .

[28]  C. Hofsten Eye–hand coordination in the newborn. , 1982 .

[29]  Marc Timme,et al.  Topological speed limits to network synchronization. , 2003, Physical review letters.

[30]  H. Markram,et al.  Frequency and Dendritic Distribution of Autapses Established by Layer 5 Pyramidal Neurons in the Developing Rat Neocortex: Comparison with Synaptic Innervation of Adjacent Neurons of the Same Class , 1996, The Journal of Neuroscience.

[31]  R. Ivry,et al.  The neural representation of time , 2004, Current Opinion in Neurobiology.

[32]  Benjamin Schrauwen,et al.  Reservoir Computing Trends , 2012, KI - Künstliche Intelligenz.

[33]  Stefan Schaal,et al.  Reinforcement learning of motor skills in high dimensions: A path integral approach , 2010, 2010 IEEE International Conference on Robotics and Automation.

[34]  M. Shadlen,et al.  A role for neural integrators in perceptual decision making. , 2003, Cerebral cortex.

[35]  P. Dayan,et al.  Supporting Online Material Materials and Methods Som Text Figs. S1 to S9 References the Asynchronous State in Cortical Circuits , 2022 .

[36]  Sten Grillner,et al.  Biological Pattern Generation: The Cellular and Computational Logic of Networks in Motion , 2006, Neuron.

[37]  Stefan J. Kiebel,et al.  Re-visiting the echo state property , 2012, Neural Networks.

[38]  Mark M. Churchland,et al.  Using Firing-Rate Dynamics to Train Recurrent Networks of Spiking Model Neurons , 2016, 1601.07620.

[39]  Christian K. Machens,et al.  Learning optimal spike-based representations , 2012, NIPS.

[40]  Y. Yaari,et al.  Spike Ca2+ influx upmodulates the spike afterdepolarization and bursting via intracellular inhibition of KV7/M channels , 2008, The Journal of physiology.

[41]  R. Passingham,et al.  Prefrontal interactions reflect future task operations , 2003, Nature Neuroscience.

[42]  B. Bean The action potential in mammalian central neurons , 2007, Nature Reviews Neuroscience.

[43]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[44]  B. McNaughton,et al.  Preferential Reactivation of Motivationally Relevant Information in the Ventral Striatum , 2008, The Journal of Neuroscience.

[45]  Karl J. Friston What Is Optimal about Motor Control? , 2011, Neuron.

[46]  Christian Lüscher,et al.  G Protein-Coupled Inwardly Rectifying K+ Channels (GIRKs) Mediate Postsynaptic but Not Presynaptic Transmitter Actions in Hippocampal Neurons , 1997, Neuron.

[47]  A. Litwin-Kumar,et al.  Formation and maintenance of neuronal assemblies through synaptic plasticity , 2014, Nature Communications.

[48]  H. Kappen Linear theory for control of nonlinear stochastic systems. , 2004, Physical review letters.

[49]  Trevor Bekolay,et al.  Supplementary Materials for A Large-Scale Model of the Functioning Brain , 2012 .

[50]  Haim Sompolinsky,et al.  Short-term memory in orthogonal neural networks. , 2004, Physical review letters.

[51]  Emanuel Todorov,et al.  Efficient computation of optimal actions , 2009, Proceedings of the National Academy of Sciences.

[52]  Wolfgang Maass,et al.  Movement Generation with Circuits of Spiking Neurons , 2005, Neural Computation.

[53]  L. Abbott,et al.  Synaptic computation , 2004, Nature.

[54]  J. Bekkers Synaptic Transmission: Excitatory Autapses Find a Function? , 2009, Current Biology.

[55]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[56]  Srdjan Ostojic,et al.  Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons , 2014, Nature Neuroscience.

[57]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[58]  Wolfgang Maass,et al.  Emergence of Dynamic Memory Traces in Cortical Microcircuit Models through STDP , 2013, The Journal of Neuroscience.

[59]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[60]  Everton J. Agnes,et al.  Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks , 2015, Nature Communications.

[61]  Robert E. Hampson,et al.  Temporal coupling between subicular and hippocampal neurons underlies retention of trial-specific events , 2006, Behavioural Brain Research.

[62]  Chris Eliasmith,et al.  A Unified Approach to Building and Controlling Spiking Attractor Networks , 2005, Neural Computation.

[63]  J. Storm,et al.  Action potential repolarization and a fast after‐hyperpolarization in rat hippocampal pyramidal cells. , 1987, The Journal of physiology.

[64]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[65]  Brad E. Pfeiffer,et al.  Hippocampal place cell sequences depict future paths to remembered goals , 2013, Nature.

[66]  L. Abbott,et al.  Transferring Learning from External to Internal Weights in Echo-State Networks with Sparse Connectivity , 2012, PloS one.

[67]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[68]  Wade G. Regehr,et al.  Short-term synaptic plasticity: a comparison of two synapses , 2004, Nature Reviews Neuroscience.

[69]  Costas Papatheodoropoulos,et al.  The GABAA receptor-mediated recurrent inhibition in ventral compared with dorsal CA1 hippocampal region is weaker, decays faster and lasts less , 2007, Experimental Brain Research.

[70]  S. Barry Cooper,et al.  Computability In Context: Computation and Logic in the Real World , 2009 .

[71]  W. Gerstner,et al.  Signal buffering in random networks of spiking neurons: microscopic versus macroscopic phenomena. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[72]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[73]  Sommers,et al.  Chaos in random neural networks. , 1988, Physical review letters.

[74]  H. Seung,et al.  Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. , 2007, Journal of neurophysiology.

[75]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[76]  David Hansel,et al.  Asynchronous Rate Chaos in Spiking Neuronal Circuits , 2015, bioRxiv.

[77]  M. Tsodyks,et al.  Working models of working memory , 2014, Current Opinion in Neurobiology.

[78]  W. Gerstner,et al.  Optimal Control of Transient Dynamics in Balanced Networks Supports Generation of Complex Movements , 2014, Neuron.

[79]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[80]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[81]  Stefan Schaal,et al.  Variable Impedance Control - A Reinforcement Learning Approach , 2010, Robotics: Science and Systems.

[82]  Boris S. Gutkin,et al.  Passive Dendrites Enable Single Neurons to Compute Linearly Non-separable Functions , 2013, PLoS Comput. Biol..

[83]  Matthijs A. A. van der Meer,et al.  Expectancies in Decision Making, Reinforcement Learning, and Ventral Striatum , 2009, Frontiers in neuroscience.

[84]  Wolfgang Maass,et al.  Liquid State Machines: Motivation, Theory, and Applications , 2010 .

[85]  Huaixing Wang,et al.  A specialized NMDA receptor function in layer 5 recurrent microcircuitry of the adult rat prefrontal cortex , 2008, Proceedings of the National Academy of Sciences.