A Scalable Weight-Free Learning Algorithm for Regulatory Control of Cell Activity in Spiking Neuronal Networks

Recent developments in neural stimulation and recording technologies are providing scientists with the ability of recording and controlling the activity of individual neurons in vitro or in vivo, with very high spatial and temporal resolution. Tools such as optogenetics, for example, are having a significant impact in the neuroscience field by delivering optical firing control with the precision and spatiotemporal resolution required for investigating information processing and plasticity in biological brains. While a number of training algorithms have been developed to date for spiking neural network (SNN) models of biological neuronal circuits, exiting methods rely on learning rules that adjust the synaptic strengths (or weights) directly, in order to obtain the desired network-level (or functional-level) performance. As such, they are not applicable to modifying plasticity in biological neuronal circuits, in which synaptic strengths only change as a result of pre- and post-synaptic neuron firings or biological mechanisms beyond our control. This paper presents a weight-free training algorithm that relies solely on adjusting the spatiotemporal delivery of neuron firings in order to optimize the network performance. The proposed weight-free algorithm does not require any knowledge of the SNN model or its plasticity mechanisms. As a result, this training approach is potentially realizable in vitro or in vivo via neural stimulation and recording technologies, such as optogenetics and multielectrode arrays, and could be utilized to control plasticity at multiple scales of biological neuronal circuits. The approach is demonstrated by training SNNs with hundreds of units to control a virtual insect navigating in an unknown environment.

[1]  Razvan V. Florian,et al.  Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity , 2007, Neural Computation.

[2]  Qing Song,et al.  Adaptive learning rate of SpikeProp based on weight convergence analysis , 2015, Neural Networks.

[3]  Malek Adjouadi,et al.  A generalized leaky Integrate-and-Fire Neuron Model with Fast Implementation Method , 2014, Int. J. Neural Syst..

[4]  P. Holmes,et al.  Neuromechanical models for insect locomotion: Stability, maneuverability, and proprioceptive feedback. , 2009, Chaos.

[5]  Krzysztof J. Cios,et al.  Simulating Vertical and Horizontal Inhibition with Short-Term Dynamics in a Multi-column Multi-Layer Model of neocortex , 2014, Int. J. Neural Syst..

[6]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[7]  Brian R. Tietz,et al.  Deciding Which Way to Go: How Do Insects Alter Movements to Negotiate Barriers? , 2012, Front. Neurosci..

[8]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[9]  Henry Markram,et al.  Computer models and analysis tools for neural microcircuits , 2003 .

[10]  Haruhiko Nishimura,et al.  Enhancement of Spike-Timing-Dependent Plasticity in Spiking Neural Systems with Noise , 2016, Int. J. Neural Syst..

[11]  Nikola Kasabov,et al.  Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. , 2013, Neural networks : the official journal of the International Neural Network Society.

[12]  Hojjat Adeli,et al.  Spiking Neural Networks , 2009, Int. J. Neural Syst..

[13]  Neal Sweeney,et al.  Synaptic Strength Regulated by Palmitate Cycling on PSD-95 , 2002, Cell.

[14]  André van Schaik.  Building blocks for electronic spiking neural networks , 2001, Neural Networks.

[15]  V. Mountcastle.  Modality and topographic properties of single neurons of cat's somatic sensory cortex. , 1957, Journal of neurophysiology.

[16]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[17]  Sander M. Bohte,et al.  Unsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks , 2002, IEEE Trans. Neural Networks.

[18]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[19]  R. Ritzmann,et al.  Central-Complex Control of Movement in the Freely Walking Cockroach , 2015, Current Biology.

[20]  X. Zhang,et al.  Digital implementation of a virtual insect trained by spike-timing dependent plasticity , 2016, Integr..

[21]  T. Sejnowski,et al.  Ionic mechanisms underlying synchronized oscillations and propagating waves in a model of ferret thalamic slices. , 1996, Journal of neurophysiology.

[22]  Y. Dan,et al.  Receptive-Field Modification in Rat Visual Cortex Induced by Paired Visual Stimulation and Single-Cell Spiking , 2006, Neuron.

[23]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[24]  Stefan Schliebs,et al.  Training spiking neural networks to associate spatio-temporal input-output spike patterns , 2013, Neurocomputing.

[25]  Walter Senn,et al.  Code-Specific Learning Rules Improve Action Selection by Populations of Spiking Neurons , 2014, Int. J. Neural Syst..

[26]  Antonius M J VanDongen,et al.  Short-Term Memory in Networks of Dissociated Cortical Neurons , 2013, The Journal of Neuroscience.

[27]  Hojjat Adeli,et al.  A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection , 2009, Neural Networks.

[28]  Pinaki Mazumder,et al.  Digital implementation of a spiking neural network (SNN) capable of spike-timing-dependent plasticity (STDP) learning , 2014, 14th IEEE International Conference on Nanotechnology.

[29]  Ansgar Büschges,et al.  Adaptive motor behavior in insects , 2007, Current Opinion in Neurobiology.

[30]  Ferrante Neri,et al.  An Optimization Spiking Neural P System for Approximately Solving Combinatorial Optimization Problems , 2014, Int. J. Neural Syst..

[31]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[32]  Christopher J. Rozell,et al.  Optimal Sparse Approximation with Integrate and Fire Neurons , 2014, Int. J. Neural Syst..

[33]  Ammar Belatreche,et al.  DL-ReSuMe: A Delay Learning-Based Remote Supervised Method for Spiking Neurons , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Han Ju,et al.  Spatiotemporal Memory Is an Intrinsic Property of Networks of Dissociated Cortical Neurons , 2015, The Journal of Neuroscience.

[35]  W. Levy,et al.  Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus , 1983, Neuroscience.

[36]  R. Douglas,et al.  Long-term potentiation of the perforant path-granule cell synapse in the rat hippocampus , 1975, Brain Research.

[37]  Y. Dan,et al.  Spike timing-dependent plasticity: a Hebbian learning rule. , 2008, Annual review of neuroscience.

[38]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[39]  D. Hubel,et al.  Receptive fields of single neurones in the cat's striate cortex , 1959, The Journal of physiology.

[40]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[41]  Xingyu Wang,et al.  Sparse Bayesian Learning for Obtaining Sparsity of EEG Frequency Bands Based Feature Vectors in Motor Imagery Classification , 2017, Int. J. Neural Syst..

[42]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[43]  C. Pennartz.  Reinforcement learning by Hebbian synapses with adaptive thresholds , 1997, Neuroscience.

[44]  Razvan V. Florian,et al.  The Chronotron: A Neuron That Learns to Fire Temporally Precise Spike Patterns , 2010, PloS one.

[45]  Jeffrey L. Krichmar,et al.  Brain-Based Devices for the Study of Nervous Systems and the Development of Intelligent Machines , 2005, Artificial Life.

[46]  G. Ermentrout,et al.  Coupled oscillators and the design of central pattern generators , 1988 .

[47]  Silvia Ferrari,et al.  Indirect training of a spiking neural network for flight control via spike-timing-dependent synaptic plasticity , 2010, 49th IEEE Conference on Decision and Control (CDC).

[48]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[49]  Wolfgang Maass,et al.  A Reward-Modulated Hebbian Learning Rule Can Explain Experimentally Observed Network Reorganization in a Brain Control Task , 2010, The Journal of Neuroscience.