Combining Supervised, Unsupervised, and Reinforcement Learning in a Network of Spiking Neurons

The human brain constantly learns via mutiple different learning strategies. It can learn by simply having stimuli being presented to its sensory organs which is considered unsupervised learning. In addition, it can learn associations between inputs and outputs when a teacher provides the output which is considered as supervised learning. Most importantly, it can learn very efficiently if correct behaviour is followed by reward and/or incorrect behaviour is followed by punishment which is considered reinforcement learning. So far, most artificial neural architectures implement only one of the three learning mechanisms — even though the brain integrates all three. Here, we have implemented unsupervised, supervised, and reinforcement learning within a network of spiking neurons. In order to achieve this ambitious goal, the existing learning rule called spike-timing-dependent plasticity had to be extended such that it is modulated by the reward signal dopamine.

[1]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[2]  G. Turrigiano The Self-Tuning Neuron: Synaptic Scaling of Excitatory Synapses , 2008, Cell.

[3]  David T. J. Liley,et al.  Simulation of electrocortical waves , 1995, Biological Cybernetics.

[4]  Mingqi Deng,et al.  Winner-take-all networks , 1992 .

[5]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[6]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[7]  Bernd Michaelis,et al.  Contribution of the GABA shift to the transition from structural initialization to working stage in biologically realistic networks , 2008, Neurocomputing.

[8]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[9]  Christoph S. Herrmann,et al.  Prerequisites for integrating unsupervised and reinforcement learning in a single network of spiking neurons , 2009, 2009 International Joint Conference on Neural Networks.

[10]  P. Greengard,et al.  Dichotomous Dopaminergic Control of Striatal Synaptic Plasticity , 2008, Science.

[11]  R. Desimone,et al.  Neural mechanisms for visual memory and their role in attention. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Walter J. Freeman,et al.  A field-theoretic approach to understanding scale-free neocortical dynamics , 2005, Biological Cybernetics.

[13]  Christoph S. Herrmann,et al.  A Biologically Plausible Winner-Takes-All Architecture , 2009, ICIC.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[16]  J. Bolam,et al.  Uniform Inhibition of Dopamine Neurons in the Ventral Tegmental Area by Aversive Stimuli , 2004, Science.

[17]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[18]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[19]  Bruno A. Olshausen,et al.  Book Review , 2003, Journal of Cognitive Neuroscience.

[20]  R. Desimone,et al.  Object and place memory in the macaque entorhinal cortex. , 1997, Journal of neurophysiology.

[21]  Christoph Herrmann,et al.  Simulating Evoked Gamma Oscillations of Human EEG in a Network of Spiking Neurons Reveals an Early Mechanism of Memory Matching , 2007 .

[22]  G. Edelman,et al.  Spike-timing dynamics of neuronal groups. , 2004, Cerebral cortex.

[23]  W. Schultz Dopamine neurons and their role in reward mechanisms , 1997, Current Opinion in Neurobiology.

[24]  Jürgen Kurths,et al.  Simulating global properties of electroencephalograms with minimal random neural networks , 2008, Neurocomputing.

[25]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[26]  Karl J. Friston,et al.  A neural mass model for MEG/EEG: coupling and neuronal dynamics , 2003, NeuroImage.