Learning flexible sensori-motor mappings in a complex network

Given the complex structure of the brain, how can synaptic plasticity explain the learning and forgetting of associations when these are continuously changing? We address this question by studying different reinforcement learning rules in a multilayer network in order to reproduce monkey behavior in a visuomotor association task. Our model can only reproduce the learning performance of the monkey if the synaptic modifications depend on the pre- and postsynaptic activity, and if the intrinsic level of stochasticity is low. This favored learning rule is based on reward modulated Hebbian synaptic plasticity and shows the interesting feature that the learning performance does not substantially degrade when adding layers to the network, even for a complex problem.

[1]  Xiao-Jing Wang,et al.  The Frontal Lobes: A microcircuit model of prefrontal functions: ying and yang of reverberatory neurodynamics in cognition , 2006 .

[2]  Xiao-Jing Wang,et al.  A Biophysically Based Neural Model of Matching Law Behavior: Melioration by Stochastic Synapses , 2006, The Journal of Neuroscience.

[3]  W. Senn,et al.  Convergence of stochastic learning in perceptrons with binary synapses. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  P. Bak,et al.  Learning from mistakes , 1997, Neuroscience.

[5]  Domenico Parisi,et al.  Evolutionary Connectionism and Mind/Brain Modularity , 2001 .

[6]  Walter Senn,et al.  Learning Only When Necessary: Better Memories of Correlated Patterns in Networks with Bounded Synapses , 2005, Neural Computation.

[7]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Walter Senn,et al.  Eluding oblivion with smart stochastic selection of synaptic updates. , 2006, Chaos.

[9]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[10]  Xiaohui Xie,et al.  Learning Curves for Stochastic Gradient Descent in Linear Feedforward Networks , 2003, Neural Computation.

[11]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[12]  L. Abbott,et al.  Cascade Models of Synaptically Stored Memories , 2005, Neuron.

[13]  E. Miller,et al.  A Neural Circuit Model of Flexible Sensorimotor Mapping: Learning and Forgetting on Multiple Timescales , 2007, Neuron.

[14]  Daniel J. Amit,et al.  Learning in Neural Networks with Material Synapses , 1994, Neural Computation.

[15]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[16]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[17]  P. Bak,et al.  Adaptive learning by extremal dynamics and negative feedback. , 2000, Physical review. E, Statistical, nonlinear, and soft matter physics.

[18]  Peter Dayan,et al.  Optimal Plasticity from Matrix Memories: What Goes Up Must Come Down , 1990, Neural Computation.

[19]  Richard Durbin,et al.  The computing neuron , 1989 .

[20]  H. Horner,et al.  Storage capacity of a two-layer perceptron with fixed preprocessing in the first layer , 1994 .

[21]  E. Miller,et al.  Neural Activity in the Primate Prefrontal Cortex during Associative Learning , 1998, Neuron.

[22]  Michael I. Jordan,et al.  A more biologically plausible learning rule for neural networks. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[24]  P. Dayan,et al.  Optimising synaptic learning rules in linear associative memories , 1991, Biological Cybernetics.

[25]  Xiao-Jing Wang,et al.  Neural mechanism for stochastic behaviour during a competitive game , 2006, Neural Networks.

[26]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[27]  Ila R Fiete,et al.  Gradient learning in spiking neural networks by dynamic perturbation of conductances. , 2006, Physical review letters.

[28]  Eduardo D. Sontag,et al.  Feedback Stabilization Using Two-Hidden-Layer Nets , 1991, 1991 American Control Conference.

[29]  A G Barto,et al.  Learning by statistical cooperation of self-interested neuron-like computing elements. , 1985, Human neurobiology.

[30]  Andrew G. Barto,et al.  From Chemotaxis to cooperativity: abstract exercises in neuronal learning strategies , 1989 .

[31]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[32]  G. Wagner,et al.  A case study of the evolution of modularity: towards a bridge between evolutionary biology, artificial life, neuro- and cognitive science , 1998 .

[33]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[34]  Jason Weston,et al.  Large-scale kernel machines , 2007 .