Rapid learning with highly localized synaptic plasticity

The brains of all animals are plastic, allowing us to form new memories, adapt to new environments, and to learn new tasks. What is less clear is how much plasticity is required to perform these cognitive functions: does learning require widespread plasticity across the brain, or can learning occur with more rigid networks, in which plasticity is highly localized? Here, we use biologically-inspired recurrent neural network (RNN) models to show that rapid multitask learning can be accomplished with sparse and localized synaptic plasticity. Crucially, only RNNs with highly specific combinations of network properties, such as topology, normalization and reciprocal connection strength, are capable of such learning. Finally, we show that this rapid learning with localized plasticity can be accomplished with purely local error signals, without backpropagation, using a reinforcement learning setup. This work suggests that rapid learning in artificial (and potentially biological) agents can be accomplished with mostly-rigid networks, in which synaptic plasticity is highly constrained.

[1]  R. Buckner The Cerebellum and Cognitive Function: 25 Years of Insight from Anatomy and Neuroimaging , 2013, Neuron.

[2]  Zhangyang Wang,et al.  A Unified Lottery Ticket Hypothesis for Graph Neural Networks , 2021, ICML.

[3]  Peter Hagoort,et al.  Neuronal spike-rate adaptation supports working memory in language processing , 2020, Proceedings of the National Academy of Sciences.

[4]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[5]  Gintare Karolina Dziugaite,et al.  Stabilizing the Lottery Ticket Hypothesis , 2019 .

[6]  Benjamin F. Grewe,et al.  Minimizing Control for Credit Assignment with Strong Feedback , 2022, ICML.

[7]  Guangyu R. Yang,et al.  Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework , 2016, PLoS Comput. Biol..

[8]  Gilad Yehudai,et al.  Proving the Lottery Ticket Hypothesis: Pruning is All You Need , 2020, ICML.

[9]  Nicolas Y. Masse,et al.  Circuit mechanisms for the maintenance and manipulation of information in working memory , 2018 .

[10]  Michael Carbin,et al.  The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks , 2018, ICLR.

[11]  Maneesh Sahani,et al.  Organizing recurrent network dynamics by task-computation to enable continual learning , 2020, NeurIPS.

[12]  David J. Freedman,et al.  Experience-dependent representation of visual categories in parietal cortex , 2006, Nature.

[13]  Anthony M. Zador,et al.  A critique of pure learning and what artificial neural networks can learn from animal brains , 2019, Nature Communications.

[14]  Willie F. Tobin,et al.  Rapid formation and selective stabilization of synapses for enduring motor memories , 2009, Nature.

[15]  James C. R. Whittington,et al.  Theories of Error Back-Propagation in the Brain , 2019, Trends in Cognitive Sciences.

[16]  Shinichiro Tsutsumi,et al.  Optical interrogation of multi-scale neuronal plasticity underlying behavioral learning , 2020, Current Opinion in Neurobiology.

[17]  Yuandong Tian,et al.  Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP , 2019, ICLR.

[18]  Yang Zhang,et al.  The Lottery Ticket Hypothesis for Pre-trained BERT Networks , 2020, NeurIPS.

[19]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[20]  Qiang Zhou,et al.  NMDA receptor subunit diversity: impact on receptor properties, synaptic plasticity and disease , 2013, Nature Reviews Neuroscience.

[21]  Timothy P Lillicrap,et al.  Backpropagation through time and the brain , 2019, Current Opinion in Neurobiology.

[22]  Robert H. Cudmore,et al.  Cortical Synaptic AMPA Receptor Plasticity during Motor Learning , 2019, Neuron.

[23]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[24]  S. Nelson,et al.  Homeostatic plasticity in the developing nervous system , 2004, Nature Reviews Neuroscience.

[25]  Joel Z. Leibo,et al.  Prefrontal cortex as a meta-reinforcement learning system , 2018, bioRxiv.

[26]  Nicolas Y. Masse,et al.  Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization , 2018, Proceedings of the National Academy of Sciences.

[27]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[28]  William Mau,et al.  The role of intrinsic excitability in the evolution of memory: Significance in memory allocation, consolidation, and updating , 2020, Neurobiology of Learning and Memory.

[29]  Xiao-Jing Wang,et al.  Task representations in neural networks trained to perform many cognitive tasks , 2019, Nature Neuroscience.

[30]  Xiao-Jing Wang,et al.  Reward-based training of recurrent neural networks for cognitive and value-based tasks , 2016, bioRxiv.

[31]  B. Peterson,et al.  Normal Development of Brain Circuits , 2010, Neuropsychopharmacology.

[32]  A. Zador,et al.  Selective corticostriatal plasticity during acquisition of an auditory discrimination task , 2014, Nature.

[33]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[34]  Bozhkov Lachezar,et al.  Echo State Network , 2017, Encyclopedia of Machine Learning and Data Mining.

[35]  Wolfgang Maass,et al.  A solution to the learning dilemma for recurrent networks of spiking neurons , 2019, Nature Communications.

[36]  K. Svoboda,et al.  Experience-dependent structural synaptic plasticity in the mammalian brain , 2009, Nature Reviews Neuroscience.

[37]  Sadegh Nabavi,et al.  Engineering a memory with LTD and LTP , 2014, Nature.

[38]  C. Law,et al.  Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area , 2008, Nature Neuroscience.

[39]  Aditya Gilra,et al.  Thalamic regulation of switching between cortical representations enables cognitive flexibility , 2018, Nature Neuroscience.

[40]  Susumu Tonegawa,et al.  Memory engrams: Recalling the past and imagining the future , 2020, Science.

[41]  William A. Catterall,et al.  Calcium Channels, Synaptic Plasticity, and Neuropsychiatric Disease , 2018, Neuron.

[42]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[43]  Surya Ganguli,et al.  Continual Learning Through Synaptic Intelligence , 2017, ICML.

[44]  Richard Naud,et al.  Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits , 2020, Nature Neuroscience.

[45]  Claudia Clopath,et al.  Dimensionality and flexibility of learning in biological recurrent neural networks , 2020 .

[46]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[47]  Yann Humeau,et al.  The next generation of approaches to investigate the link between synaptic plasticity and learning , 2019, Nature Neuroscience.

[48]  David J Heeger,et al.  A recurrent circuit implements normalization, simulating the dynamics of V1 activity , 2020, Proceedings of the National Academy of Sciences.

[49]  Michael M. Halassa,et al.  Thalamocortical contribution to solving credit assignment in neural systems , 2021, 2104.01474.

[50]  J. Vogelstein,et al.  Visualizing synaptic plasticity in vivo by large-scale imaging of endogenous AMPA receptors , 2020, bioRxiv.

[51]  Sergey Levine,et al.  High-Dimensional Continuous Control Using Generalized Advantage Estimation , 2015, ICLR.

[52]  Timothy P Lillicrap,et al.  Towards deep learning with segregated dendrites , 2016, eLife.

[53]  Wolfgang Maass,et al.  CCN GAC Workshop: Issues with learning in biological recurrent neural networks , 2021, ArXiv.

[54]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[55]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[56]  S. Kastner,et al.  Topographic maps in human frontal and parietal cortex , 2009, Trends in Cognitive Sciences.

[57]  T. Hensch Critical period plasticity in local cortical circuits , 2005, Nature Reviews Neuroscience.

[58]  Joseph A. Gogos,et al.  Synaptic plasticity, neural circuits, and the emerging role of altered short-term information processing in schizophrenia , 2014, Front. Synaptic Neurosci..

[59]  Adam Santoro,et al.  Backpropagation and the brain , 2020, Nature Reviews Neuroscience.

[60]  R. Desimone,et al.  Neural Mechanisms of Visual Working Memory in Prefrontal Cortex of the Macaque , 1996, The Journal of Neuroscience.

[61]  David J. Freedman,et al.  Interaction between neuronal encoding and population dynamics during categorization task switching in parietal cortex , 2020, Neuron.