Continual Learning with Deep Generative Replay

Attempts to train a comprehensive artificial intelligence capable of solving multiple tasks have been impeded by a chronic problem called catastrophic forgetting. Although simply replaying all previous data alleviates the problem, it requires large memory and even worse, often infeasible in real world applications where the access to past data is limited. Inspired by the generative nature of hippocampus as a short-term memory system in primate brain, we propose the Deep Generative Replay, a novel framework with a cooperative dual model architecture consisting of a deep generative model ("generator") and a task solving model ("solver"). With only these two models, training data for previous tasks can easily be sampled and interleaved with those for a new task. We test our methods in several sequential learning settings involving image classification tasks.

[1]  Geoffrey E. Hinton.  Using fast weights to deblur old memories , 1987 .

[2]  Michael McCloskey,et al.  Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , 1989 .

[3]  R Ratcliff,et al.  Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. , 1990, Psychological review.

[4]  Anthony V. Robins,et al.  Catastrophic forgetting in neural networks: the role of rehearsal mechanisms , 1993, Proceedings 1993 The First New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems.

[5]  Dare A. Baldwin,et al.  Infants' ability to draw inferences about nonobvious object properties: evidence from exploratory play. , 1993, Child development.

[6]  Anthony V. Robins,et al.  Catastrophic Forgetting, Rehearsal and Pseudorehearsal , 1995, Connect. Sci..

[7]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[8]  R. French.  Catastrophic forgetting in connectionist networks , 1999, Trends in Cognitive Sciences.

[9]  Thomas G. Dietterich.  Ensemble Methods in Machine Learning , 2000, Multiple Classifier Systems.

[10]  R. O’Reilly,et al.  Opinion TRENDS in Cognitive Sciences Vol.6 No.12 December 2002 , 2022 .

[11]  N. Kasthuri,et al.  Long-term dendritic spine stability in the adult cortex , 2002, Nature.

[12]  L’oubli catastrophique it,et al.  Avoiding catastrophic forgetting by coupling two reverberating neural networks , 2004 .

[13]  W. Abraham,et al.  Memory retention – the synaptic stability versus plasticity dilemma , 2005, Trends in Neurosciences.

[14]  J. Fagot,et al.  Evidence for large long-term memory capacities in baboons and pigeons and its implications for learning and the evolution of cognition , 2006, Proceedings of the National Academy of Sciences.

[15]  R. Stickgold,et al.  Sleep-dependent memory consolidation and reconsolidation. , 2007, Sleep medicine.

[16]  I. Fried,et al.  Internally Generated Reactivation of Single Neurons in Human Hippocampus During Free Recall , 2008, Science.

[17]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[18]  M. Arterberry,et al.  The development of object categorization in young children: hierarchical inclusiveness, age, perceptual attribute, and group versus individual analyses. , 2010, Developmental psychology.

[19]  J. O’Neill,et al.  Play it again: reactivation of waking experience and memory , 2010, Trends in Neurosciences.

[20]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[21]  S. Tonegawa,et al.  Creating a False Memory in the Hippocampus , 2013, Science.

[22]  Jürgen Schmidhuber,et al.  Compete to Compute , 2013, NIPS.

[23]  Yoshua Bengio,et al.  An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks , 2013, ICLR.

[24]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[26]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[27]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[28]  Motonobu Hattori,et al.  A biologically inspired dual-network memory model for reduction of catastrophic forgetting , 2014, Neurocomputing.

[29]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[30]  Eric Eaton,et al.  Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data , 2016, ArXiv.

[31]  Byoung-Tak Zhang,et al.  Overcoming Catastrophic Forgetting by Incremental Moment Matching , 2017, NIPS.

[32]  Ian J. Goodfellow,et al.  NIPS 2016 Tutorial: Generative Adversarial Networks , 2016, ArXiv.

[33]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[34]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[35]  Derek Hoiem,et al.  Learning without Forgetting , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.