Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes

Les humains apprennent toute leur vie. Ils accumulent des connaissances a partir d'une succession d'experiences d'apprentissage et en memorisent les aspects essentiels sans les oublier. Les reseaux de neurones artificiels ont des difficultes a apprendre dans de telles conditions. Ils ont en general besoin d'ensembles de donnees rigoureusement prepares pour pouvoir apprendre a resoudre des problemes comme de la classification ou de la regression. En particulier, lorsqu'ils apprennent sur des sequences d'ensembles de donnees, les nouvelles experiences leurs font oublier les anciennes. Ainsi, ils sont souvent incapables d'apprehender des scenarios reels tels ceux de robots autonomes apprenant en temps reel a s'adapter a de nouvelles situations et devant resoudre des problemes sans oublier leurs experiences passees.L'apprentissage continu est une branche de l'apprentissage automatique s'attaquant a ce type de scenarios. Les algorithmes continus sont crees pour apprendre des connaissances, les enrichir et les ameliorer au cours d'un curriculum d'experiences d'apprentissage.Dans cette these, nous proposons d'explorer l'apprentissage continu avec rejeu de donnees. Les methodes de rejeu de donnees rassemblent les methodes de repetitions et les methodes de rejeu par generation. Le rejeu par generation consiste a utiliser un reseau de neurones auxiliaire apprenant a generer les donnees actuelles. Ainsi plus tard le reseau auxiliaire pourra etre utilise pour regenerer des donnees du passe et les rememorer au modele principal. La repetition a le meme objectif, mais cette methode sauve simplement des images specifiques et les rejoue plus tard au modele principal pour eviter qu'il ne les oublie. Les methodes de rejeu permettent de trouver un compromis entre l'optimisation de l'objectif d'apprentissage actuel et ceux du passe. Elles permettent ainsi d'apprendre sans oublier sur des sequences de tâches.Nous montrons que ces methodes sont prometteuses pour l'apprentissage continu.En particulier, elles permettent la reevaluation des donnees du passe avec des nouvelles connaissances et de confronter des donnees issues de differentes experiences. Nous demontrons la capacite des methodes de rejeu a apprendre continuellement a travers des tâches d'apprentissage non-supervisees, supervisees et de renforcements.

[1]  Rama Chellappa,et al.  Visual Domain Adaptation: A survey of recent advances , 2015, IEEE Signal Processing Magazine.

[2]  Jakub W. Pachocki,et al.  Dota 2 with Large Scale Deep Reinforcement Learning , 2019, ArXiv.

[3]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[4]  Giulio Sandini,et al.  Developmental robotics: a survey , 2003, Connect. Sci..

[5]  Matthew B. Blaschko,et al.  Encoder Based Lifelong Learning , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Benedikt Pfülb,et al.  A comprehensive, application-oriented study of catastrophic forgetting in DNNs , 2019, ICLR.

[7]  Jay M. Wong,et al.  Towards Lifelong Self-Supervision: A Deep Learning Direction for Robotics , 2016, ArXiv.

[8]  Long Ji Lin,et al.  Self-improving reactive agents based on reinforcement learning, planning and teaching , 1992, Machine Learning.

[9]  Siddhartha S. Srinivasa,et al.  HerbDisc: Towards lifelong robotic object discovery , 2015, Int. J. Robotics Res..

[10]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  David Barber,et al.  Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting , 2018, NeurIPS.

[13]  Byoung-Tak Zhang,et al.  Overcoming Catastrophic Forgetting by Incremental Moment Matching , 2017, NIPS.

[14]  Cordelia Schmid,et al.  How good is my GAN? , 2018, ECCV.

[15]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[16]  Jeff Clune,et al.  Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks , 2017, PloS one.

[17]  Marvin Minsky,et al.  Steps toward Artificial Intelligence , 1995, Proceedings of the IRE.

[18]  Sung Ju Hwang,et al.  Lifelong Learning with Dynamically Expandable Networks , 2017, ICLR.

[19]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[20]  Xu Jia,et al.  Continual learning: A comparative study on how to defy forgetting in classification tasks , 2019, ArXiv.

[21]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[22]  David Filliat,et al.  From passive to interactive object learning and recognition through self-identification on a humanoid robot , 2016, Auton. Robots.

[23]  Razvan Pascanu,et al.  Theano: new features and speed improvements , 2012, ArXiv.

[24]  Freek Stulp,et al.  Simultaneous on-line Discovery and Improvement of Robotic Skill options , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Giorgio Metta,et al.  Incremental semiparametric inverse dynamics learning , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Stefan Wermter,et al.  Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization , 2018, Front. Neurorobot..

[27]  Victor Talpaert,et al.  Exploring applications of deep reinforcement learning for real-world autonomous driving systems , 2019, VISIGRAPP.

[28]  Marcin Andrychowicz,et al.  Hindsight Experience Replay , 2017, NIPS.

[29]  Laurent Itti,et al.  Active Long Term Memory Networks , 2016, ArXiv.

[30]  Andrew Zisserman,et al.  Multi-task Self-Supervised Visual Learning , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Pierre-Yves Oudeyer,et al.  GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms , 2017, ICML.

[32]  David Filliat,et al.  S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replay , 2019, ArXiv.

[33]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[34]  Zachary Chase Lipton,et al.  Born Again Neural Networks , 2018, ICML.

[35]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[36]  David Filliat,et al.  DisCoRL: Continual Reinforcement Learning via Policy Distillation , 2019, ArXiv.

[37]  Yarin Gal,et al.  Towards Robust Evaluations of Continual Learning , 2018, ArXiv.

[38]  Alessandro Chiuso,et al.  Derivative-Free Online Learning of Inverse Dynamics Models , 2018, IEEE Transactions on Control Systems Technology.

[39]  Trevor Darrell,et al.  Loss is its own Reward: Self-Supervision for Reinforcement Learning , 2016, ICLR.

[40]  David Filliat,et al.  Marginal Replay vs Conditional Replay for Continual Learning , 2018, ICANN.

[41]  Alexandre Zénon,et al.  Learning and forgetting using reinforced Bayesian change detection , 2018, bioRxiv.

[42]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[43]  Ricardo Vilalta,et al.  Metalearning - Applications to Data Mining , 2008, Cognitive Technologies.

[44]  Silvio Savarese,et al.  ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation , 2018, CoRL.

[45]  Lorenzo Rosasco,et al.  Object identification from few examples by improving the invariance of a Deep Convolutional Neural Network , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[46]  Yoshua Bengio,et al.  Gradient based sample selection for online continual learning , 2019, NeurIPS.

[47]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[48]  Adrian Popescu,et al.  DeeSIL: Deep-Shallow Incremental Learning , 2018, ECCV Workshops.

[49]  Jan Peters,et al.  Stable reinforcement learning with autoencoders for tactile and visual data , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[50]  Stefan Wermter,et al.  Continual Lifelong Learning with Neural Networks: A Review , 2019, Neural Networks.

[51]  Han Liu,et al.  Continual Learning in Generative Adversarial Nets , 2017, ArXiv.

[52]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[53]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[54]  Lorenzo Rosasco,et al.  Teaching iCub to recognize objects using deep Convolutional Neural Networks , 2015, MLIS@ICML.

[55]  Sebastian Thrun,et al.  Lifelong robot learning , 1993, Robotics Auton. Syst..

[56]  Michael McCloskey,et al.  Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , 1989 .

[57]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[58]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[59]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Cordelia Schmid,et al.  End-to-End Incremental Learning , 2018, ECCV.

[61]  Tinne Tuytelaars,et al.  Expert Gate: Lifelong Learning with a Network of Experts , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Davide Maltoni,et al.  Continuous Learning in Single-Incremental-Task Scenarios , 2018, Neural Networks.

[63]  Silvio Savarese,et al.  SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark , 2018, CoRL.

[64]  Razvan Pascanu,et al.  Policy Distillation , 2015, ICLR.

[65]  Victor Talpaert,et al.  Deep Reinforcement Learning for Autonomous Driving: A Survey , 2020, IEEE Transactions on Intelligent Transportation Systems.

[66]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[67]  Anthony V. Robins,et al.  Catastrophic Forgetting, Rehearsal and Pseudorehearsal , 1995, Connect. Sci..

[68]  Y. Dudai,et al.  Science of Memory , 2007 .

[69]  Estevam R. Hruschka,et al.  Toward an Architecture for Never-Ending Language Learning , 2010, AAAI.

[70]  Tinne Tuytelaars,et al.  Online Continual Learning with Maximally Interfered Retrieval , 2019, ArXiv.

[71]  Rama Chellappa,et al.  Learning Without Memorizing , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[72]  Barbara Hammer,et al.  Incremental learning algorithms and applications , 2016, ESANN.

[73]  Pierre-Yves Oudeyer,et al.  Computational Theories of Curiosity-Driven Learning , 2018, ArXiv.

[74]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[75]  Michael Gasser,et al.  The Development of Embodied Cognition: Six Lessons from Babies , 2005, Artificial Life.

[76]  Razvan Pascanu,et al.  Sim-to-Real Robot Learning from Pixels with Progressive Nets , 2016, CoRL.

[77]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[78]  Herbert Jaeger,et al.  Using Conceptors to Manage Neural Long-Term Memories for Temporal Patterns , 2017, J. Mach. Learn. Res..

[79]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[80]  Surya Ganguli,et al.  Continual Learning Through Synaptic Intelligence , 2017, ICML.

[81]  Wolfram Burgard,et al.  The limits and potentials of deep learning for robotics , 2018, Int. J. Robotics Res..

[82]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[83]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[84]  Davide Maltoni,et al.  Comparing Incremental Learning Strategies for Convolutional Neural Networks , 2016, ANNPR.

[85]  Ali Borji,et al.  Pros and Cons of GAN Evaluation Measures , 2018, Comput. Vis. Image Underst..

[86]  A. Gopnik,et al.  The scientist in the crib : minds, brains, and how children learn , 1999 .

[87]  Martial Mermillod,et al.  The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects , 2013, Front. Psychol..

[88]  Martin A. Riedmiller,et al.  Learn to Swing Up and Balance a Real Pole Based on Raw Visual Input Data , 2012, ICONIP.

[89]  Richard E. Turner,et al.  Variational Continual Learning , 2017, ICLR.

[90]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[91]  Yandong Guo,et al.  Large Scale Incremental Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[92]  Yoshua Bengio,et al.  An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks , 2013, ICLR.

[93]  Min Lin,et al.  Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning , 2020, ArXiv.

[94]  Nathan D. Cahill,et al.  Memory Efficient Experience Replay for Streaming Learning , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[95]  David Berthelot,et al.  BEGAN: Boundary Equilibrium Generative Adversarial Networks , 2017, ArXiv.

[96]  David Filliat,et al.  State Representation Learning for Control: An Overview , 2018, Neural Networks.

[97]  Roger B. Grosse,et al.  Optimizing Neural Networks with Kronecker-factored Approximate Curvature , 2015, ICML.

[98]  Abhinav Gupta,et al.  Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[99]  David Filliat,et al.  Exploration strategies for incremental learning of object-based visual saliency , 2015, 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[100]  Mark B. Ring.  Continual learning in reinforcement environments , 1995, GMD-Bericht.

[101]  Razvan Pascanu,et al.  Progressive Neural Networks , 2016, ArXiv.

[102]  Abraham Prieto,et al.  Embodied Evolution in Collective Robotics: A Review , 2017, Front. Robot. AI.

[103]  Yi Sun,et al.  Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments , 2011, AGI.

[104]  Lorien Y. Pratt,et al.  Discriminability-Based Transfer between Neural Networks , 1992, NIPS.

[105]  Honglak Lee,et al.  Online Incremental Feature Learning with Denoising Autoencoders , 2012, AISTATS.

[106]  Laurent Itti,et al.  Closed-Loop Memory GAN for Continual Learning , 2018, IJCAI.

[107]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[108]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[109]  Nathan D. Cahill,et al.  New Metrics and Experimental Paradigms for Continual Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[110]  Pierre-Yves Oudeyer,et al.  Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning , 2017, J. Mach. Learn. Res..

[111]  Denis Pellerin,et al.  Classifier Training from a Generative Model , 2019, 2019 International Conference on Content-Based Multimedia Indexing (CBMI).

[112]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[113]  Wei Yang,et al.  Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[114]  Xu He,et al.  Overcoming Catastrophic Interference using Conceptor-Aided Backpropagation , 2018, ICLR.

[115]  Cordelia Schmid,et al.  Incremental Learning of Object Detectors without Catastrophic Forgetting , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[116]  Junmo Kim,et al.  Less-forgetting Learning in Deep Neural Networks , 2016, ArXiv.

[117]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[118]  David Filliat,et al.  Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges , 2020, Inf. Fusion.

[119]  Pietro Perona,et al.  Caltech-UCSD Birds 200 , 2010 .

[120]  Dahua Lin,et al.  Learning a Unified Classifier Incrementally via Rebalancing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[121]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[122]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[123]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[124]  David Filliat,et al.  Don't forget, there is more than forgetting: new metrics for Continual Learning , 2018, ArXiv.

[125]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[126]  David Filliat,et al.  S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning , 2018, ArXiv.

[127]  Stéphane Doncieux,et al.  From exploration to control: learning object manipulation skills through novelty search and local adaptation , 2019, Robotics Auton. Syst..

[128]  Aaron Klein,et al.  Efficient and Robust Automated Machine Learning , 2015, NIPS.

[129]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[130]  David Filliat,et al.  Training Discriminative Models to Evaluate Generative Ones , 2019, ICANN.

[131]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[132]  Sergey Levine,et al.  Deep Imitative Models for Flexible Inference, Planning, and Control , 2018, ICLR.

[133]  Honglak Lee,et al.  Learning Structured Output Representation using Deep Conditional Generative Models , 2015, NIPS.

[134]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[135]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[136]  Adrian Popescu,et al.  ScaIL: Classifier Weights Scaling for Class Incremental Learning , 2020, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[137]  Joel Lehman,et al.  Learning to Continually Learn , 2020, ECAI.

[138]  Chrisantha Fernando,et al.  PathNet: Evolution Channels Gradient Descent in Super Neural Networks , 2017, ArXiv.

[139]  Yoshua Bengio,et al.  On Training Recurrent Neural Networks for Lifelong Learning , 2018, ArXiv.

[140]  Abhinav Gupta,et al.  Learning to fly by crashing , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[141]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[142]  Olivier Sigaud,et al.  Tensor Based Knowledge Transfer Across Skill Categories for Robot Control , 2017, IJCAI.

[143]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[144]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[145]  Jürgen Schmidhuber,et al.  Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) , 2010, IEEE Transactions on Autonomous Mental Development.

[146]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[147]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents , 2012, J. Artif. Intell. Res..

[148]  Alexander Gepperth,et al.  A Bio-Inspired Incremental Learning Architecture for Applied Perceptual Problems , 2016, Cognitive Computation.

[149]  Yee Whye Teh,et al.  Distral: Robust multitask reinforcement learning , 2017, NIPS.

[150]  David Filliat,et al.  Generative Models from the perspective of Continual Learning , 2018, 2019 International Joint Conference on Neural Networks (IJCNN).

[151]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[152]  Conrad D. James,et al.  Neurogenesis deep learning: Extending deep networks to accommodate new classes , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[153]  David Filliat,et al.  Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics , 2018, ArXiv.

[154]  Luigi di Stefano,et al.  On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[155]  Gabriela Csurka,et al.  Domain Adaptation for Visual Applications: A Comprehensive Survey , 2017, ArXiv.

[156]  Rahaf Aljundi,et al.  Continual Learning in Neural Networks , 2019, ArXiv.

[157]  Richard J. Duro,et al.  Using promoters and functional introns in genetic algorithms for neuroevolutionary learning in non-stationary problems , 2009, Neurocomputing.

[158]  Sergey Levine,et al.  Deep spatial autoencoders for visuomotor learning , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[159]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[160]  Yee Whye Teh,et al.  Progress & Compress: A scalable framework for continual learning , 2018, ICML.

[161]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[162]  Alex Lamb,et al.  Deep Learning for Classical Japanese Literature , 2018, ArXiv.

[163]  William Q. Meeker,et al.  A BAYESIAN ON-LINE CHANGE DETECTION ALGORITHM WITH PROCESS MONITORING APPLICATIONS , 1998 .

[164]  Yoshua Bengio,et al.  Toward Training Recurrent Neural Networks for Lifelong Learning , 2018, Neural Computation.

[165]  Andrés Faiña,et al.  A cognitive developmental robotics architecture for lifelong learning by evolution in real robots , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[166]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[167]  A. Cangelosi,et al.  From Babies to Robots: The Contribution of Developmental Robotics to Developmental Psychology , 2018 .

[168]  Jiwon Kim,et al.  Continual Learning with Deep Generative Replay , 2017, NIPS.

[169]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[170]  Tom Eccles,et al.  Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies , 2018, NeurIPS.

[171]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[172]  Pietro Zanuttigh,et al.  Knowledge Distillation for Incremental Learning in Semantic Segmentation , 2019, Comput. Vis. Image Underst..

[173]  Faisal Shafait,et al.  Distillation Techniques for Pseudo-rehearsal Based Incremental Learning , 2018, ArXiv.

[174]  Alexandros Karatzoglou,et al.  Overcoming Catastrophic Forgetting with Hard Attention to the Task , 2018 .

[175]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[176]  Alexei A. Efros,et al.  Dataset Distillation , 2018, ArXiv.

[177]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[178]  Derek Hoiem,et al.  Learning without Forgetting , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[179]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[180]  Christoph H. Lampert,et al.  Lifelong Learning with Non-i.i.d. Tasks , 2015, NIPS.

[181]  Oriol Vinyals,et al.  Neural Discrete Representation Learning , 2017, NIPS.

[182]  Martial Hebert,et al.  Growing a Brain: Fine-Tuning by Increasing Model Capacity , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[183]  Aleksander Madry,et al.  A Classification-Based Study of Covariate Shift in GAN Distributions , 2017, ICML.

[184]  Joshua B. Tenenbaum,et al.  One shot learning of simple visual concepts , 2011, CogSci.

[185]  Alexei A. Efros,et al.  Large-Scale Study of Curiosity-Driven Learning , 2018, ICLR.

[186]  Andrew J. Davison,et al.  Sim-to-Real Reinforcement Learning for Deformable Object Manipulation , 2018, CoRL.

[187]  Giorgio Metta,et al.  Incremental robot learning of new objects with fixed update time , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[188]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[189]  Vincenzo Lomonaco,et al.  Continual Learning with Deep Architectures , 2019 .

[190]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[191]  Suman V. Ravuri,et al.  Classification Accuracy Score for Conditional Generative Models , 2019, NeurIPS.

[192]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[193]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[194]  Svetlana Lazebnik,et al.  Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights , 2018, ECCV.

[195]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[196]  Alexey Dosovitskiy,et al.  End-to-End Driving Via Conditional Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[197]  Yinda Zhang,et al.  LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop , 2015, ArXiv.

[198]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[199]  Tom Schaul,et al.  Prioritized Experience Replay , 2015, ICLR.

[200]  Philip H. S. Torr,et al.  Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence , 2018, ECCV.

[201]  Pierre-Yves Oudeyer,et al.  Curiosity Driven Exploration of Learned Disentangled Goal Spaces , 2018, CoRL.

[202]  Richard J. Duro,et al.  Open-Ended Learning: A Conceptual Framework Based on Representational Redescription , 2018, Front. Neurorobot..

[203]  Wojciech Zaremba,et al.  Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model , 2016, ArXiv.

[204]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[205]  Xinlei Chen,et al.  Never-Ending Learning , 2012, ECAI.

[206]  Peng Cui,et al.  NICO: A Dataset Towards Non-I.I.D. Image Classification , 2019, ArXiv.

[207]  Chunyan Miao,et al.  A Survey of Zero-Shot Learning , 2019, ACM Trans. Intell. Syst. Technol..

[208]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[209]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[210]  Sergey Levine,et al.  QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation , 2018, CoRL.

[211]  Pierre-Yves Oudeyer,et al.  CURIOUS: Intrinsically Motivated Multi-Task, Multi-Goal Reinforcement Learning , 2018, ICML 2019.

[212]  Ronald Kemker,et al.  FearNet: Brain-Inspired Model for Incremental Learning , 2017, ICLR.

[213]  Oliver Brock,et al.  Learning state representations with robotic priors , 2015, Auton. Robots.

[214]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[215]  Yan Liu,et al.  Deep Generative Dual Memory Network for Continual Learning , 2017, ArXiv.

[216]  Marc'Aurelio Ranzato,et al.  Gradient Episodic Memory for Continual Learning , 2017, NIPS.

[217]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[218]  Eugenio Culurciello,et al.  Continual Reinforcement Learning in 3D Non-stationary Environments , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[219]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[220]  Davide Maltoni,et al.  CORe50: a New Dataset and Benchmark for Continuous Object Recognition , 2017, CoRL.

[221]  Martin A. Riedmiller,et al.  Learning by Playing - Solving Sparse Reward Tasks from Scratch , 2018, ICML.

[222]  Manuel Lopes,et al.  A multimodal dataset for object model learning from natural human-robot interaction , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[223]  Alessandro Chiuso,et al.  Online semi-parametric learning for inverse dynamics modeling , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[224]  David Filliat,et al.  Exploring to learn visual saliency: The RL-IAC approach , 2018, Robotics Auton. Syst..

[225]  Joelle Pineau,et al.  Online Learned Continual Compression with Adaptive Quantization Modules , 2019, ICML.

[226]  Gerald Tesauro,et al.  Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference , 2018, ICLR.

[227]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[228]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[229]  Yadin Dudai,et al.  Science of memory: concepts , 2007 .

[230]  Jitendra Malik,et al.  Learning to Poke by Poking: Experiential Learning of Intuitive Physics , 2016, NIPS.

[231]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[232]  Marc'Aurelio Ranzato,et al.  Efficient Lifelong Learning with A-GEM , 2018, ICLR.

[233]  Davide Maltoni,et al.  Latent Replay for Real-Time Continual Learning , 2019, ArXiv.

[234]  Sebastian Nowozin,et al.  f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization , 2016, NIPS.

[235]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[236]  James L. McClelland,et al.  Autonomous Mental Development by Robots and Animals , 2001, Science.

[237]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[238]  Razvan Pascanu,et al.  Memory-based Parameter Adaptation , 2018, ICLR.

[239]  Christoph H. Lampert,et al.  iCaRL: Incremental Classifier and Representation Learning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[240]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[241]  Svetlana Lazebnik,et al.  PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[242]  David Filliat,et al.  Deep unsupervised state representation learning with robotic priors: a robustness analysis , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[243]  Yandong Guo,et al.  Incremental Classifier Learning with Generative Adversarial Networks , 2018, ArXiv.

[244]  David Filliat,et al.  Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer , 2019, ArXiv.

[245]  Mark B. Ring.  Toward a Formal Framework for Continual Learning , 2005 .

[246]  Bing Liu,et al.  Lifelong machine learning: a paradigm for continuous learning , 2017, Frontiers of Computer Science.

[247]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[248]  Sara van de Geer,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2011 .

[249]  Andrés Faiña,et al.  Multilevel Darwinist Brain (MDB): Artificial Evolution in a Cognitive Architecture for Real Robots , 2010, IEEE Transactions on Autonomous Mental Development.

[250]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[251]  Ronald Kemker,et al.  Measuring Catastrophic Forgetting in Neural Networks , 2017, AAAI.

[252]  He Ma,et al.  Quantitatively Evaluating GANs With Divergences Proposed for Training , 2018, ICLR.

[253]  Alexandros Kalousis,et al.  Lifelong Generative Modeling , 2017, Neurocomputing.

[254]  Joachim Denzler,et al.  Fine-Tuning Deep Neural Networks in Continuous Learning Scenarios , 2016, ACCV Workshops.

[255]  Bogdan Raducanu,et al.  Memory Replay GANs: Learning to Generate New Categories without Forgetting , 2018, NeurIPS.

[256]  Fawzi Nashashibi,et al.  End-to-End Race Driving with Deep Reinforcement Learning , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[257]  Donald Geman,et al.  Visual Turing test for computer vision systems , 2015, Proceedings of the National Academy of Sciences.

[258]  Demis Hassabis,et al.  A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play , 2018, Science.

[259]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[260]  Martha White,et al.  Meta-Learning Representations for Continual Learning , 2019, NeurIPS.

[261]  David Filliat,et al.  Regularization Shortcomings for Continual Learning , 2019, ArXiv.

[262]  Harm de Vries,et al.  RMSProp and equilibrated adaptive learning rates for non-convex optimization. , 2015 .

[263]  R. French.  Catastrophic forgetting in connectionist networks , 1999, Trends in Cognitive Sciences.

[264]  Tom Schaul,et al.  Unicorn: Continual Learning with a Universal, Off-policy Agent , 2018, ArXiv.