CHILD: A First Step Towards Continual Learning

Continual learning is the constant development of increasingly complex behaviors; the process of building more complicated skills on top of those already developed. A continual-learning agent should therefore learn incrementally and hierarchically. This paper describes CHILD, an agent capable of Continual, Hierarchical, Incremental Learning and Development. CHILD can quickly solve complicated non-Markovian reinforcement-learning tasks and can then transfer its skills to similar but even more complicated tasks, learning these faster still.

[1]  J. Albus.  Mechanisms of planning and problem solving in the brain , 1979 .

[2]  Avron Barr,et al.  The Handbook of Artificial Intelligence, Volume 1 , 1982 .

[3]  Barr and Feigenbaum Edward A. Avron,et al.  The Handbook of Artificial Intelligence , 1981 .

[4]  C. Roads,et al.  The Handbook of Artificial Intelligence, Volume 1 , 1982 .

[5]  Stewart W. Wilson.  Hierarchical Credit Allocation in a Classifier System , 1987, IJCAI.

[6]  C. Watkins.  Learning from delayed rewards , 1989 .

[7]  Scott E. Fahlman,et al.  The Recurrent Cascade-Correlation Architecture , 1990, NIPS.

[8]  Jürgen Schmidhuber,et al.  Learning Unambiguous Reduced Sequence Descriptions , 1991, NIPS.

[9]  Lambert E. Wixson,et al.  Scaling Reinforcement Learning Techniques via Modularity , 1991, ML.

[10]  H. L. Roitblat,et al.  Cognitive action theory as a control architecture , 1991 .

[11]  Terence D. Sanger,et al.  A tree-structured adaptive network for function approximation in high-dimensional spaces , 1991, IEEE Trans. Neural Networks.

[12]  Satinder Singh.  Transfer of learning by composing solutions of elemental sequential tasks , 2004, Machine Learning.

[13]  Geoffrey E. Hinton,et al.  Feudal Reinforcement Learning , 1992, NIPS.

[14]  Long-Ji Lin,et al.  Reinforcement learning for robots using neural networks , 1992 .

[15]  Lonnie Chrisman,et al.  Reinforcement Learning with Perceptual Aliasing: The Perceptual Distinctions Approach , 1992, AAAI.

[16]  Lorien Y. Pratt,et al.  Discriminability-Based Transfer between Neural Networks , 1992, NIPS.

[17]  Mark B. Ring.  Learning Sequential Tasks by Incrementally Adding Higher Orders , 1992, NIPS.

[18]  Leslie Pack Kaelbling,et al.  Learning to Achieve Goals , 1993, IJCAI.

[19]  Jürgen Schmidhuber,et al.  Planning simple trajectories using neural subgoal generators , 1993 .

[20]  Rich Caruana,et al.  Multitask Learning: A Knowledge-Based Source of Inductive Bias , 1993, ICML.

[21]  Andrew McCallum,et al.  Overcoming Incomplete Perception with Utile Distinction Memory , 1993, ICML.

[22]  Marco C. Bettoni,et al.  Made-Up Minds: A Constructivist Approach to Artificial Intelligence , 1993, IEEE Expert.

[23]  J. Elman.  Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[24]  Leslie Pack Kaelbling,et al.  Hierarchical Learning in Stochastic Domains: Preliminary Results , 1993, ICML.

[25]  C. L. Giles,et al.  Constructive learning of recurrent neural networks , 1993, IEEE International Conference on Neural Networks.

[26]  Juergen Schmidhuber,et al.  On learning how to learn learning strategies , 1994 .

[27]  Sebastian Thrun,et al.  Finding Structure in Reinforcement Learning , 1994, NIPS.

[28]  Mark B. Ring.  Continual learning in reinforcement environments , 1995, GMD-Bericht.

[29]  Sebastian Thrun,et al.  Is Learning The n-th Thing Any Easier Than Learning The First? , 1995, NIPS.

[30]  C. Lee Giles,et al.  Constructive learning of recurrent neural networks: limitations of recurrent cascade correlation and a simple solution , 1995, IEEE Trans. Neural Networks.

[31]  Jonathan Baxter,et al.  Learning Model Bias , 1995, NIPS.

[32]  Andrew McCallum,et al.  Learning to Use Selective Attention and Short-Term Memory in Sequential Tasks , 1996 .

[33]  Maja J. Matarić,et al.  Learning to Use Selective Attention and Short-Term Memory in Sequential Tasks , 1996 .

[34]  Michael I. Jordan.  Serial Order: A Parallel Distributed Processing Approach , 1997 .

[35]  Allen Newell,et al.  Chunking in Soar: The anatomy of a general learning mechanism , 1985, Machine Learning.

[36]  Long Ji Lin,et al.  Self-improving reactive agents based on reinforcement learning, planning and teaching , 1992, Machine Learning.

[37]  Noel E. Sharkey,et al.  Adaptive generalisation , 1994, Artificial Intelligence Review.

[38]  Mike Wynne-Jones,et al.  Node splitting: A constructive algorithm for feed-forward neural networks , 1991, Neural Computing & Applications.

[39]  J. Pollack.  The Induction of Dynamical Recognizers , 1996, Machine Learning.