Lifelong Learning Algorithms

Machine learning has not yet succeeded in the design of robust learning algorithms that generalize well from very small datasets. In contrast, humans often generalize correctly from only a single training example, even if the number of potentially relevant features is large. To do so, they successfully exploit knowledge acquired in previous learning tasks, to bias subsequent learning.

[1]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[2]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[3]  Gerald DeJong.  Investigating Explanation-Based Learning , 1992 .

[4]  Sebastian Thrun,et al.  Is Learning The n-th Thing Any Easier Than Learning The First? , 1995, NIPS.

[5]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[6]  Michael J. Pazzani,et al.  A Knowledge-intensive Approach to Learning Relational Concepts , 1991, ML.

[7]  David L. Waltz,et al.  Toward memory-based reasoning , 1986, CACM.

[8]  R. Franke.  Scattered data interpolation: tests of some methods , 1982 .

[9]  Larry A. Rendell,et al.  Layered Concept-Learning and Dynamically Variable Bias Management , 1987, IJCAI.

[10]  Michael I. Jordan,et al.  Hierarchies of Adaptive Experts , 1991, NIPS.

[11]  Jonathan Baxter,et al.  Learning internal representations , 1995, COLT '95.

[12]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[13]  L.-M. Fu,et al.  Integration of neural heuristics into knowledge-based inference , 1989, International 1989 Joint Conference on Neural Networks.

[14]  Tomaso Poggio,et al.  Example Based Image Analysis and Synthesis , 1993 .

[15]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[16]  D. Shepard.  A two-dimensional interpolation function for irregularly-spaced data , 1968, ACM National Conference.

[17]  Tom Michael Mitchell.  Version spaces: an approach to concept learning. , 1979 .

[18]  Sebastian Thrun,et al.  Learning One More Thing , 1994, IJCAI.

[19]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[20]  J. Rennie.  Cancer catcher. Neural net catches errors that slip through Pap tests. , 1990, Scientific American.

[21]  Andrew W. Moore,et al.  Efficient memory-based learning for robot control , 1990 .

[22]  Yann LeCun,et al.  Tangent Prop - A Formalism for Specifying Selected Invariances in an Adaptive Network , 1991, NIPS.

[23]  Steven C. Suddarth,et al.  Symbolic-Neural Systems and the Use of Hints for Developing Complex Systems , 1991, Int. J. Man Mach. Stud..

[24]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[25]  Rich Caruana,et al.  Greedy Attribute Selection , 1994, ICML.

[26]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.

[27]  Andrew W. Moore,et al.  An Introduction to Reinforcement Learning , 1995 .

[28]  W. Ahn,et al.  Psychological Studies of Explanation—Based Learning , 1993 .

[29]  Raymond J. Mooney,et al.  Theory Refinement with Noisy Data , 1991 .

[30]  Sebastian Thrun,et al.  An approach to learning mobile robot navigation , 1995, Robotics Auton. Syst..

[31]  Manuela Veloso.  Learning by analogical reasoning in general problem-solving , 1992 .

[32]  Sebastian Thrun,et al.  Explanation-Based Neural Network Learning for Robot Control , 1992, NIPS.

[33]  Gerald DeJong,et al.  Schema Acquisition from One Example: Psychological Evidence for Explanation-Based Learning. , 1987 .

[34]  Sebastian Thrun,et al.  Explanation-based neural network learning a lifelong learning approach , 1995 .

[35]  Jude Shavlik,et al.  An Approach to Combining Explanation-based and Neural Learning Algorithms , 1989 .

[36]  Douglas H. Fisher,et al.  Knowledge Acquisition Via Incremental Conceptual Clustering , 1987, Machine Learning.

[37]  Sebastian Thrun,et al.  Discovering Structure in Multiple Learning Tasks: The TC Algorithm , 1996, ICML.

[38]  Lorien Y. Pratt,et al.  Transferring previously learned back-propagation neural networks to new learning tasks , 1993 .

[39]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[40]  Dean A. Pomerleau,et al.  Knowledge-Based Training of Artificial Neural Networks for Autonomous Robot Driving , 1993 .

[41]  Gerald DeJong,et al.  Explanation-Based Learning: An Alternative View , 2005, Machine Learning.

[42]  Tom M. Mitchell,et al.  Generalization as Search , 2002 .

[43]  A. Waibel,et al.  Multi-speaker/speaker-independent architectures for the multi-state time delay neural network , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[44]  Stefan Schaal,et al.  Robot learning by nonparametric regression , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[45]  David Tcheng,et al.  MORE ROBUST CONCEPT LEARNING USING DYNAMICALLY – VARIABLE BIAS , 1987 .

[46]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[47]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  J. Jeffrey Mahoney and Raymond J. Mooney,et al.  Combining Symbolic and Neural Learning to Revise Probabilistic Theories , 1992 .

[49]  Bernard Widrow,et al.  The basic ideas in neural networks , 1994, CACM.

[50]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[51]  Rich Caruana,et al.  Multitask Learning: A Knowledge-Based Source of Inductive Bias , 1993, ICML.

[52]  Francesco Bergadano,et al.  Guiding induction with domain theories , 1990 .

[53]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[54]  Richard S. Sutton,et al.  Integrated Modeling and Control Based on Reinforcement Learning and Dynamic Programming , 1990, NIPS 1990.

[55]  Christopher G. Atkeson,et al.  Using locally weighted regression for robot learning , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.