Learning to Learn: Introduction and Overview

Over the past three decades or so, research on machine learning and data mining has led to a wide variety of algorithms that learn general functions from experience. As machine learning is maturing, it has begun to make the successful transition from academic research to various practical applications. Generic techniques such as decision trees and artificial neural networks, for example, are now being used in various commercial and industrial applications (see e.g., [Langley, 1992; Widrow et al., 1994]).

[1]  R. Franke.  Scattered data interpolation: tests of some methods , 1982 .

[2]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[3]  Bernard Silver,et al.  Using meta-level inference to constrain search and to learn strategies in equation solving , 1984 .

[4]  Paul E. Utgoff,et al.  Shift of bias for inductive concept learning , 1984 .

[5]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[6]  David L. Waltz,et al.  Toward memory-based reasoning , 1986, CACM.

[7]  Paul E. Utgoff,et al.  Machine Learning of Inductive Bias , 1986 .

[8]  David Haussler,et al.  Occam's Razor , 1987, Inf. Process. Lett..

[9]  Gerald DeJong,et al.  Schema Acquisition from One Example: Psychological Evidence for Explanation-Based Learning. , 1987 .

[10]  David Tcheng,et al.  MORE ROBUST CONCEPT LEARNING USING DYNAMICALLY – VARIABLE BIAS , 1987 .

[11]  Larry A. Rendell,et al.  Layered Concept-Learning and Dynamically Variable Bias Management , 1987, IJCAI.

[12]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[13]  Andrew W. Moore,et al.  Efficient memory-based learning for robot control , 1990 .

[14]  S. C. Suddarth,et al.  Rule-Injection Hints as a Means of Improving Network Performance and Learning Time , 1990, EURASIP Workshop.

[15]  Stuart J. Russell.  Prior knowledge and autonomous learning , 1991, Robotics Auton. Syst..

[16]  Christopher G. Atkeson,et al.  Using locally weighted regression for robot learning , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[17]  Steven C. Suddarth,et al.  Symbolic-Neural Systems and the Use of Hints for Developing Complex Systems , 1991, Int. J. Man Mach. Stud..

[18]  Yann LeCun,et al.  Tangent Prop - A Formalism for Specifying Selected Invariances in an Adaptive Network , 1991, NIPS.

[19]  Geoffrey E. Hinton,et al.  Feudal Reinforcement Learning , 1992, NIPS.

[20]  Gerald DeJong.  Investigating Explanation-Based Learning , 1992 .

[21]  T. Poggio,et al.  Recognition and Structure from one 2D Model View: Observations on Prototypes, Object Classes and Symmetries , 1992 .

[22]  Satinder Singh.  Transfer of Learning by Composing Solutions of Elemental Sequential Tasks , 1992, Mach. Learn..

[23]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[24]  Yaser S. Abu-Mostafa,et al.  A Method for Learning From Hints , 1992, NIPS.

[25]  Richard S. Sutton,et al.  Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta , 1992, AAAI.

[26]  Sebastian Thrun,et al.  Explanation-Based Neural Network Learning for Robot Control , 1992, NIPS.

[27]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[28]  Jonas Karlsson,et al.  Learning Multiple Goal Behavior via Task Decomposition and Dynamic Policy Merging , 1993 .

[29]  Lorien Y. Pratt,et al.  Transferring previously learned back-propagation neural networks to new learning tasks , 1993 .

[30]  Rich Caruana,et al.  Multitask Learning: A Knowledge-Based Source of Inductive Bias , 1993, ICML.

[31]  Sebastian Thrun,et al.  Integrating Inductive Neural Network Learning and Explanation-Based Learning , 1993, IJCAI.

[32]  Mark Ring.  Two methods for hierarchy learning in reinforcement environments , 1993 .

[33]  Luc De Raedt,et al.  Multiple Predicate Learning , 1993, IJCAI.

[34]  Dean A. Pomerleau,et al.  Knowledge-Based Training of Artificial Neural Networks for Autonomous Robot Driving , 1993 .

[35]  A. Waibel,et al.  Multi-speaker/speaker-independent architectures for the multi-state time delay neural network , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[36]  W. Ahn,et al.  Psychological Studies of Explanation—Based Learning , 1993 .

[37]  Leslie Pack Kaelbling,et al.  Hierarchical Learning in Stochastic Domains: Preliminary Results , 1993, ICML.

[38]  Bernard Widrow,et al.  Neural networks: applications in industry, business and science , 1994, CACM.

[39]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[40]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex adaptive systems.

[41]  Sebastian Thrun,et al.  Finding Structure in Reinforcement Learning , 1994, NIPS.

[42]  Tom M. Mitchell,et al.  Using the Future to Sort Out the Present: Rankprop and Multitask Learning for Medical Risk Evaluation , 1995, NIPS.

[43]  David Beymer,et al.  Face recognition from one example view , 1995, Proceedings of IEEE International Conference on Computer Vision.

[44]  Sebastian Thrun,et al.  Explanation-based neural network learning a lifelong learning approach , 1995 .

[45]  Carla E. Brodley.  Recursive automatic algorithm selection for inductive learning , 1995 .

[46]  Jonathan Baxter,et al.  Learning internal representations , 1995, COLT '95.

[47]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[48]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Bartlett W. Mel.  SEEMORE: a view-based approach to 3-D object recognition using multiple visual cues , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[50]  Sebastian Thrun,et al.  Discovering Structure in Multiple Learning Tasks: The TC Algorithm , 1996, ICML.

[51]  Rich Caruana,et al.  Algorithms and Applications for Multitask Learning , 1996, ICML.

[52]  Michael J. Pazzani,et al.  Learning sets of related concepts: A shared task model , 1996 .

[53]  Astro Teller,et al.  Evolving programmers: the co-evolution of intelligent recombination operators , 1996 .

[54]  Lorien Y. Pratt,et al.  A Survey of Transfer Between Connectionist Networks , 1996, Connect. Sci..

[55]  Jonathan Baxter,et al.  The Canonical Distortion Measure for Vector Quantization and Function Approximation , 1997, ICML.

[56]  R. Mooney,et al.  A Multistrategy Approach to Theory Refinement , 1997 .

[57]  Astro Teller,et al.  PADO: a new learning architecture for object recognition , 1997 .

[58]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .