On Training Automated Agents

Learning to solve problem-solving tasks is a hallmark of intelligence. Intelligent agents learn not only from their own experiences but also from the experiences of others. One would also like a computerized agent to do this: to exploit both its own experiences and those of other agents when learning to solve problem-solving tasks. To this end, we introduce a model of learner/trainer interaction that describes how a learning agent and training agent work together to help the learning agent learn. This proposed model presents the learner as an agent that must make decisions about how it is going to learn. For example, when should the learning agent ask the trainer for help? The training agent must also make decisions about how it interacts with the learning agent. For example, when the learning agent requests help, should the trainer provide it or ignore the request? These decisions drive the interactions, which are the mechanisms by which the training agent provides knowledge to the learning agent. We propose to examine the issues that arise in implementing particular aspects of the learner/trainer model. We will present a restricted learner/trainer model, Interactive Training, and discuss the requirements on the training agent and the learning agent. In our proposed research we will explore the trainer's ability to interact with the learner as well as the learner's ability to beneet from that interaction. Our goal is to develop a systematic method by which a training agent|human or automated|can train a learning agent eeectively, allowing automated agents to be built more quickly. Furthermore, such a training method will allow agents to be built for problems that are currently deemed too diicult for automated learning agents to tackle.

[1]  J. Ross Quinlan,et al.  Simplifying Decision Trees , 1987, Int. J. Man Mach. Stud..

[2]  Kai-Fu Lee,et al.  The Development of a World Class Othello Program , 1990, Artif. Intell..

[3]  Steven Douglas Whitehead,et al.  Reinforcement learning for the adaptive control of perception and action , 1992 .

[4]  Vijaykumar Gullapalli,et al.  Reinforcement learning and its application to control , 1992 .

[5]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[6]  James Patrick Callan,et al.  Knowledge-based feature generation for inductive learning , 1993 .

[7]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[8]  Long Ji Lin,et al.  Programming Robots Using Reinforcement Learning and Teaching , 1991, AAAI.

[9]  Paul E. Utgoff,et al.  Automatic Feature Generation for Problem Solving Systems , 1992, ML.

[10]  Sebastian Thrun,et al.  Active Exploration in Dynamic Environments , 1991, NIPS.

[11]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[12]  Claude Sammut,et al.  Learning to Fly , 1992, ML.

[13]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[14]  S. Thrun.  Eecient Exploration in Reinforcement Learning , 1992 .

[15]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[16]  Leslie Pack Kaelbling,et al.  Learning in embedded systems , 1993 .

[17]  Tom Elliott Fawcett.  Feature discovery for problem solving systems , 1993 .

[18]  Andrew G. Barto,et al.  Connectionist learning for control: an overview , 1990 .

[19]  B. Woolf.  Intelligent tutoring systems: A survey , 1988 .

[20]  Paul E. Utgoff,et al.  Two Kinds of Training Information For Evaluation Function Learning , 1991, AAAI.

[21]  Sanjoy Mahajan,et al.  A Pattern Classification Approach to Evaluation Function Learning , 1988, Artif. Intell..

[22]  Sridhar Mahadevan,et al.  Automatic Programming of Behavior-Based Robots Using Reinforcement Learning , 1991, Artif. Intell..

[23]  Dean Pomerleau,et al.  Rapidly Adapting Artificial Neural Networks for Autonomous Navigation , 1990, NIPS.

[24]  Steven D. Whitehead,et al.  A Complexity Analysis of Cooperative Mechanisms in Reinforcement Learning , 1991, AAAI.

[25]  Paul E. Utgoff,et al.  A Teaching Method for Reinforcement Learning , 1992, ML.

[26]  Paul S. Rosenbloom,et al.  A World-Championship-Level Othello Program , 1982, Artif. Intell..

[27]  Satinder Singh.  Transfer of Learning by Composing Solutions of Elemental Sequential Tasks , 1992, Mach. Learn..

[28]  C.W. Anderson,et al.  Learning to control an inverted pendulum using neural networks , 1989, IEEE Control Systems Magazine.