Learning from Observation and Practice at the Action Generation Level

For humanoid robots to be accepted as partners with humans they will be expected to learn quickly and adapt to environment changes in ways similar to humans. Within a single task there may be many things that can change as humans interact with the robot. The placement of items within the workspace, the physics of the environment, and the reaction of the human to the robot’s movements are some examples of the things which can vary during the interaction. This paper presents a method for humanoid robots to quickly learn new dynamic tasks from observing others and from practice. Ways in which the robot can adapt to the initial and slowly changing environment conditions are also described. Agents are given domain knowledge in the form of task primitives. A key element of our approach is to break learning problems up into as many simple learning problems as possible. This process of “divide and conquer” is limited only by the measurements available to the robot. We present a case study of a humanoid robot learning to play air hockey.

[1]  Wayne Iba,et al.  Learning to Classify Observed Motor Behavior , 1991, IJCAI.

[2]  Mark W. Spong,et al.  An experimental testbed for intelligent control , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[3]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[4]  Stefan Schaal,et al.  Locally Weighted Projection Regression : An O(n) Algorithm for Incremental Real Time Learning in High Dimensional Space , 2000 .

[5]  Mark W. Spong,et al.  Control of Planar Rigid Body Sliding with Impacts and Friction , 2000, Int. J. Robotics Res..

[6]  Andrew G. Barto,et al.  Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density , 2001, ICML.

[7]  Mark W. Spong,et al.  Neural network-based state prediction for strategy planning of an air hockey robot , 2001, J. Field Robotics.

[8]  Christopher G. Atkeson,et al.  Learning from observation using primitives , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[9]  Maja J. Mataric,et al.  Automated Derivation of Primitives for Movement Classification , 2000, Auton. Robots.

[10]  Manuela M. Veloso,et al.  Learning the Sequential Coordinated Behavior of Teams from Observations , 2002, RoboCup.

[11]  Christopher G. Atkeson,et al.  A Framework for Learning from Observation Using Primitives , 2002, RoboCup.

[12]  Gordon Cheng,et al.  Humanoid robot learning and game playing using PC-based vision , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Stefan Schaal,et al.  Statistical Learning for Humanoid Robots , 2002, Auton. Robots.

[14]  Andrew W. Moore,et al.  Locally Weighted Learning , 1997, Artificial Intelligence Review.