High Dimension Action Spaces in Robot Skill Learning

Table lookup with interpolation is used for many learning and adaptation tasks. Redundant mappings capture the important concept of "motor skill," which is important in real, behaving systems. Few robot skill implementations have dealt with redundant mappings, in which the space to be searched to create the table has much higher dimensionality than the table. A practical method for inverting redundant mappings is important in physical systems with limited time for trials. We present the "Guided table Fill In" algorithm, which uses data already stored in the table to guide search through the space of potential table entries. The algorithm is illustrated and tested on a robot skill learning task both in simulation and on a robot with a flexible link. Our experiments show that the ability to search high dimensional action spaces efficiently allows skill learners to find new behaviors that are qualitatively different from what they were presented or what the system designer may have expected. Thus the use of this technique can allow researchers to seek higher dimensional action spaces for their systems rather than constraining their search space at the risk of excluding the best actions.

[1]  Bartlett W. Mel.  Connectionist Robot Motion Planning: A Neurally-Inspired Approach to Visually-Guided Reaching , 1990 .

[2]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[3]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[4]  S. Kawamura,et al.  Robot Skill Learning And The Effects Of Basis Function Choice , 1992 .

[5]  Andrew W. Moore,et al.  Efficient memory-based learning for robot control , 1990 .

[6]  C. Gielen,et al.  Neural computation and self-organizing maps, an introduction , 1993 .

[7]  Daniel E. Koditschek,et al.  Progress in spatial robot juggling , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[8]  Amitabha Mukerjee,et al.  Self-calibration in robot manipulators , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[9]  Tomaso A. Poggio,et al.  Extensions of a Theory of Networks for Approximation and Learning , 1990, NIPS.

[10]  Stefan Schaal,et al.  Open loop stable control strategies for robot juggling , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[11]  Michael S. Branicky.  Task-level learning: experiments and extensions , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[12]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[13]  Jeff G. Schneider,et al.  Robot skill learning, basis functions, and control regimes , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[14]  Christopher G. Atkeson,et al.  Using locally weighted regression for robot learning , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[15]  M. Raibert.  Analytical equations vs. table look-up for manipulation: A unifying concept , 1977, 1977 IEEE Conference on Decision and Control including the 16th Symposium on Adaptive Processes and A Special Symposium on Fuzzy Set Theory and Applications.

[16]  Andrew W. Moore,et al.  Variable Resolution Dynamic Programming , 1991, ML Workshop.

[17]  Christopher M. Brown,et al.  Notes on Control with Delay , 1991 .

[18]  W. Thomas Miller,et al.  Real-time application of neural networks for sensor-based control of robots with vision , 1989, IEEE Trans. Syst. Man Cybern..

[19]  David J. Reinkensmeyer,et al.  Using associative content-addressable memories to control robots , 1989, Proceedings, 1989 International Conference on Robotics and Automation.