The task matrix: a robot-independent framework for programming humanoids

The present state of designing behaviors for humanoid robots can be viewed as analogous to programming a computer using machine language. Even when such complex issues as interaction control and locomotion can be bypassed, programming generally is still pursued in a highly robot-dependent manner. Though roboticists currently use sophisticated tools like human motion capture systems to assist with behavior design, a core tenet of software development-- component reuse-- is frequently overlooked. As a result, a behavior that allows a robot to perform a task will often fail on a different robot, even if the kinematic and dynamic properties between the two robots are highly similar. This problem prevents software reuse and, by extension, limits development of high-level humanoid behaviors for performing a broad range of tasks. This dissertation aims to address the above limitations in the state-of-the-art by first separating humanoid tasks into robot-dependent and robot-independent components, thereby facilitating component reuse, and then identifying and implementing a set of primitive task programs for a humanoid robot that, when executed sequentially and concurrently, allow a broad range of humanoid tasks to be performed. This dissertation presents a framework for integrating a library of performable task behaviors that are portable across many humanoid robots with little or no modification. The framework, called the Task Matrix, provides an application-programming interface (API) to program humanoid robots in an abstract manner; the robot-specific components lie in the implementation of the API. The Task Matrix describes formally how tasks can interact, both in sequence and concurrence. This specification permits complex behaviors to be constructed from a small vocabulary of primitive task behaviors. Such a vocabulary is provided and is applied toward implementing a set of behaviors for executing human occupational tasks. This work is validated on three simulated humanoid robots by performing both the primitive behaviors in various circumstances and complex behaviors composed of the primitives. The contributions of this dissertation are the Task Matrix framework and the set of primitive task programs. It is hoped that roboticists can use these contributions to facilitate performance of occupational tasks on various humanoid robots with minimal effort.

[1]  Stefano Stramigioli,et al.  Impedance control as merging mechanism for a behaviour-based architecture , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[2]  T. Takenaka,et al.  The development of Honda humanoid robot , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[3]  M. Arbib Coordinated control programs for movements of the hand , 1985 .

[4]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[5]  Hirochika Inoue,et al.  Computer Controlled Bilateral Manipulator , 1971 .

[6]  L. C. Pigage,et al.  Motion and time study , 1954 .

[7]  Rüdiger Dillmann,et al.  Teaching and learning of robot tasks via observation of human performance , 2004, Robotics Auton. Syst..

[8]  John F. Hughes,et al.  Plausible motion simulation for computer graphics animation , 1996 .

[9]  Yoshihiko Nakamura,et al.  Inverse kinematic solutions with singularity robustness for robot manipulator control , 1986 .

[10]  Tomás Lozano-Pérez,et al.  Task-level planning of pick-and-place robot motions , 1989, Computer.

[11]  John H. Reif,et al.  Complexity of the mover's problem and generalizations , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[12]  Christopher G. Atkeson,et al.  Learning from observation using primitives , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[13]  Allen Newell,et al.  Chunking in Soar: The anatomy of a general learning mechanism , 1985, Machine Learning.

[14]  J. Hodgins,et al.  Animating Human Athletes , 1998 .

[15]  Maja J. Mataric,et al.  Performance-Derived Behavior Vocabularies: Data-Driven Acquisition of Skills from Motion , 2004, Int. J. Humanoid Robotics.

[16]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[17]  Jason R. Chen,et al.  Constructing Task-Level Assembly Strategies in Robot Programming by Demonstration , 2005, Int. J. Robotics Res..

[18]  Oussama Khatib,et al.  Synthesis of Whole-Body Behaviors through Hierarchical Control of Behavioral Primitives , 2005, Int. J. Humanoid Robotics.

[19]  Roland Carlyle Groome Force feedback steering of a teleoperator system. , 1972 .

[20]  Jeffrey C. Trinkle,et al.  An implicit time-stepping scheme for rigid body dynamics with Coulomb friction , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[21]  Adi Ben-Israel A Newton-Raphson method for the solution of systems of equations , 1966 .

[22]  Walton M. Hancock,et al.  Advanced Work Measurement , 1982 .

[23]  Tomás Lozano-Pérez,et al.  Spatial Planning: A Configuration Space Approach , 1983, IEEE Transactions on Computers.

[24]  J. M. Watt Numerical Initial Value Problems in Ordinary Differential Equations , 1972 .

[25]  Pascal Raymond,et al.  The synchronous data flow programming language LUSTRE , 1991, Proc. IEEE.

[26]  David Baraff,et al.  Fast contact force computation for nonpenetrating rigid bodies , 1994, SIGGRAPH.

[27]  R. Gilmore,et al.  Lie Groups, Lie Algebras, and Some of Their Applications , 1974 .

[28]  Russell H. Taylor,et al.  AML: A Manufacturing Language , 1982 .

[29]  Fred E. Meyers Motion and Time Study for Lean Manufacturing , 1998 .

[30]  Reid G. Simmons,et al.  Structured control for autonomous robots , 1994, IEEE Trans. Robotics Autom..

[31]  Roderic A. Grupen,et al.  A Hybrid Architecture for Learning Robot Control Tasks , 1999 .

[32]  Amy L. Lansky,et al.  Reactive Reasoning and Planning , 1987, AAAI.

[33]  Michael Gleicher,et al.  Retargetting motion to new characters , 1998, SIGGRAPH.

[34]  Ales Ude,et al.  Programming full-body movements for humanoid robots by observation , 2004, Robotics Auton. Syst..

[35]  Victor Ng-Thow-Hing,et al.  The task matrix: an extensible framework for creating versatile humanoid robots , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[36]  Rüdiger Dillmann,et al.  Learning Robot Behaviour and Skills Based on Human Demonstration and Advice: The Machine Learning Paradigm , 2000 .

[37]  Ève Coste-Manière,et al.  Architecture, the backbone of robotic systems , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[38]  Steven M. LaValle,et al.  RRT-connect: An efficient approach to single-query path planning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[39]  Tucker R. Balch,et al.  AuRA: principles and practice in review , 1997, J. Exp. Theor. Artif. Intell..

[40]  Sundar Narasimhan,et al.  Task level strategies for robots , 1994 .

[41]  Gérard Berry,et al.  The foundations of Esterel , 2000, Proof, Language, and Interaction.

[42]  Jun Nakanishi,et al.  Learning Attractor Landscapes for Learning Motor Primitives , 2002, NIPS.

[43]  Ronald C. Arkin,et al.  An Behavior-based Robotics , 1998 .

[44]  Robert James Firby,et al.  Adaptive execution in complex dynamic worlds , 1989 .

[45]  Norman I. Badler,et al.  Real-time virtual humans , 1997, Proceedings The Fifth Pacific Conference on Computer Graphics and Applications.

[46]  Masayuki Inaba,et al.  Humanoid arm motion planning using stereo vision and RRT search , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[47]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[48]  Éric Rutten,et al.  A task-level robot programming language and its reactive execution , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[49]  Bruno Siciliano,et al.  Modeling and Control of Robot Manipulators , 1995 .

[50]  Brian Mirtich,et al.  Fast and Accurate Computation of Polyhedral Mass Properties , 1996, J. Graphics, GPU, & Game Tools.

[51]  Florence Maraninchi,et al.  The Argos Language: Graphical Representation of Automata and Description of Reactive Systems , 2007 .

[52]  Jonathan H. Connell,et al.  SSS: a hybrid architecture applied to robot navigation , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[53]  Ronald Fedkiw,et al.  Nonconvex rigid bodies with stacking , 2003, ACM Trans. Graph..

[54]  Tomás Lozano-Pérez,et al.  An algorithm for planning collision-free paths among polyhedral obstacles , 1979, CACM.

[55]  Paolo Pirjanian,et al.  Behavior Coordination Mechanisms - State-of-the-art , 1999 .

[56]  William S. Davis Robotic task planning: independent of agents but dependent on time , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[57]  Aaron F. Bobick,et al.  Recognition of human body motion using phase space constraints , 1995, Proceedings of IEEE International Conference on Computer Vision.

[58]  Roy Featherstone,et al.  An Empirical Study of the Joint Space Inertia Matrix , 2004, Int. J. Robotics Res..

[59]  Ronan Boulic,et al.  An inverse kinematics architecture enforcing an arbitrary number of strict priority levels , 2004, The Visual Computer.

[60]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[61]  Anthony Levas,et al.  A user-friendly high-level robot teaching system , 1984, ICRA.

[62]  J. Latombe,et al.  Adaptive dynamic collision checking for single and multiple articulated robots in complex environments , 2005, IEEE Transactions on Robotics.

[63]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[64]  Chrystopher L. Nehaniv,et al.  Imitation with ALICE: learning to imitate corresponding actions across dissimilar embodiments , 2002, IEEE Trans. Syst. Man Cybern. Part A.

[65]  Allen Newell,et al.  THE THEORY OF HUMAN PROBLEM SOLVING , 1988 .

[66]  Richard T. Vaughan,et al.  On device abstractions for portable, reusable robot code , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[67]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[68]  Victor Ng-Thow-Hing,et al.  Toward Interactive Reaching in Static Environments for Humanoid Robots , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[69]  Erann Gat,et al.  ESL: a language for supporting robust plan execution in embedded autonomous agents , 1997, 1997 IEEE Aerospace Conference.

[70]  Tomás Lozano-Pérez,et al.  Automatic Planning of Manipulator Transfer Movements , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[71]  N. Hogan,et al.  Impedance Control:An Approach to Manipulation,Parts I,II,III , 1985 .

[72]  Alberta Maria Segre,et al.  Machine Learning of Robot Assembly Plans , 1988 .

[73]  Peter K. Allen,et al.  Graspit! A versatile simulator for robotic grasping , 2004, IEEE Robotics & Automation Magazine.