Abstraction in data-sparse task transfer

Abstract When a robot adapts a learned task for a novel environment, any changes to objects in the novel environment have an unknown effect on its task execution. For example, replacing an object in a pick-and-place task affects where the robot should target its actions, but does not necessarily affect the underlying action model. In contrast, replacing a tool that the robot will use to complete a task will effectively alter its end-effector pose with respect to the robot's base coordinate system, and thus the robot's motion must be replanned accordingly. These examples highlight the relationship among (i) differences between the source and target environments, (ii) the level of abstraction at which a robot's task model should be represented to enable transfer to the target environment, and (iii) the information needed to ground the abstracted task representation in the target environment. In this article, we present a taxonomy of transfer problems based on this relationship. We also describe a knowledge representation called the Tiered Task Abstraction (TTA) and demonstrate its applicability to a variety of transfer problems in the taxonomy. Our experimental results indicate a trade-off between the generality and data requirements of a task representation, and reinforce the need for multiple transfer methods that operate at different levels of abstraction.

[1]  Moritz Tenorth,et al.  Understanding and executing instructions for everyday manipulation tasks from the World Wide Web , 2010, 2010 IEEE International Conference on Robotics and Automation.

[2]  Danica Kragic,et al.  Learning Task Models from Multiple Human Demonstrations , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[3]  Yuchen Cui,et al.  Active Reward Learning from Critiques , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Maya Cakmak,et al.  Trajectories and keyframes for kinesthetic teaching: A human-robot interaction perspective , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[5]  John R. Anderson,et al.  Interactive Task Learning , 2017, IEEE Intelligent Systems.

[6]  Kevin Lee,et al.  Tell me Dave: Context-sensitive grounding of natural language to manipulation instructions , 2014, Int. J. Robotics Res..

[7]  Finale Doshi-Velez,et al.  Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes , 2017, AAAI.

[8]  John E. Laird,et al.  Learning Hierarchical Symbolic Representations to Support Interactive Task Learning and Knowledge Transfer , 2019, IJCAI.

[9]  Darwin G. Caldwell,et al.  Learning adaptive movements from demonstration and self-guided exploration , 2014, 4th International Conference on Development and Learning and on Epigenetic Robotics.

[10]  Brian Falkenhainer,et al.  The Structure-Mapping Engine: Algorithm and Examples , 1989, Artif. Intell..

[11]  Francesco Chinello,et al.  KUKA Control Toolbox , 2011, IEEE Robotics & Automation Magazine.

[12]  Anca D. Dragan,et al.  Active Preference-Based Learning of Reward Functions , 2017, Robotics: Science and Systems.

[13]  Shimon Whiteson,et al.  Transfer via inter-task mappings in policy search reinforcement learning , 2007, AAMAS '07.

[14]  Aude Billard,et al.  A probabilistic Programming by Demonstration framework handling constraints in joint space and task space , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[16]  Zoe Falomir,et al.  Object replacement and object composition in a creative cognitive system. Towards a computational solver of the Alternative Uses Test , 2016, Cognitive Systems Research.

[17]  Andrea Lockerd Thomaz,et al.  Robot Learning from Human Teachers , 2014, Robot Learning from Human Teachers.

[18]  J. Sinapov,et al.  Detecting the functional similarities between tools using a hierarchical representation of outcomes , 2008, 2008 7th IEEE International Conference on Development and Learning.

[19]  Sergey Levine,et al.  One-shot learning of manipulation skills with online dynamics adaptation and neural network priors , 2015, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[20]  Ashok K. Goel.  Design, Analogy, and Creativity , 1997, IEEE Expert.

[21]  Paul Thagard,et al.  Analog Wetrieval by Constraint Satisfaction , 1990, Artif. Intell..

[22]  Philippe A. Palanque,et al.  A generic tool-supported framework for coupling task models and interactive applications , 2015, EICS.

[23]  John E. Laird,et al.  Learning task goals interactively with visual demonstrations , 2016, BICA 2016.

[24]  Manuela M. Veloso,et al.  Confidence-based policy learning from demonstration using Gaussian mixture models , 2007, AAMAS '07.

[25]  Scott Kuindersma,et al.  Robot learning from demonstration by constructing skill trees , 2012, Int. J. Robotics Res..

[26]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[27]  Ashok K. Goel,et al.  Use of design patterns in analogy-based design , 2004, Adv. Eng. Informatics.

[28]  Allan Jabri,et al.  Universal Planning Networks , 2018, ICML.

[29]  Matthias Scheutz,et al.  Spoken Instruction-Based One-Shot Object and Action Learning in a Cognitive Robotic Architecture , 2017, AAMAS.

[30]  Andrea Lockerd Thomaz,et al.  Human-Robot Co-Creativity: Task Transfer on a Spectrum of Similarity , 2017, ICCC.

[31]  S. Schaal.  Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics , 2006 .

[32]  Marcin Andrychowicz,et al.  One-Shot Imitation Learning , 2017, NIPS.

[33]  Babak Esfandiari,et al.  A Case-Based Reasoning Framework for Developing Agents Using Learning by Observation , 2011, 2011 IEEE 23rd International Conference on Tools with Artificial Intelligence.

[34]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[35]  Hexiang Hu,et al.  Synthesize Policies for Transfer and Adaptation across Tasks and Environments , 2019, NeurIPS.

[36]  Ashok K. Goel,et al.  Human-Guided Object Mapping for Task Transfer , 2018, ACM Transactions on Human-Robot Interaction.

[37]  Andrea Lockerd Thomaz,et al.  Human-guided Trajectory Adaptation for Tool Transfer , 2019, AAMAS.

[38]  Ayanna M. Howard,et al.  Case-Based Reasoning for planning turn-taking strategy with a therapeutic robot playmate , 2010, 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.

[39]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[40]  Andrea Lockerd Thomaz,et al.  Grounding action parameters from demonstration , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[41]  Janet L. Kolodner,et al.  Case-Based Reasoning , 1989, IJCAI 1989.

[42]  Andrea Lockerd Thomaz,et al.  Simultaneously learning actions and goals from demonstration , 2016, Auton. Robots.