Online Development of Assistive Robot Behaviors for Collaborative Manipulation and Human-Robot Teamwork

Collaborative robots that operate in the same immediate environment as human workers have the potential to improve their co-workers' efficiency and quality of work. In this paper we present a taxonomy of assistive behavior types alongside methods that enable a robot to learn assistive behaviors from interactions with a human collaborator during live activity completion. We begin with a brief survey of the state of the art in human-robot collaboration. We proceed to focus on the challenges and issues surrounding the online development of assistive robot behaviors. Finally, we describe approaches for learning when and how to apply these behaviors, as well as for integrating them into a full end-to-end system utilizing techniques derived from the learning from demonstration, policy iteration, and task network communities.

[1]  Maya Cakmak,et al.  Keyframe-based Learning from Demonstration , 2012, Int. J. Soc. Robotics.

[2]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[3]  Manuela M. Veloso,et al.  Interactive robot task training through dialog and demonstration , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[4]  Alexander Zelinsky,et al.  Programing by Demonstration: Coping with Suboptimal Teaching Actions , 2003 .

[5]  G. Monahan.  State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[6]  Scott Kuindersma,et al.  Autonomous Skill Acquisition on a Mobile Manipulator , 2011, AAAI.

[7]  S. Schaal.  Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics , 2006 .

[8]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[9]  Katsushi Ikeuchi,et al.  Generation of a task model by integrating multiple observations of human demonstrations , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[10]  Doina Precup,et al.  Learning Options in Reinforcement Learning , 2002, SARA.

[11]  Fei Yu,et al.  Towards Successful Coordination of Human and Robotic Work using Automated Scheduling Tools : An Initial Pilot Study , 2013 .

[12]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[13]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[14]  Ross A. Knepper,et al.  IkeaBot: An autonomous multi-robot coordinated furniture assembly system , 2013, 2013 IEEE International Conference on Robotics and Automation.

[15]  Stefanos Nikolaidis,et al.  Human-robot cross-training: Computational formulation, modeling and evaluation of a human team training strategy , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[16]  Scott Kuindersma,et al.  Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories , 2010, NIPS.

[17]  Ben Tse,et al.  Autonomous Inverted Helicopter Flight via Reinforcement Learning , 2004, ISER.

[18]  Roderic A. Grupen,et al.  A model of shared grasp affordances from demonstration , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[19]  Siddhartha S. Srinivasa,et al.  CHOMP: Covariant Hamiltonian optimization for motion planning , 2013, Int. J. Robotics Res..

[20]  Lydia E. Kavraki,et al.  The Open Motion Planning Library , 2012, IEEE Robotics & Automation Magazine.

[21]  Aaron F. Bobick,et al.  Modeling structured activity to support human-robot collaboration in the presence of task and sensor uncertainty , 2013 .

[22]  Rüdiger Dillmann,et al.  Learning Robot Behaviour and Skills Based on Human Demonstration and Advice: The Machine Learning Paradigm , 2000 .