Human-robot cross-training: Computational formulation, modeling and evaluation of a human team training strategy

We design and evaluate human-robot cross-training, a strategy widely used and validated for effective human team training. Cross-training is an interactive planning method in which a human and a robot iteratively switch roles to learn a shared plan for a collaborative task. We first present a computational formulation of the robot's interrole knowledge and show that it is quantitatively comparable to the human mental model. Based on this encoding, we formulate human-robot cross-training and evaluate it in human subject experiments (n = 36). We compare human-robot cross-training to standard reinforcement learning techniques, and show that cross-training provides statistically significant improvements in quantitative team performance measures. Additionally, significant differences emerge in the perceived robot performance and human trust. These results support the hypothesis that effective and fluent human-robot teaming may be best achieved by modeling effective practices for human teamwork.

[1]  Clint A. Bowers,et al.  The Impact of Cross-Training and Workload on Team Functioning: A Replication and Extension of Initial Findings , 1998, Hum. Factors.

[2]  Peter Stone,et al.  Reinforcement learning from simultaneous human and MDP reward , 2012, AAMAS.

[3]  Andrea Lockerd Thomaz,et al.  Reinforcement Learning with Human Teachers: Evidence of Feedback and Guidance with Implications for Learning Performance , 2006, AAAI.

[4]  C. Burke,et al.  The impact of cross-training on team effectiveness. , 2002, The Journal of applied psychology.

[5]  Manuela M. Veloso,et al.  Teaching multi-robot coordination using demonstration of communication and state sharing , 2008, AAMAS.

[6]  Tamio Arai,et al.  Assessment of operator stress induced by robot collaboration in assembly , 2010 .

[7]  Manuela M. Veloso,et al.  Multi-thresholded approach to demonstration selection for interactive robot learning , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[8]  Maya Cakmak,et al.  Trajectories and keyframes for kinesthetic teaching: A human-robot interaction perspective , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[9]  J. Mathieu,et al.  Performance implications of leader briefings and team-interaction training for team adaptation to novel environments. , 2000, The Journal of applied psychology.

[10]  Eduardo Salas,et al.  Making decisions under stress: Implications for individual and team training. , 1998 .

[11]  Peter Stone,et al.  Combining manual feedback with subsequent MDP reward signals for reinforcement learning , 2010, AAMAS.

[12]  Bruce Blumberg,et al.  Integrated learning for interactive synthetic characters , 2002, SIGGRAPH.

[13]  Pierre-Yves Oudeyer,et al.  Robotic clicker training , 2002, Robotics Auton. Syst..

[14]  Eduardo F. Morales,et al.  Dynamic Reward Shaping: Training a Robot by Voice , 2010, IBERAMIA.

[15]  Thomas M. Cover,et al.  The entropy of Markov trajectories , 1993, IEEE Trans. Inf. Theory.

[16]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[17]  Janice Langan-Fox,et al.  Team Mental Models: Techniques, Methods, and Analytic Approaches , 2000, Hum. Factors.

[18]  Stefanos Nikolaidis,et al.  Human-Robot Interactive Planning using Cross-Training: A Human Team Training Approach , 2012, Infotech@Aerospace.

[19]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[20]  Cynthia Breazeal,et al.  Effects of anticipatory action on human-robot teamwork: Efficiency, fluency, and perception of team , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[21]  Nicholas Roy,et al.  Efficient model learning for dialog management , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[22]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[23]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[24]  E. Salas,et al.  Cross-training and team performance. , 1998 .

[25]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[26]  Cynthia Breazeal,et al.  Real-Time Interactive Reinforcement Learning for Robots , 2005 .

[27]  Monica N. Nicolescu,et al.  Natural methods for robot task learning: instructive demonstrations, generalization and practice , 2003, AAMAS '03.

[28]  Stefan Wermter,et al.  Real-World Reinforcement Learning for Autonomous Humanoid Robot Charging in a Home Environment , 2011, TAROS.

[29]  Cynthia Breazeal,et al.  Improved human-robot team performance using Chaski, A human-inspired plan execution system , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[30]  Rakesh Gupta,et al.  Smoothed Sarsa: Reinforcement learning for robot delivery tasks , 2009, 2009 IEEE International Conference on Robotics and Automation.

[31]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[32]  Kevin Waugh,et al.  Computational Rationalization: The Inverse Equilibrium Problem , 2011, ICML.

[33]  Peter Stone,et al.  Interactively shaping agents via human reinforcement: the TAMER framework , 2009, K-CAP '09.