Evaluating Fluency in Human–Robot Collaboration

Collaborative fluency is the coordinated meshing of joint activities between members of a well-synchronized team. In recent years, researchers in human–robot collaboration have been developing robots to work alongside humans aiming not only at task efficiency, but also at human–robot fluency. As part of this effort, we have developed a number of metrics to evaluate the level of fluency in human–robot shared-location teamwork. While these metrics are being used in existing research, there has been no systematic discussion on how to measure fluency and how the commonly used metrics perform and compare. In this paper, we codify subjective and objective human–robot fluency metrics, provide an analytical model for four objective metrics, and assess their dynamics in a turn-taking framework. We also report on a user study linking objective and subjective fluency metrics and survey recent use of these metrics in the literature.

[1]  Stefanos Nikolaidis,et al.  Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[2]  Maya Cakmak,et al.  Adaptive Coordination Strategies for Human-Robot Handovers , 2015, Robotics: Science and Systems.

[3]  N. Moray,et al.  Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. , 1996, Ergonomics.

[4]  Julie A. Shah,et al.  Coordination of Human-Robot Teaming with Human Task Preferences , 2015, AAAI Fall Symposia.

[5]  Marek P. Michalowski,et al.  A dancing robot for rhythmic social interaction , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[6]  Panos E. Trahanias,et al.  Time-informed task planning in multi-agent collaboration , 2017, Cognitive Systems Research.

[7]  Cynthia Breazeal,et al.  Cost-Based Anticipatory Action Selection for Human–Robot Fluency , 2007, IEEE Transactions on Robotics.

[8]  Guy Hoffman,et al.  Computational Human-Robot Interaction , 2016, Found. Trends Robotics.

[9]  Cynthia Breazeal,et al.  Improved human-robot team performance using Chaski, A human-inspired plan execution system , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[10]  Takayuki Kanda,et al.  How quickly should communication robots respond? , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[11]  Pamela J. Hinds,et al.  Whose job is it anyway? a study of human-robot interaction in a collaborative task , 2004 .

[12]  Crystal Chao,et al.  Timing in multimodal turn-taking interactions , 2012, HRI 2012.

[13]  Muhammad Awais,et al.  Proactive premature intention estimation for intuitive human-robot collaboration , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Minoru Asada,et al.  Initiative in robot assistance during collaborative task execution , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[15]  Julie A. Shah,et al.  Decision-making authority, team efficiency and human worker satisfaction in mixed human–robot teams , 2015, Auton. Robots.

[16]  Siddhartha S. Srinivasa,et al.  Human-robot mutual adaptation in collaborative tasks: Models and experiments , 2017, Int. J. Robotics Res..

[17]  Bilge Mutlu,et al.  Coordination Mechanisms in Human-Robot Collaboration , 2013 .

[18]  Cynthia Breazeal,et al.  Effects of anticipatory perceptual simulation on practiced human-robot tasks , 2010, Auton. Robots.

[19]  Siddhartha S. Srinivasa,et al.  Effects of Robot Motion on Human-Robot Collaboration , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[20]  Gil Weinberg,et al.  Robot-human interaction with an anthropomorphic percussionist , 2006, CHI.

[21]  Ho Chit Siu,et al.  Comparative Performance of Human and Mobile Robotic Assistants in Collaborative Fetch-and-Deliver Tasks , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[22]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[23]  Katsushi Ikeuchi,et al.  Task-model based human robot cooperation using vision , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[24]  Patricia Alves-Oliveira,et al.  “Me and you together” movement impact in multi-user collaboration tasks , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Katsushi Ikeuchi,et al.  Flexible cooperation between human and robot by interpreting human intention from gaze information , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[26]  Stefanos Nikolaidis,et al.  Human-robot cross-training: Computational formulation, modeling and evaluation of a human team training strategy , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  Cynthia Breazeal,et al.  Collaboration in Human-Robot Teams , 2004, AIAA 1st Intelligent Systems Technical Conference.

[28]  A. Horvath,et al.  Development and validation of the Working Alliance Inventory. , 1989 .

[29]  S. M. Mizanoor Rahman,et al.  Cyber-physical-social system between a humanoid robot and a virtual human through a shared platform for adaptive agent ecology , 2018, IEEE/CAA Journal of Automatica Sinica.

[30]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[31]  Donald D. Dudenhoeffer,et al.  Dynamic-Autonomy for Urban Search and Rescue , 2002, AAAI Mobile Robot Competition.

[32]  Wendy Ju,et al.  Expressing thought: Improving robot readability with animation principles , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[33]  Siddhartha S. Srinivasa,et al.  Using spatial and temporal contrast for fluent robot-human hand-overs , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[34]  Julie A. Shah,et al.  Computational design of mixed-initiative human–robot teaming that considers human factors: situational awareness, workload, and workflow preferences , 2017, Int. J. Robotics Res..