Developing performance metrics for the supervisory control of multiple robots

Efforts are underway to make it possible for a single operator to effectively control multiple robots. In these high workload situations, many questions arise including how many robots should be in the team (Fan-out), what level of autonomy should the robots have, and when should this level of autonomy change (i.e., dynamic autonomy). We propose that a set of metric classes should be identified that can adequately answer these questions. Toward this end, we present a potential set of metric classes for human-robot teams consisting of a single human operator and multiple robots. To test the usefulness and appropriateness of this set of metric classes, we conducted a user study with simulated robots. Using the data obtained from this study, we explore the ability of this set of metric classes to answer these questions.

[1]  J. Veltman,et al.  Physiological workload reactions to increasing levels of task difficulty. , 1998, Ergonomics.

[2]  Jean Scholtz,et al.  Awareness in human-robot interactions , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[3]  Michael A. Goodrich,et al.  Task Switching and Multi-Robot Teams , 2005 .

[4]  Heath A. Ruff,et al.  EXPLORING AUTOMATION ISSUES IN SUPERVISORY CONTROL OF MULTIPLE UAVS , 2004 .

[5]  Hankins Tc,et al.  A comparison of heart rate, eye activity, EEG and subjective measures of pilot mental workload during flight. , 1998, Aviation, space, and environmental medicine.

[6]  Stephanie Guerlain,et al.  An Interactive Decision Support Tool for Real-time In-flight Replanning of Autonomous Vehicles , 2004 .

[7]  Thomas B. Sheridan,et al.  Telerobotics , 1989, Autom..

[8]  Thomas B. Sheridan,et al.  Telerobotics, Automation, and Human Supervisory Control , 2003 .

[9]  Michael A. Goodrich,et al.  Seven principles of efficient human robot interaction , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[10]  Thomas B. Sheridan,et al.  Mitigation of human supervisory control wait times through automation strategies , 2005 .

[11]  J. Gregory Trafton,et al.  Human control of multiple unmanned vehicles: effects of interface type on execution and task switching times , 2006, HRI '06.

[12]  Thomas B. Sheridan,et al.  Human and Computer Control of Undersea Teleoperators , 1978 .

[13]  Mica R. Endsley,et al.  Design and Evaluation for Situation Awareness Enhancement , 1988 .

[14]  Reid G. Simmons,et al.  Preliminary results in sliding autonomy for assembly by coordinated teams , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[15]  Jean Scholtz,et al.  Common metrics for human-robot interaction , 2006, HRI '06.

[16]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[17]  Hiroshi Furukawa,et al.  A flexible delegation-type interface enhances system performance in human supervision of multiple robots: empirical studies with RoboFlag , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[18]  Jacob W. Crandall,et al.  Predicting Operator Capacity for Supervisory Control of Multiple UAVs , 2007, Innovations in Intelligent Machines.

[19]  Dan R. Olsen,et al.  Metrics for Evaluating Human-Robot Interactions , 2003 .

[20]  Christopher D. Wickens,et al.  Unmanned Aerial Vehicle Flight Control: False Alarms versus Misses , 2004 .

[21]  Jeffrey D. Anderson,et al.  Managing autonomy in robot teams: Observations from four experiments , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[22]  A. Freedy,et al.  A Comprehensive Methodology for Assessing Human-Robot Team Performance for Use in Training and Simulation , 2006 .

[23]  Guillermo Rodríguez-Ortiz,et al.  A New Method to Evaluate Human-Robot System Performance , 2003, Auton. Robots.

[24]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[25]  Michael A. Goodrich,et al.  Validating human-robot interaction schemes in multitasking environments , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[26]  Dan R. Olsen,et al.  Fan-out: measuring human control of multiple robots , 2004, CHI.