Mixed-initiative Variable Autonomy for Remotely Operated Mobile Robots

This article presents an Expert-guided Mixed-initiative Control Switcher (EMICS) for remotely operated mobile robots. The EMICS enables switching between different levels of autonomy during task execution initiated by either the human operator and/or the EMICS. The EMICS is evaluated in two disaster-response-inspired experiments, one with a simulated robot and test arena, and one with a real robot in a realistic environment. Analyses from the two experiments provide evidence that: (a) Human-Initiative (HI) systems outperform systems with single modes of operation, such as pure teleoperation, in navigation tasks; (b) in the context of the simulated robot experiment, Mixed-initiative (MI) systems provide improved performance in navigation tasks, improved operator performance in cognitive demanding secondary tasks, and improved operator workload compared to HI. Last, our experiment on a physical robot provides empirical evidence that identify two major challenges for MI control: (a) the design of context-aware MI control systems; and (b) the conflict for control between the robot’s MI control system and the operator. Insights regarding these challenges are discussed and ways to tackle them are proposed.

[1]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[2]  Robert Goodell Brown,et al.  Smoothing, forecasting and prediction of discrete time series , 1964 .

[3]  Thomas B. Sheridan,et al.  Human and Computer Control of Undersea Teleoperators , 1978 .

[4]  K J Rothman,et al.  No Adjustments Are Needed for Multiple Comparisons , 1990, Epidemiology.

[5]  Darwin G. Caldwell,et al.  Tele-presence: visual, audio and tactile feedback and control of a twin armed mobile robot , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[6]  Darwin G. Caldwell,et al.  Sensory requirements and performance assessment of tele-presence controlled robots , 1996 .

[7]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[8]  T. Perneger.  What's wrong with Bonferroni adjustments , 1998, BMJ.

[9]  Ebrahim H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Hum. Comput. Stud..

[10]  Michael A. Goodrich,et al.  Experiments in adjustable autonomy , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[11]  Robin R. Murphy,et al.  Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[12]  Donald D. Dudenhoeffer,et al.  Mixed-initiative control for remote characterization of hazardous environments , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[13]  Brian A. Weiss,et al.  Test arenas and performance metrics for urban search and rescue robots , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[14]  Robin R. Murphy,et al.  Moonlight in Miami : A Field Study of Human-Robot Interaction in the Context of an Urban Search and Rescue Disaster Response Training Exercise , 2003 .

[15]  Holly A. Yanco,et al.  Improved interfaces for human-robot interaction in urban search and rescue , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[16]  Holly A. Yanco,et al.  Autonomy mode suggestions for improving human-robot interaction , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[17]  Donald D. Dudenhoeffer,et al.  Evaluation of supervisory vs. peer-peer interaction with human-robot teams , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[18]  Robin R. Murphy,et al.  Moonlight in Miami: a field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise , 2004 .

[19]  Ronald L. Boring,et al.  "I call shotgun!": an evaluation of mixed-initiative control for novice users of a search and rescue robot , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[20]  N. Sarkar,et al.  Mixed-Initiative Interaction and Robotic Systems , 2004 .

[21]  Robin R. Murphy,et al.  Human-robot interaction in rescue robotics , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[22]  Jian Shen,et al.  A collaborative-shared control system with safe obstacle avoidance capability , 2004, IEEE Conference on Robotics, Automation and Mechatronics, 2004..

[23]  R. Murphy,et al.  Up from the Rubble: Lessons Learned about HRI from Search and Rescue , 2005 .

[24]  Ronald L. Boring,et al.  Shared understanding for collaborative control , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[25]  Alberto Finzi,et al.  Human-Robot Interaction Through Mixed-Initiative Planning for Rescue and Search Rovers , 2005, AI*IA.

[26]  Douglas A. Few,et al.  Improved Human-Robot Teaming through Facilitated Initiative , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[27]  H. Yanco,et al.  Analysis of Human-Robot Interaction for Urban Search and Rescue , 2006 .

[28]  Vikram Manikonda,et al.  A Mixed Initiative Controller and Testbed for Human Robot Teams in Tactical Operations , 2007, AAAI Fall Symposium: Regarding the Intelligence in Distributed Intelligent Systems.

[29]  Jessie Y. C. Chen,et al.  Human Performance Issues and User Interface Design for Teleoperated Robots , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[30]  C.W. Nielsen,et al.  Using mixed-initiative human-robot interaction to bound performance in a search task , 2008, 2008 International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[31]  Yiannis Demiris,et al.  Collaborative control in human wheelchair interaction reduces the need for dexterity in precise manoeuvres , 2008 .

[32]  Logah Perumal,et al.  A new integrated fuzzy bang–bang relay control system , 2009 .

[33]  Michael A. Goodrich,et al.  On using mixed-initiative control: A perspective for managing large-scale robotic teams , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[34]  Kurt Konolige,et al.  The Office Marathon: Robust navigation in an indoor office environment , 2010, 2010 IEEE International Conference on Robotics and Automation.

[35]  D. Streiner,et al.  Correction for multiple testing: is there a resolution? , 2011, Chest.

[36]  David Sharek,et al.  A Useable, Online NASA-TLX Tool , 2011 .

[37]  Ricardo Chavarriaga,et al.  Online modulation of the level of assistance in shared control systems , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[38]  Juan Rada-Vilela.  fuzzylite a fuzzy logic control library in C + + , 2013 .

[39]  Kazuya Yoshida,et al.  Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots , 2013, J. Field Robotics.

[40]  Siddhartha S. Srinivasa,et al.  A policy-blending formalism for shared control , 2013, Int. J. Robotics Res..

[41]  Julie A. Shah,et al.  Decision-making authority, team efficiency and human worker satisfaction in mixed human–robot teams , 2014, Autonomous Robots.

[42]  G. Ganis,et al.  A New Set of Three-Dimensional Shapes for Investigating Mental Rotation Processes: Validation Data and Stimulus Set , 2015 .

[43]  Holly A. Yanco,et al.  Analysis of Human‐robot Interaction at the DARPA Robotics Challenge Trials , 2015, J. Field Robotics.

[44]  Jess R. Kerlin,et al.  Towards the Principled Study of Variable Autonomy in Mobile Robots , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[45]  Julie A. Shah,et al.  Decision-making authority, team efficiency and human worker satisfaction in mixed human–robot teams , 2015, Auton. Robots.

[46]  Ronald C. Arkin,et al.  Mixed-Initiative Human-Robot Interaction: Definition, Taxonomy, and Survey , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[47]  Robin R. Murphy,et al.  Robots need humans in the loop to improve the hopefulness of disaster survivors , 2015, 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[48]  Thibault Gateau,et al.  Considering human's non-deterministic behavior and his availability state when designing a collaborative human-robots system , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[49]  Goldie Nejat,et al.  Multirobot Cooperative Learning for Semiautonomous Control in Urban Search and Rescue Applications , 2016, J. Field Robotics.

[50]  Rustam Stolkin,et al.  Human-Initiative Variable Autonomy: An Experimental Analysis of the Interactions Between a Human Operator and a Remotely Operated Mobile Robot which also Possesses Autonomous Capabilities , 2016, AAAI Fall Symposia.

[51]  Rustam Stolkin,et al.  Experimental analysis of a variable autonomy framework for controlling a remotely operating mobile robot , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[52]  Julie A. Shah,et al.  Computational design of mixed-initiative human–robot teaming that considers human factors: situational awareness, workload, and workflow preferences , 2017, Int. J. Robotics Res..

[53]  Jean Scholtz,et al.  Analysis of human–robot interaction at the DARPA Robotics Challenge Finals , 2017, Int. J. Robotics Res..

[54]  Thomas B. Sheridan,et al.  A Closed-Loop Model of Operator Visual Attention, Situation Awareness, and Performance Across Automation Mode Transitions , 2017, Hum. Factors.

[55]  Sandra Hirche,et al.  Control sharing in human-robot team interaction , 2017, Annu. Rev. Control..

[56]  Rustam Stolkin,et al.  Learning effects in variable autonomy human-robot systems: how much training is enough? , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).

[57]  Michael Flad,et al.  A Concept for Human-Machine Negotiation in Advanced Driving Assistance Systems , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).

[58]  Brenna Argall,et al.  Probabilistic Human Intent Recognition for Shared Autonomy in Assistive Robotics , 2019, ACM transactions on human-robot interaction.

[59]  Beno Benhabib,et al.  Investigating Human-Robot Teams for Learning-Based Semi-autonomous Control in Urban Search and Rescue Environments , 2018, J. Intell. Robotic Syst..

[60]  Marc Rigter,et al.  A Framework for Learning From Demonstration With Minimal Human Effort , 2020, IEEE Robotics and Automation Letters.

[61]  Frédéric Dehais,et al.  Towards Mixed-Initiative Human–Robot Interaction: Assessment of Discriminative Physiological and Behavioral Features for Performance Prediction , 2020, Sensors.