Towards Bi-Directional Communication in Human-Swarm Teaming: A Survey

Swarm systems consist of large numbers of robots that collaborate autonomously. With an appropriate level of human control, swarm systems could be applied in a variety of contexts ranging from search-and-rescue situations to Cyber defence. The two decision making cycles of swarms and humans operate on two different time-scales, where the former is normally orders of magnitude faster than the latter. Closing the loop at the intersection of these two cycles will create fast and adaptive human-swarm teaming networks. This paper brings desperate pieces of the ground work in this research area together to review this multidisciplinary literature. We conclude with a framework to synthesize the findings and summarize the multi-modal indicators needed for closed-loop human-swarm adaptive systems.

[1]  Michael C. Dorneich,et al.  Towards a Characterization of Adaptive Systems: a Framework for Researchers and System Designers , 2017 .

[2]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[3]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[4]  Raja Parasuraman,et al.  Designing for Flexible Interaction Between Humans and Automation: Delegation Interfaces for Supervisory Control , 2007, Hum. Factors.

[5]  Dan R. Olsen,et al.  Fan-out: measuring human control of multiple robots , 2004, CHI.

[6]  Majid Fallahi,et al.  Effects of mental workload on physiological and subjective responses during traffic density monitoring: A field study. , 2016, Applied ergonomics.

[7]  Michael A. Goodrich,et al.  Scalable Human Interaction with Robotic Swarms , 2013 .

[8]  T. Finkenzeller,et al.  Frontal midline theta in the pre-shot phase of rifle shooting: Differences between experts and novices , 2008, Neuropsychologia.

[9]  F. Bullo,et al.  Visibility-based multi-agent deployment in orthogonal environments , 2007, 2007 American Control Conference.

[10]  Gaurav S. Sukhatme,et al.  Mobile Sensor Network Deployment using Potential Fields : A Distributed , Scalable Solution to the Area Coverage Problem , 2002 .

[11]  Daniel J. Barber,et al.  The Psychometrics of Mental Workload , 2015, Hum. Factors.

[12]  Stephen M. Fiore,et al.  Towards Modeling Social-Cognitive Mechanisms in Robots to Facilitate Human-Robot Teaming , 2013, Proceedings of the Human Factors and Ergonomics Society Annual Meeting.

[13]  Chongxun Zheng,et al.  Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. , 2012, Accident; analysis and prevention.

[14]  William M. Jones,et al.  Situation Awareness Information Dominance & Information Warfare. , 1997 .

[15]  Katia P. Sycara,et al.  Human Interaction With Robot Swarms: A Survey , 2016, IEEE Transactions on Human-Machine Systems.

[16]  Julie A. Adams,et al.  Heuristic Evaluation of Swarm Metrics' Effectiveness , 2015, HRI.

[17]  Sean Hayes,et al.  Biologically-Inspired Human-Swarm Interaction Metrics , 2014 .

[18]  Julie A. Adams,et al.  Human-Swarm Interaction: Sources of Uncertainty , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[19]  Mark W. Scerbo,et al.  Effects of a Psychophysiological System for Adaptive Automation on Performance, Workload, and the Event-Related Potential P300 Component , 2003, Hum. Factors.

[20]  Raja Parasuraman,et al.  Performance Consequences of Automation-Induced 'Complacency' , 1993 .

[21]  Katia P. Sycara,et al.  Neglect Benevolence in human control of robotic swarms , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Nigel H. Lovell,et al.  Estimating cognitive workload using wavelet entropy-based features during an arithmetic task , 2013, Comput. Biol. Medicine.

[23]  M. Hakel,et al.  Effects of Objective and Subjective Task Complexity on Performance , 1997 .

[24]  Hussein A. Abbass,et al.  Augmented Cognition using Real-time EEG-based Adaptive Strategies for Air Traffic Control , 2014 .

[25]  Jean Scholtz,et al.  Evaluation of Operator Interventions in Autonomous Off-road Driving , 2003 .

[26]  Sven Fuchs,et al.  Towards a Dynamic Selection and Configuration of Adaptation Strategies in Augmented Cognition , 2017, HCI.

[27]  Joost C. F. de Winter,et al.  Review of eye-related measures of drivers’ mental workload , 2015 .

[28]  Mica R. Endsley,et al.  The Hunt for Situation Awareness: Human-Robot Interaction in Search and Rescue , 2004 .

[29]  Jean Scholtz,et al.  Evaluation of a human-robot interface: development of a situational awareness methodology , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[30]  Katia Sycara,et al.  Integrating intelligent agents into human teams. , 2004 .

[31]  Dan R. Olsen,et al.  Metrics for Evaluating Human-Robot Interactions , 2003 .

[32]  Adam Jacoff,et al.  A Standard Test Course for Urban Search and Rescue Robots , 2000 .

[33]  Michael A. Goodrich,et al.  Seven principles of efficient human robot interaction , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[34]  Sean Kerman,et al.  Methods and Metrics for Human Interaction with Bio-Inspired Robot Swarms , 2013 .

[35]  Adam Jacoff,et al.  Reference test courses for autonomous mobile robots , 2001, SPIE Defense + Commercial Sensing.

[36]  Christina F. Rusnock,et al.  The Impact of Adaptive Automation Invoking Thresholds on Cognitive Workload and Situational Awareness , 2013 .

[37]  E. Donchin,et al.  P300 and tracking difficulty: evidence for multiple resources in dual-task performance. , 1980, Psychophysiology.

[38]  Jean Scholtz,et al.  Common metrics for human-robot interaction , 2006, HRI '06.

[39]  Gita Sukthankar,et al.  Literature Review of Teamwork Models , 2006 .

[40]  Katia P. Sycara,et al.  Investigating Neglect Benevolence and Communication Latency During Human-Swarm Interaction , 2012, AAAI Fall Symposium: Human Control of Bioinspired Swarms.

[41]  N. John Castellan,et al.  Individual and group decision making : current issues , 2013 .

[42]  Nancy J. Cooke,et al.  On Teams, Teamwork, and Team Performance: Discoveries and Developments , 2008, Hum. Factors.

[43]  Yang Yang,et al.  Human-Robot Interaction in UVs Swarming: A Survey , 2013 .

[44]  A. Edmondson,et al.  Teaming: How Organizations Learn, Innovate, and Compete in the Knowledge Economy , 2012 .

[45]  Michael Lewis,et al.  Towards human control of robot swarms , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[46]  B. B. Morgan,et al.  An Analysis of Team Evolution and Maturation , 1993 .

[47]  Michael A. Goodrich,et al.  Human influence of robotic swarms with bandwidth and localization issues , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[48]  Jacob W. Crandall,et al.  Identifying Predictive Metrics for Supervisory Control of Multiple Robots , 2007, IEEE Transactions on Robotics.

[49]  Iñaki Navarro,et al.  A Proposal of a Set of Metrics for Collective Movement of Robots , 2009 .

[50]  Katia P. Sycara,et al.  Neglect benevolence in human control of swarms in the presence of latency , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[51]  Michael A. Goodrich,et al.  Validating human-robot interaction schemes in multitasking environments , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[52]  John M. O'Hara,et al.  Identifying Requirements for Effective Human-Automation Teamwork , 2014 .

[53]  Jean Scholtz,et al.  Beyond usability evaluation: analysis of human-robot interaction at a major robotics competition , 2004 .

[54]  Jessie Y. C. Chen,et al.  Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues , 2014, IEEE Transactions on Human-Machine Systems.

[55]  Hussein A. Abbass,et al.  Trusted Autonomy and Cognitive Cyber Symbiosis: Open Challenges , 2015, Cognitive Computation.

[56]  J. L. Dyer,et al.  Team research and team training: A state-of-the-art review , 1984 .

[57]  James McLurkin,et al.  Speaking Swarmish: Human-Robot Interface Design for Large Swarms of Autonomous Mobile Robots , 2006, AAAI Spring Symposium: To Boldly Go Where No Human-Robot Team Has Gone Before.

[58]  A. Ollero,et al.  Immersive displays for building spatial knowledge in multi-UAV operations , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[59]  J. Katzenbach,et al.  The discipline of teams. , 1993, Harvard business review.

[60]  Robert P. Goldman,et al.  The Playbook™ Approach to Adaptive Automation , 2005 .

[61]  Iris Vessey,et al.  Cognitive Fit: A Theory‐Based Analysis of the Graphs Versus Tables Literature* , 1991 .

[62]  Zhizhong Li,et al.  Task complexity: A review and conceptualization framework , 2012 .

[63]  Steven V. Viscido,et al.  Self-Organized Fish Schools: An Examination of Emergent Properties , 2002, The Biological Bulletin.

[64]  Heath A. Ruff,et al.  Human Interaction with Levels of Automation and Decision-Aid Fidelity in the Supervisory Control of Multiple Simulated Unmanned Air Vehicles , 2002, Presence: Teleoperators & Virtual Environments.

[65]  Jennifer M. Riley,et al.  Effects of Robot Control Mode on Situation Awareness and Performance in a Navigation Task , 2006 .

[66]  Brian Hilburn,et al.  DYNAMIC DECISION AIDING: THE IMPACT OF ADAPTIVE AUTOMATION ON MENTAL WORKLOAD , 1997 .

[67]  James S. Albus,et al.  Toward a Generic Model for Autonomy Levels for Unmanned Systems (ALFUS) , 2003 .

[68]  Bart van den Hooff,et al.  Lost in space? Cognitive fit and cognitive load in 3D virtual environments , 2013, Comput. Hum. Behav..