Adaptive Workload Allocation for Multi-Human Multi-Robot Teams for Independent and Homogeneous Tasks

Multi-human multi-robot (MH-MR) systems have the ability to combine the potential advantages of robotic systems with those of having humans in the loop. Robotic systems contribute precision performance and long operation on repetitive tasks without tiring, while humans in the loop improve situational awareness and enhance decision-making abilities. A system’s ability to adapt allocated workload to changing conditions and the performance of each individual (human and robot) during the mission is vital to maintaining overall system performance. Previous works from literature including market-based and optimization approaches have attempted to address the task/workload allocation problem with focus on maximizing the system output without regarding individual agent conditions, lacking in real-time processing and have mostly focused exclusively on multi-robot systems. Given the variety of possible combination of teams (autonomous robots and human-operated robots: any number of human operators operating any number of robots at a time) and the operational scale of MH-MR systems, development of a generalized framework of workload allocation has been a particularly challenging task. In this article, we present such a framework for independent homogeneous missions, capable of adaptively allocating the system workload in relation to health conditions and work performances of human-operated and autonomous robots in real-time. The framework consists of removable modular function blocks ensuring its applicability to different MH-MR scenarios. A new workload transition function block ensures smooth transition without the workload change having adverse effects on individual agents. The effectiveness and scalability of the system’s workload adaptability is validated by experiments applying the proposed framework in a MH-MR patrolling scenario with changing human and robot condition, and failing robots.

[1]  J. Christopher Beck,et al.  A Constraint Programming Approach to Multi-Robot Task Allocation and Scheduling in Retirement Homes , 2016, CP.

[2]  Jean Scholtz,et al.  Awareness in human-robot interactions , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[3]  Erik Prytz,et al.  Workload transitions and stress: Changes over time , 2014 .

[4]  Marjorie Darrah,et al.  Multiple UAV Dynamic Task Allocation using Mixed Integer Linear Programming in a SEAD Mission , 2005 .

[5]  Xiaomin Li,et al.  Multi-robot Task Allocation Based on Ant Colony Algorithm , 2012, J. Comput..

[6]  Dov Eden,et al.  The inverted-U relationship between stress and performance: A field study , 1996 .

[7]  Sergio Escalera,et al.  Survey on Emotional Body Gesture Recognition , 2018, IEEE Transactions on Affective Computing.

[8]  Sandra Hirche,et al.  Control sharing in human-robot team interaction , 2017, Annu. Rev. Control..

[9]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[10]  Ioannis M. Rekleitis,et al.  Efficient multi-robot coverage of a known environment , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  D. Bai,et al.  Stress and Heart Rate Variability: A Meta-Analysis and Review of the Literature , 2018, Psychiatry investigation.

[12]  Alexis Drogoul,et al.  Multi-agent Patrolling: An Empirical Analysis of Alternative Architectures , 2002, MABS.

[13]  Davide Carneiro,et al.  Multimodal behavioral analysis for non-invasive stress detection , 2012, Expert Syst. Appl..

[14]  Jennifer A. Healey,et al.  Wearable and automotive systems for affect recognition from physiology , 2000 .

[15]  Lorenzo Sabattini,et al.  Natural interaction based on affective robotics for multi-robot systems , 2017, 2017 International Symposium on Multi-Robot and Multi-Agent Systems (MRS).

[16]  Wolfram Burgard,et al.  A hybrid collision avoidance method for mobile robots , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[17]  Takashi Minato,et al.  How Should a Robot React Before People's Touch?: Modeling a Pre-Touch Reaction Distance for a Robot's Face , 2018, IEEE Robotics and Automation Letters.

[18]  Dong-Hyun Lee,et al.  Resource-based task allocation for multi-robot systems , 2018, Robotics Auton. Syst..

[19]  Francesco Leali,et al.  Survey on Human-Robot Interaction for Robot Programming in Industrial Applications , 2018 .

[20]  Jean Scholtz,et al.  Common metrics for human-robot interaction , 2006, HRI '06.

[21]  Trang T. Le,et al.  Scaling tree-based automated machine learning to biomedical big data with a feature set selector , 2019, Bioinform..

[22]  Salim Lahmiri,et al.  Physiological signal denoising with variational mode decomposition and weighted reconstruction after DWT thresholding , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[23]  Dayal R. Parhi,et al.  Mobile Robot Navigation and Obstacle Avoidance Techniques: A Review , 2017, ICRA 2017.

[24]  Katia P. Sycara,et al.  Scheduling operator attention for Multi-Robot Control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Raja Parasuraman,et al.  Designing for Flexible Interaction Between Humans and Automation: Delegation Interfaces for Supervisory Control , 2007, Hum. Factors.

[26]  Anthony Stentz,et al.  Market-based Multirobot Coordination for Complex Tasks , 2006, Int. J. Robotics Res..

[27]  Sotiris Makris,et al.  On a human–robot workplace design and task allocation system , 2017, Int. J. Comput. Integr. Manuf..

[28]  Aude Billard,et al.  A dynamical system approach to task-adaptation in physical human–robot interaction , 2019, Auton. Robots.

[29]  Jose C. Alves,et al.  Navigation performance of an autonomous sailing robot , 2014, 2014 Oceans - St. John's.

[30]  Hong Ren Wu,et al.  Facial Expression Recognition in Perceptual Color Space , 2012, IEEE Transactions on Image Processing.

[31]  Raja Parasuraman,et al.  Adaptive Automation for Human-Robot Teaming in Future Command and Control Systems , 2007 .

[32]  Emil Jovanov,et al.  Stress monitoring using a distributed wireless intelligent sensor system. , 2003, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[33]  Magnus Egerstedt,et al.  Energy-Constrained Coordination of Multi-Robot Teams , 2017, IEEE Transactions on Control Systems Technology.

[34]  Himanshu Thapliyal,et al.  A Survey of Affective Computing for Stress Detection: Evaluating technologies in stress detection for better health , 2016, IEEE Consumer Electronics Magazine.

[35]  Donald A. Sofge,et al.  Auctions for multi-robot task allocation in communication limited environments , 2020, Auton. Robots.

[36]  D. Tzovaras,et al.  Using Activity-Related Behavioural Features towards More Effective Automatic Stress Detection , 2012, PloS one.

[37]  Edward Tunstel,et al.  Fuzzy logic based collision avoidance for a mobile robot , 1994, Robotica.

[38]  Mehrdad Nojoumian,et al.  A Survey on Trust in Autonomous Systems , 2018, Advances in Intelligent Systems and Computing.

[39]  Anthony Stentz,et al.  Time-extended multi-robot coordination for domains with intra-path constraints , 2011, Auton. Robots.

[40]  Daniel Gatica-Perez,et al.  StressSense: detecting stress in unconstrained acoustic environments using smartphones , 2012, UbiComp.

[41]  A. Barreto,et al.  Stress Detection in Computer Users Based on Digital Signal Processing of Noninvasive Physiological Variables , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[42]  Tamás D. Gedeon,et al.  Objective measures, sensors and computational techniques for stress recognition and classification: A survey , 2012, Comput. Methods Programs Biomed..

[43]  Siddhartha S. Mehta,et al.  Optimal task allocation in multi-human multi-robot interaction , 2015, Optimization Letters.

[44]  Akane Sano,et al.  Stress Recognition Using Wearable Sensors and Mobile Phones , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[45]  Ifeyinwa E. Achumba,et al.  Real-Time Human Ambulation, Activity, and Physiological Monitoring: Taxonomy of Issues, Techniques, Applications, Challenges and Limitations , 2013, Sensors.

[46]  Lynne E. Parker,et al.  A Complete Methodology for Generating Multi-Robot Task Solutions using ASyMTRe-D and Market-Based Task Allocation , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[47]  Glenn F. Wilson,et al.  Operator Functional State Classification Using Multiple Psychophysiological Features in an Air Traffic Control Task , 2003, Hum. Factors.

[48]  R G Mark,et al.  Robust heart rate estimation from multiple asynchronous noisy sources using signal quality indices and a Kalman filter , 2008, Physiological measurement.

[49]  Daniel McDuff,et al.  Remote measurement of cognitive stress via heart rate variability , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[50]  Nidhi Kalra,et al.  Market-Based Multirobot Coordination: A Survey and Analysis , 2006, Proceedings of the IEEE.

[51]  Zhiwei Zhu,et al.  A Real-Time Human Stress Monitoring System Using Dynamic Bayesian Network , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[52]  Karl M. Reichard.  Integrating self-health awareness in autonomous systems , 2004, Robotics Auton. Syst..

[53]  F. Freeman,et al.  A Closed-Loop System for Examining Psychophysiological Measures for Adaptive Task Allocation , 2000, The International journal of aviation psychology.

[54]  Siddharth Mayya,et al.  Adaptive Task Allocation for Heterogeneous Multi-Robot Teams with Evolving and Unknown Robot Capabilities , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[55]  Mohamad Khalil,et al.  Driver stress level detection using HRV analysis , 2015, 2015 International Conference on Advances in Biomedical Engineering (ICABME).

[56]  David B. Kaber,et al.  Situation awareness implications of adaptive automation for information processing in an air traffic control-related task , 2006 .

[57]  Nuzhet Atay,et al.  Mixed-Integer Linear Programming Solution to Multi-Robot Task Allocation Problem , 2006 .

[58]  Alejandro R. Mosteo,et al.  Simulated annealing for multi-robot hierarchical task allocation with flexible constraints and objective functions , 2006 .

[59]  Holly A. Yanco,et al.  Classifying human-robot interaction: an updated taxonomy , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[60]  Martin L. Griss,et al.  Activity-Aware Mental Stress Detection Using Physiological Sensors , 2010, MobiCASE.

[61]  Oscar Mayora-Ibarra,et al.  Automatic Stress Detection in Working Environments From Smartphones’ Accelerometer Data: A First Step , 2015, IEEE Journal of Biomedical and Health Informatics.

[62]  T. Dang-Vu,et al.  High-frequency heart rate variability during worry predicts stress-related increases in sleep disturbances. , 2015, Sleep medicine.

[63]  Michael Balzer,et al.  Capacity-Constrained Voronoi Diagrams in Continuous Spaces , 2009, 2009 Sixth International Symposium on Voronoi Diagrams.

[64]  Fadel Adib,et al.  Emotion recognition using wireless signals , 2016, MobiCom.

[65]  Sylvia C. Wong,et al.  Performance Metrics for Robot Coverage Tasks , 2002 .

[66]  Sahar Trigui,et al.  General Background on Multi-robot Task Allocation , 2018 .

[67]  Michael A. Goodrich,et al.  Seven principles of efficient human robot interaction , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[68]  Prasanna Velagapudi,et al.  Teams organization and performance in multi-human/multi-robot teams , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[69]  A. Muaremi,et al.  Towards Measuring Stress with Smartphones and Wearable Devices During Workday and Sleep , 2013, BioNanoScience.

[70]  Inbal Nahum-Shani,et al.  Finding Significant Stress Episodes in a Discontinuous Time Series of Rapidly Varying Mobile Sensor Data , 2016, CHI.