Press Start to Play: Classifying Multi-Robot Operators and Predicting Their Strategies through a Videogame

One of the active challenges in multi-robot missions is related to managing operator workload and situational awareness. Currently, the operators are trained to use interfaces, but in the near future this can be turned inside out: the interfaces will adapt to operators so as to facilitate their tasks. To this end, the interfaces should manage models of operators and adapt the information to their states and preferences. This work proposes a videogame-based approach to classify operator behavior and predict their actions in order to improve teleoperated multi-robot missions. First, groups of operators are generated according to their strategies by means of clustering algorithms. Second, the operators’ strategies are predicted, taking into account their models. Multiple information sources and modeling methods are used to determine the approach that maximizes the mission goal. The results demonstrate that predictions based on previous data from single operators increase the probability of success in teleoperated multi-robot missions by 19%, whereas predictions based on operator clusters increase this probability of success by 28%.

[1]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[2]  Jacob W. Crandall,et al.  Predicting Operator Capacity for Supervisory Control of Multiple UAVs , 2007, Innovations in Intelligent Machines.

[3]  Víctor Rodríguez-Fernández,et al.  A study on performance metrics and clustering methods for analyzing behavior in UAV operations , 2017, J. Intell. Fuzzy Syst..

[4]  Mark D. Griffiths,et al.  Online Data Collection from Video Game Players: Methodological Issues , 2004, Cyberpsychology Behav. Soc. Netw..

[5]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[6]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[7]  Miguel A. Olivares-Méndez,et al.  Analyzing and improving multi-robot missions by using process mining , 2018, Auton. Robots.

[8]  Dirk Ifenthaler,et al.  Serious games analytics : methodologies for performance measurement, assessment, and improvement , 2015 .

[9]  D. Botstein,et al.  Cluster analysis and display of genome-wide expression patterns. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[11]  Miguel A. Olivares-Méndez,et al.  Multi-Robot Interfaces and Operator Situational Awareness: Study of the Impact of Immersion and Prediction , 2017, Sensors.

[12]  Shauna J. Sweet,et al.  Analysis of Multivariate Social Science Data (2nd ed.) , 2011 .

[13]  Robin R. Murphy,et al.  Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[14]  Mike Thomas,et al.  Cluster analysis and clinical asthma phenotypes. , 2008, American journal of respiratory and critical care medicine.

[15]  Martial Van der Linden,et al.  Problematic involvement in online games: A cluster analytic approach , 2015, Comput. Hum. Behav..

[16]  T.W.J. Van Ruijven Serious games as experiments for emergency management research: A review , 2011, ISCRAM.

[17]  James C. Bezdek,et al.  Partially supervised clustering for image segmentation , 1996, Pattern Recognit..

[18]  MengChu Zhou,et al.  Optimizing Operator–Agent Interaction in Intelligent Adaptive Interface Design: A Conceptual Framework , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[19]  Gerald Seet,et al.  Workload, awareness and automation in multiple-robot supervision , 2017 .

[20]  K. Bailey Typologies and taxonomies: An introduction to classification techniques. , 1994 .

[21]  A. O. Dick,et al.  Operator Workload: Comprehensive Review and Evaluation of Operator Workload Methodologies , 1989 .

[22]  David Camacho,et al.  Analysing temporal performance profiles of UAV operators using time series clustering , 2016, Expert Syst. Appl..

[23]  Chang S Nam,et al.  A meta-analysis of human-system interfaces in unmanned aerial vehicle (UAV) swarm management. , 2017, Applied ergonomics.

[24]  Holly A. Yanco,et al.  Analysis of Human‐robot Interaction at the DARPA Robotics Challenge Trials , 2015, J. Field Robotics.

[25]  Antonio Barrientos,et al.  Bringing Adaptive and Immersive Interfaces to Real-World Multi-Robot Scenarios: Application to Surveillance and Intervention in Infrastructures , 2019, IEEE Access.

[26]  H. Goor,et al.  Saving robots improves laparoscopic performance: transfer of skills from a serious game to a virtual reality simulator , 2018, Surgical Endoscopy.

[27]  David Camacho,et al.  Automatic profile generation for UAV operators using a simulation-based training environment , 2015, Progress in Artificial Intelligence.

[28]  David Camacho,et al.  A multi-UAV mission planning videogame-based framework for player analysis , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[29]  Johan Hagelbäck,et al.  Physiological Affect and Performance in a Collaborative Serious Game Between Humans and an Autonomous Robot , 2018, ICEC.

[30]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  T. Caliński,et al.  A dendrite method for cluster analysis , 1974 .

[32]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[33]  N. Moray Mental workload : its theory and measurement , 1979 .

[34]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[35]  Zahir Tari,et al.  A Survey of Clustering Algorithms for Big Data: Taxonomy and Empirical Analysis , 2014, IEEE Transactions on Emerging Topics in Computing.

[36]  Mica R. Endsley,et al.  Design and Evaluation for Situation Awareness Enhancement , 1988 .

[37]  Roger K. Blashfield,et al.  Mixture model tests of cluster analysis: Accuracy of four agglomerative hierarchical methods. , 1976 .