Analysing temporal performance profiles of UAV operators using time series clustering

The continuing growth in the use of Unmanned Aerial Vehicles (UAVs) is causing an important social step forward in the performance of many sensitive tasks, reducing both human and economical risks. The work of UAV operators is a key aspect to guarantee the success of this kind of tasks, and thus UAV operations are studied in many research fields, ranging from human factors to data analysis and machine learning. The present work aims to describe the behaviour of operators over time using a profile-based model where the evolution of the operator performance during a mission is the main unit of measure. In order to compare how different operators act throughout a mission, we describe a methodology based of multivariate-time series clustering to define and analyse a set of representative temporal performance profiles. The proposed methodology is applied in a multi-UAV simulation environment with inexperienced operators, obtaining a fair description of the temporal behavioural patterns followed during the course of the simulation.

[1]  Jean Scholtz,et al.  Awareness in human-robot interactions , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[2]  M. Kendall,et al.  The Problem of $m$ Rankings , 1939 .

[3]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[4]  Nancy J. Cooke,et al.  20. Acquiring Team-Level Command and Control Skill for UAV Operation , 2006 .

[5]  J. Bartko.  Corrective Note to: “The Intraclass Correlation Coefficient as a Measure of Reliability” , 1974 .

[6]  Ricardo Bencatel,et al.  Unmanned Air Vehicles for coastal and environmental research , 2009 .

[7]  Mary L. Cummings,et al.  Predictive models of human supervisory control behavioral patterns using hidden semi-Markov models , 2011, Eng. Appl. Artif. Intell..

[8]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[9]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[10]  Rachid Alami,et al.  A distributed tasks allocation scheme in multi-UAV context , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[11]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1990 .

[12]  Michalis Vazirgiannis,et al.  On Clustering Validation Techniques , 2001, Journal of Intelligent Information Systems.

[13]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[14]  T. Caliński,et al.  A dendrite method for cluster analysis , 1974 .

[15]  Peter J. Rousseeuw,et al.  Clustering by means of medoids , 1987 .

[16]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[17]  David Camacho,et al.  Automatic profile generation for UAV operators using a simulation-based training environment , 2015, Progress in Artificial Intelligence.

[18]  H. Mannila,et al.  Computing Discrete Fréchet Distance ∗ , 1994 .

[19]  Pablo Montero,et al.  TSclust: An R Package for Time Series Clustering , 2014 .

[20]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[21]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[22]  Guy N. Brock,et al.  clValid , an R package for cluster validation , 2008 .

[23]  Yuanyuan Zhang,et al.  Clustering Mobile Apps Based on Mined Textual Features , 2016, ESEM.

[24]  Jure Leskovec,et al.  Patterns of temporal variation in online media , 2011, WSDM '11.

[25]  L. Hubert,et al.  Measuring the Power of Hierarchical Cluster Analysis , 1975 .

[26]  Saiful Islam,et al.  Mahalanobis Distance , 2009, Encyclopedia of Biometrics.

[27]  Patrick Paroubek,et al.  Twitter as a Corpus for Sentiment Analysis and Opinion Mining , 2010, LREC.

[28]  Gian Luca Foresti,et al.  Trajectory-Based Anomalous Event Detection , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  David Camacho,et al.  Combining Time Series and Clustering to Extract Gamer Profile Evolution , 2014, ICCCI.

[30]  Hilde Remøy,et al.  Out of Office: A Study on the Cause of Office Vacancy and Transformation as a Means to Cope and Prevent , 2010 .

[31]  R Andy McKinley,et al.  Operator selection for unmanned aerial systems: comparing video game players and pilots. , 2011, Aviation, space, and environmental medicine.

[32]  C. Hennig,et al.  How to find an appropriate clustering for mixed‐type variables with application to socio‐economic stratification , 2013 .

[33]  David Camacho,et al.  Extracting behavioural models from 2010 FIFA world cup , 2013, J. Syst. Sci. Complex..

[34]  Víctor Rodríguez-Fernández,et al.  Design and development of a lightweight multi-UAV simulator , 2015, 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF).

[35]  Alex Alves Freitas,et al.  A Survey of Evolutionary Algorithms for Clustering , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[36]  Carmela Iorio,et al.  Parsimonious time series clustering using P-splines , 2016, Expert Syst. Appl..

[37]  L. Penrose.  Distance, size and shape. , 1954, Annals of eugenics.

[38]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[39]  S. White,et al.  A Universal Density Profile from Hierarchical Clustering , 1996, astro-ph/9611107.

[40]  Leonid Portnoy,et al.  Intrusion detection with unlabeled data using clustering , 2000 .

[41]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[42]  Susmita Datta,et al.  Comparisons and validation of statistical clustering techniques for microarray gene expression data , 2003, Bioinform..

[43]  David Camacho,et al.  User Profile Analysis for UAV Operators in a Simulation Environment , 2015, ICCCI.

[44]  Nada Lavrac,et al.  Selected techniques for data mining in medicine , 1999, Artif. Intell. Medicine.

[45]  André Hardy,et al.  An examination of procedures for determining the number of clusters in a data set , 1994 .

[46]  Satu Elisa Schaeffer,et al.  Graph Clustering , 2017, Encyclopedia of Machine Learning and Data Mining.

[47]  Antonio González-Pardo,et al.  Modeling the Behavior of Unskilled Users in a Multi-UAV Simulation Environment , 2015, IDEAL.

[48]  T. Warren Liao,et al.  Clustering of time series data - a survey , 2005, Pattern Recognit..

[49]  Wendy R. Fox,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1991 .

[50]  Charu C. Aggarwal,et al.  Graph Clustering , 2010, Encyclopedia of Machine Learning and Data Mining.