Evaluation of Physiologically-Based Artificial Neural Network Models to Detect Operator Workload in Remotely Piloted Aircraft Operations

Abstract : The current research focuses on preventing performance decrements associated with mental overload during remotely piloted aircraft (RPA) operations. This can be accomplished using physiological signals to sense moments of high cognitive workload and providing augmentation to reduce workload and improve performance. Two RPA operators were interviewed to identify factors that impact workload in RPA, surveillance and target tracking missions. Performance, subjective workload, cortical, cardiac, respiration, voice stress, and ocular data were collected. Several physiological measures were sensitive to changes in workload as evidenced by performance and subjective workload data. In addition, several real-time workload models were evaluated. Potential future applications of this research include closed loop systems that employ advanced augmentation strategies, such as adaptive automation. By identifying physiological measures well suited for monitoring workload in a realistic simulation, this research advances the literature toward real-time workload mitigation in RPA field operations.

[1]  G. Wilson.  Applied use of cardiac and respiration measures: Practical considerations and precautions , 1992, Biological Psychology.

[2]  Saccade Detection Using Polar Coordinates – a New Algorithm , 2015 .

[3]  W. Suess,et al.  The effects of psychological stress on respiration: a preliminary study of anxiety and hyperventilation. , 1980, Psychophysiology.

[4]  Christian Kothe,et al.  Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general , 2011, Journal of neural engineering.

[5]  A. H. Roscoe.  Assessing pilot workload. Why measure heart rate, HRV and respiration? , 1992, Biological Psychology.

[6]  Christopher D. Wickens,et al.  Unmanned Aerial Vehicle Flight Control: False Alarms versus Misses , 2004 .

[7]  Glenn F. Wilson,et al.  Performance Enhancement in an Uninhabited Air Vehicle Task Using Psychophysiologically Determined Adaptive Aiding , 2007, Hum. Factors.

[8]  Mark W. Scerbo,et al.  Theoretical Perspectives on Adaptive Automation , 2019, Human Performance in Automated and Autonomous Systems.

[9]  J. Veltman,et al.  Physiological workload reactions to increasing levels of task difficulty. , 1998, Ergonomics.

[10]  Stephen W. Porges,et al.  Research methods for measurement of heart rate and respiration , 1992, Biological Psychology.

[11]  M. Pomplun,et al.  Pupil Dilation as an Indicator of Cognitive Workload in Human-Computer Interaction , 2003 .

[12]  J. Stern,et al.  Eye movements and blinks: their relationship to higher cognitive processes. , 1989, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[13]  Mark S. Young,et al.  Attention and automation: New perspectives on mental underload and performance , 2002 .

[14]  Scott M. Galster,et al.  Physiological Indicators of Workload in a Remotely Piloted Aircraft Simulation , 2015 .

[15]  Scott M. Galster,et al.  The Electrooculogram and a New Blink Detection Algorithm , 2015 .

[16]  Scott M. Galster,et al.  Sense-Assess-Augment: A Taxonomy for Human Effectiveness , 2013 .

[17]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[18]  William R. Howse,et al.  Unmanned Aircraft Systems Selection Practices: Current Research and Future Directions , 2013 .

[19]  J. Fox,et al.  A Correction , 1911, Bio/Technology.

[20]  Glenn F. Wilson,et al.  An Analysis of Mental Workload in Pilots During Flight Using Multiple Psychophysiological Measures , 2002 .

[21]  James C. Christensen,et al.  The effects of day-to-day variability of physiological data on operator functional state classification , 2012, NeuroImage.

[22]  P. Berg,et al.  The correction of ocular artifacts: a topographic perspective , 2000, Clinical Neurophysiology.

[23]  Michael E. Smith,et al.  Monitoring Working Memory Load during Computer-Based Tasks with EEG Pattern Recognition Methods , 1998, Hum. Factors.

[24]  Paul Grossman,et al.  Respiratory and cardiac rhythms as windows to central and autonomic biobehavioral regulation: Selection of window frames, keeping the panes clean and viewing the neural topography , 1992, Biological Psychology.

[25]  Mustapha Mouloua,et al.  Automation and Human Performance : Theory and Applications , 1996 .

[26]  Thomas E. Nygren,et al.  Psychometric Properties of Subjective Workload Measurement Techniques: Implications for Their Use in the Assessment of Perceived Mental Workload , 1991 .

[27]  H. A. Witkin,et al.  The effects of stress on components of the respiration cycle. , 1975, Psychophysiology.

[28]  J. Lance,et al.  Physiological tremor. , 1968, Lancet.

[29]  E T Doherty,et al.  Speech measures indicating workload demand. , 1994, Aviation, space, and environmental medicine.

[30]  Scott M. Galster,et al.  EEG Data Analysis Using Artifact Separation , 2015 .

[31]  Kevin Durkee,et al.  Using Context to Optimize a Functional State Estimation Engine in Unmanned Aircraft System Operations , 2015, HCI.

[32]  P. Jorna.  Spectral analysis of heart rate and psychological state: A review of its validity as a workload index , 1992, Biological Psychology.

[33]  Mustapha Mouloua,et al.  Effects of Adaptive Task Allocation on Monitoring of Automated Systems , 1996, Hum. Factors.

[34]  Kevin Durkee,et al.  Real-Time Workload Assessment as a Foundation for Human Performance Augmentation , 2013, HCI.

[35]  Arnaud Delorme,et al.  Frontal midline EEG dynamics during working memory , 2005, NeuroImage.

[36]  L. Mulder.  Measurement and analysis methods of heart rate and respiration for use in applied environments , 1992, Biological Psychology.

[37]  Ernst Fernando Lopes Da Silva Niedermeyer,et al.  Electroencephalography, basic principles, clinical applications, and related fields , 1982 .

[38]  Nigel H. Lovell,et al.  Characterizing mental load in an arithmetic task using entropy-based features , 2012, 2012 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA).

[39]  J. Beatty.  Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982, Psychological bulletin.

[40]  Susan G. Hill,et al.  Application and Validation of Workload Assessment Techniques , 1993 .

[41]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[42]  K. Hendy,et al.  Measuring Subjective Workload: When Is One Scale Better Than Many? , 1993 .

[43]  R. Ward,et al.  EMG and EOG artifacts in brain computer interface systems: A survey , 2007, Clinical Neurophysiology.

[44]  H. Jasper.  Report of the committee on methods of clinical examination in electroencephalography , 1958 .

[45]  Matthew S. Middendorf,et al.  Maintaining Subject Motivation in Long-Term Experiments Using Performance Incentives and Penalties , 1992 .