A Machine Learning Approach to the Detection of Pilot's Reaction to Unexpected Events Based on EEG Signals

This work considers the problem of utilizing electroencephalographic signals for use in systems designed for monitoring and enhancing the performance of aircraft pilots. Systems with such capabilities are generally referred to as cognitive cockpits. This article provides a description of the potential that is carried by such systems, especially in terms of increasing flight safety. Additionally, a neuropsychological background of the problem is presented. Conducted research was focused mainly on the problem of discrimination between states of brain activity related to idle but focused anticipation of visual cue and reaction to it. Especially, a problem of selecting a proper classification algorithm for such problems is being examined. For that purpose an experiment involving 10 subjects was planned and conducted. Experimental electroencephalographic data was acquired using an Emotiv EPOC+ headset. Proposed methodology involved use of a popular method in biomedical signal processing, the Common Spatial Pattern, extraction of bandpower features, and an extensive test of different classification algorithms, such as Linear Discriminant Analysis, k-nearest neighbors, and Support Vector Machines with linear and radial basis function kernels, Random Forests, and Artificial Neural Networks.

[1]  Martin Grunwald,et al.  Power of theta waves in the EEG of human subjects increases during recall of haptic information , 1999, Neuroscience Letters.

[2]  Douglas D. Noble Cockpit cognition: Education, the military and cognitive engineering , 1989, AI & SOCIETY.

[3]  Jacques Felblinger,et al.  Automated cortical projection of EEG sensors: Anatomical correlation via the international 10–10 system , 2009, NeuroImage.

[4]  Brian C. Ross Mutual Information between Discrete and Continuous Data Sets , 2014, PloS one.

[5]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[6]  Steve Winder Introduction to digital filters , 2002 .

[7]  Benjamin Blankertz,et al.  Towards a Cure for BCI Illiteracy , 2009, Brain Topography.

[8]  S. K. Setarehdan,et al.  Development of a robust method for an online P300 Speller Brain Computer Interface , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[9]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  M. Endsley Automation and situation awareness. , 1996 .

[11]  Brendan Z. Allison,et al.  Could Anyone Use a BCI? , 2010, Brain-Computer Interfaces.

[12]  Olivier Ledoit,et al.  Honey, I Shrunk the Sample Covariance Matrix , 2003 .

[13]  René van Paassen,et al.  Dealing With Unexpected Events on the Flight Deck: A Conceptual Model of Startle and Surprise , 2017, Hum. Factors.

[14]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[15]  Karel Hurts,et al.  Automation Surprise: Results of a Field Survey of Dutch Pilots , 2017 .

[16]  Mark S. Young,et al.  Malleable Attentional Resources Theory: A New Explanation for the Effects of Mental Underload on Performance , 2002, Hum. Factors.

[17]  J. Fermaglich Electric Fields of the Brain: The Neurophysics of EEG , 1982 .

[18]  Andreas Haslbeck,et al.  Flying the Needles , 2016, Hum. Factors.

[19]  Tyler S. Grummett,et al.  Measurement of neural signals from inexpensive, wireless and dry EEG systems , 2015, Physiological measurement.

[20]  A. Craig,et al.  Regional brain wave activity changes associated with fatigue. , 2012, Psychophysiology.

[21]  F. H. Lopes da Silva Neural mechanisms underlying brain waves: from neural membranes to networks. , 1991, Electroencephalography and clinical neurophysiology.

[22]  R. Hutton,et al.  Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands. , 1998, Ergonomics.

[23]  David B. Kaber,et al.  The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task , 2004 .

[24]  Michal Niezabitowski,et al.  Normalization of feature distribution in motor imagery based brain-computer interfaces , 2016, 2016 24th Mediterranean Conference on Control and Automation (MED).

[25]  Klaus-Robert Müller,et al.  Neurophysiological predictor of SMR-based BCI performance , 2010, NeuroImage.

[26]  Michal Niezabitowski,et al.  Elimination of bioelectrical source overlapping effects from the EEG measurements , 2016, 2016 17th International Carpathian Control Conference (ICCC).

[27]  R M Taylor,et al.  COGNITIVE COCKPIT SYSTEMS: INFORMATION REQUIREMENTS ANALYSIS FOR PILOT CONTROL OF COCKPIT AUTOMATION. IN: ENGINEERING PSYCHOLOGY AND COGNITIVE ERGONOMICS. AEROSPACE AND TRANSPORTATION SYSTEMS , 2001 .