Measuring the Domain Specificity of Workload Using EEG: Auditory and Visual Domains in Rotary-Wing Simulated Flight

OBJECTIVE The overarching objective was to evaluate whether workload sensory-domain specificity could be identified through electroencephalogram (EEG) recordings during simulated rotary-wing operations. BACKGROUND Rotary-wing aviators experience workload from different sensory domains, although predominantly through auditory and visual domains. Development of real-time monitoring tools using psychophysiological indices, such as EEG recordings, could enable identification of aviator overload in real time. METHOD Two studies were completed, both of which recorded EEG, task performance, and self-report data. In Study 1, 16 individuals completed a basic auditory and a basic visual laboratory task where workload was manipulated. In Study 2, 23 Army aviators completed simulated aviation flights where workload was manipulated within auditory and visual sensory domains. RESULTS Results from Study 1 found differences in frontal alpha activity during the auditory task, and that alpha and beta activities were associated with perceived workload. Frontal theta activity was found to differ during the visual task while frontal alpha was associated with perceived workload. Study 2 found support for frontal beta activity and the ratio of beta to alpha + theta to differentiate level of workload within the auditory domain. CONCLUSION There is likely a role of frontal alpha and beta activities in response to workload manipulations within the auditory domain; however, this role becomes more equivocal when examined in a multifaceted flight scenario. APPLICATION Results from this study provide a basis for understanding changes in EEG activity when workload is manipulated in sensory domains that can be used in furthering the development of real-time monitoring tools.

[1]  Raul Ramirez,et al.  Using cerebral hemovelocity to measure workload during a spatialised auditory vigilance task in novice and experienced observers , 2013, Ergonomics.

[2]  Kilseop Ryu,et al.  Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic , 2005 .

[3]  Carole R. Beal,et al.  EEG estimates of engagement and cognitive workload predict math problem solving outcomes , 2012, UMAP.

[4]  Jonas Obleser,et al.  Cortical alpha oscillations as a tool for auditory selective inhibition , 2014, Front. Hum. Neurosci..

[5]  Pamela S. Tsang,et al.  Mental Workload and Situation Awareness , 2006 .

[6]  Manuel Schabus,et al.  Fronto-parietal EEG coherence in theta and upper alpha reflect central executive functions of working memory. , 2005, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[7]  M. Just,et al.  Brain Activation Modulated by Sentence Comprehension , 1996, Science.

[8]  Herbert A Colle Mental Workload Manipulation Using Multiple Homogeneous Tasks: Performance Effects , 2010 .

[9]  Christopher D. Wickens,et al.  Situation Awareness and Workload in Aviation , 2002 .

[10]  Tanja Schultz,et al.  Mental workload during n-back task—quantified in the prefrontal cortex using fNIRS , 2014, Front. Hum. Neurosci..

[11]  Hankins Tc,et al.  A comparison of heart rate, eye activity, EEG and subjective measures of pilot mental workload during flight. , 1998, Aviation, space, and environmental medicine.

[12]  W. Klimesch,et al.  Control mechanisms in working memory: A possible function of EEG theta oscillations , 2010, Neuroscience & Biobehavioral Reviews.

[13]  R. VanRullen,et al.  Oscillatory Mechanisms of Stimulus Processing and Selection in the Visual and Auditory Systems: State-of-the-Art, Speculations and Suggestions , 2017, Front. Neurosci..

[14]  Fabio Babiloni,et al.  The Dry Revolution: Evaluation of Three Different EEG Dry Electrode Types in Terms of Signal Spectral Features, Mental States Classification and Usability , 2019, Sensors.

[15]  Rosa H. M. Chan,et al.  An evaluation of mental workload with frontal EEG , 2017, PloS one.

[16]  Nir Giladi,et al.  Increased frontal brain activation during walking while dual tasking: an fNIRS study in healthy young adults , 2014, Journal of NeuroEngineering and Rehabilitation.

[17]  Michael B. McCamy,et al.  Task complexity modulates pilot electroencephalographic activity during real flights. , 2015, Psychophysiology.

[18]  Chris Berka,et al.  Drowsiness/alertness algorithm development and validation using synchronized EEG and cognitive performance to individualize a generalized model , 2011, Biological Psychology.

[19]  Arthur Estrada,et al.  Toward an Operational Definition of Workload: A Workload Assessment of Aviation Maneuvers , 2010 .

[20]  Chris Berka,et al.  Real-Time Analysis of EEG Indexes of Alertness, Cognition, and Memory Acquired With a Wireless EEG Headset , 2004, Int. J. Hum. Comput. Interact..

[21]  A. Kramer,et al.  Event-related potentials as indices of display-monitoring performance , 1995, Biological Psychology.

[22]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[23]  S. Debener,et al.  Look now and hear what's coming: On the functional role of cross-modal phase reset , 2014, Hearing Research.

[24]  R. Romo,et al.  α-Oscillations in the monkey sensorimotor network influence discrimination performance by rhythmical inhibition of neuronal spiking , 2011, Proceedings of the National Academy of Sciences.

[25]  Hasan Ayaz,et al.  Implementation of fNIRS for Monitoring Levels of Expertise and Mental Workload , 2011, HCI.

[26]  Seppo Kähkönen,et al.  Alcohol Reduces Prefrontal Cortical Excitability in Humans: A Combined TMS and EEG Study , 2003, Neuropsychopharmacology.

[27]  Christopher D. Wickens,et al.  The Structure of Attentional Resources , 1980 .

[28]  C. Schroeder,et al.  Low-frequency neuronal oscillations as instruments of sensory selection , 2009, Trends in Neurosciences.

[29]  Fabio Babiloni,et al.  Adaptive Automation Triggered by EEG-Based Mental Workload Index: A Passive Brain-Computer Interface Application in Realistic Air Traffic Control Environment , 2016, Front. Hum. Neurosci..

[30]  Hasan Ayaz,et al.  Multisubject “Learning” for Mental Workload Classification Using Concurrent EEG, fNIRS, and Physiological Measures , 2017, Front. Hum. Neurosci..

[31]  Mickaël Causse,et al.  Neural and psychophysiological correlates of human performance under stress and high mental workload , 2016, Biological Psychology.

[32]  M. Just,et al.  Interdependence of Nonoverlapping Cortical Systems in Dual Cognitive Tasks , 2001, NeuroImage.

[33]  Luc H. Arnal Predicting “When” Using the Motor System’s Beta-Band Oscillations , 2012, Front. Hum. Neurosci..

[34]  G. Karmos,et al.  Entrainment of Neuronal Oscillations as a Mechanism of Attentional Selection , 2008, Science.

[35]  Daniel J. Barber,et al.  The Psychometrics of Mental Workload , 2015, Hum. Factors.

[36]  Fabio Babiloni,et al.  Human Factors and Neurophysiological Metrics in Air Traffic Control: A Critical Review , 2017, IEEE Reviews in Biomedical Engineering.

[37]  Michael E. Smith,et al.  Monitoring Task Loading with Multivariate EEG Measures during Complex Forms of Human-Computer Interaction , 2001, Hum. Factors.

[38]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.

[39]  Lauren Reinerman-Jones,et al.  Metrics for individual differences in EEG response to cognitive workload: Optimizing performance prediction , 2017 .

[40]  S Pozzi,et al.  A passive brain-computer interface application for the mental workload assessment on professional air traffic controllers during realistic air traffic control tasks. , 2016, Progress in brain research.

[41]  Caroline Dussault,et al.  EEG and ECG changes during selected flight sequences. , 2004, Aviation, space, and environmental medicine.

[42]  B. Hangya,et al.  Phase Entrainment of Human Delta Oscillations Can Mediate the Effects of Expectation on Reaction Speed , 2010, The Journal of Neuroscience.

[43]  Daniel A. Braun,et al.  Assessing randomness and complexity in human motion trajectories through analysis of symbolic sequences , 2014, Front. Hum. Neurosci..

[44]  Peter Nickel,et al.  Sensitivity of candidate markers of psychophysiological strain to cyclical changes in manual control load during simulated process control. , 2009, Applied ergonomics.

[45]  W. Klimesch EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[46]  Maarten A. Hogervorst,et al.  Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload , 2014, Front. Neurosci..

[47]  F. Freeman,et al.  Evaluation of a Psychophysiologically Controlled Adaptive Automation System, Using Performance on a Tracking Task , 2000, Applied psychophysiology and biofeedback.

[48]  Michael Schrauf,et al.  EEG Alpha Spindles as Indicators for Prolonged Brake Reaction Time During Auditory Secondary Tasks in a Real Road Driving Study , 2011 .

[49]  D. Gilbert,et al.  Effects of nicotine and caffeine, separately and in combination, on EEG topography, mood, heart rate, cortisol, and vigilance. , 2000, Psychophysiology.

[50]  R. VanRullen,et al.  An oscillatory mechanism for prioritizing salient unattended stimuli , 2012, Trends in Cognitive Sciences.

[51]  Troy D. Kelley,et al.  Towards the Shape of Mental Workload , 2006 .

[52]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[53]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.

[54]  F. Herrmann,et al.  Assessment of mental workload: A new electrophysiological method based on intra-block averaging of ERP amplitudes , 2016, Neuropsychologia.

[55]  Benjamin Brooks,et al.  Measuring mental workload and physiological reactions in marine pilots: Building bridges towards redlines of performance. , 2018, Applied ergonomics.

[56]  M. Just,et al.  From the Selectedworks of Marcel Adam Just the Organization of Thinking: What Functional Brain Imaging Reveals about the Neuroarchitecture of Complex Cognition , 2022 .