Estimation of Cognitive Workload during Simulated Air Traffic Control Using Optical Brain Imaging Sensors

Deployment of portable neuroimaging technologies to operating settings could help assess cognitive states of personnel assigned to perform critical tasks and thus help improve efficiency and safety of human machine systems. Functional Near Infrared Spectroscopy (fNIR) is an emerging noninvasive brain imaging technology that relies on optical techniques to detect brain hemodynamics within the prefrontal cortex in response to sensory, motor, or cognitive activation. Collaborating with the FAA William J. Hughes Technical Center, fNIR has been used to monitor twenty four certified professional controllers as they manage realistic Air Traffic Control (ATC) scenarios under typical and emergent conditions. We have implemented a normalization procedure to estimate cognitive workload levels from fNIR signals during ATC by developing linear regression models that were informed by the respective participants' prior n-back data. This normalization can account for oxygenation variance due to inter-personal physiological differences. Results indicate that fNIR is sensitive task loads during ATC.

[1]  Hasan Ayaz,et al.  Assessment of Cognitive Neural Correlates for a Functional Near Infrared-Based Brain Computer Interface System , 2009, HCI.

[2]  Hidenao Fukuyama,et al.  The neural basis of executive function in working memory: an fMRI study based on individual differences , 2004, NeuroImage.

[3]  Glenn F. Wilson,et al.  Psychophysiological responses to changes in workload during simulated air traffic control , 1996, Biological Psychology.

[4]  E. Koechlin,et al.  Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[5]  J. Veltman,et al.  Physiological indices of workload in a simulated flight task , 1996, Biological Psychology.

[6]  M. Helander.  Applicability of drivers' electrodermal response to the design of the traffic environment. , 1978, The Journal of applied psychology.

[7]  H. Damasio,et al.  Dissociation Of Working Memory from Decision Making within the Human Prefrontal Cortex , 1998, The Journal of Neuroscience.

[8]  B. Cain.  A Review of the Mental Workload Literature , 2007 .

[9]  George E. Cooper,et al.  The use of pilot rating in the evaluation of aircraft handling qualities , 1969 .

[10]  Vincent G. Duffy.  Digital Human Modeling , 2009, Lecture Notes in Computer Science.

[11]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.

[12]  Thomas B. Sheridan,et al.  Toward the definition and measurement of the mental workload of transport pilots , 1979 .

[13]  Kay M. Stanney,et al.  Augmented Cognition and Cognitive State Assessment Technology - Near-Term, Mid-Term, and Long-Term Research Objectives , 2007, HCI.

[14]  Britton Chance,et al.  Functional Optical Brain Imaging Using Near-Infrared During Cognitive Tasks , 2004, Int. J. Hum. Comput. Interact..

[15]  Glenn F. Wilson,et al.  Putting the Brain to Work: Neuroergonomics Past, Present, and Future , 2008, Hum. Factors.

[16]  S. Bunce,et al.  Functional near-infrared spectroscopy , 2006, IEEE Engineering in Medicine and Biology Magazine.

[17]  Vincent G. Duffy,et al.  A Composite Measure for the Evaluation of Mental Workload , 2007, HCI.

[18]  A. H. Roscoe.  Assessing pilot workload. Why measure heart rate, HRV and respiration? , 1992, Biological Psychology.

[19]  K. Izzetoglu,et al.  Registering fNIR Data to Brain Surface Image using MRI templates , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[20]  E. N. Corlett,et al.  Evaluation of human work : a practical ergonomics methodology , 1991 .

[21]  Waldemar Karwowski,et al.  Advances in Understanding Human Performance : Neuroergonomics, Human Factors Design, and Special Populations , 2010 .

[22]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[23]  J. Grafman,et al.  Human prefrontal cortex: processing and representational perspectives , 2003, Nature Reviews Neuroscience.

[24]  S. Bunce,et al.  Functional near-infrared neuroimaging , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  Adrian M Owen,et al.  Distinct roles for lateral and medial anterior prefrontal cortex in contextual recollection. , 2005, Journal of neurophysiology.

[26]  Ottmar V. Lipp,et al.  Spontaneous and reflexive eye activity measures of mental workload , 2002 .

[27]  Ulf Ahlstrom,et al.  Using eye movement activity as a correlate of cognitive workload , 2006 .

[28]  D. Boas,et al.  Non-invasive neuroimaging using near-infrared light , 2002, Biological Psychiatry.

[29]  Roland Traunmüller.  Information Systems: The e-Business Challenge, IFIP 17th World Computer Congress - TC8 Stream on Information Systems: The e-Business Challenge, August 25-30, 2002, Montréal, Québec, Canada , 2002, Information Systems: The e-Business Challenge.

[30]  W Karwowski,et al.  A Heart Rate Evaluation Approach to Determine Cost-Effectiveness an Ergonomics Intervention , 2001, International journal of occupational safety and ergonomics : JOSE.

[31]  Dylan D. Schmorrow,et al.  Foundations of Augmented Cognition , 2013, Lecture Notes in Computer Science.

[32]  R. Cabeza,et al.  Imaging Cognition II: An Empirical Review of 275 PET and fMRI Studies , 2000, Journal of Cognitive Neuroscience.

[33]  Dylan D. Schmorrow,et al.  DARPA's Augmented Cognition Program-tomorrow's human computer interaction from vision to reality: building cognitively aware computational systems , 2002, Proceedings of the IEEE 7th Conference on Human Factors and Power Plants.

[34]  Hasan Ayaz,et al.  Optical Brain Imaging to Enhance UAV Operator Training, Evaluation, and Interface Development , 2011, J. Intell. Robotic Syst..

[35]  B. Chance,et al.  Cognition-activated low-frequency modulation of light absorption in human brain. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[36]  A. Owen,et al.  Anterior prefrontal cortex: insights into function from anatomy and neuroimaging , 2004, Nature Reviews Neuroscience.

[37]  Hasan Ayaz,et al.  Cognitive Workload Assessment of Air Traffic Controllers Using Optical Brain Imaging Sensors , 2010 .

[38]  Mark H. Chignell,et al.  Mental workload dynamics in adaptive interface design , 1988, IEEE Trans. Syst. Man Cybern..

[39]  Sehchang Hah,et al.  The Effect of Air Traffic Increase on Controller Workload , 2006 .

[40]  Fred G. W. C. Paas,et al.  The Efficiency of Instructional Conditions: An Approach to Combine Mental Effort and Performance Measures , 1992 .

[41]  A. Villringer,et al.  Non-invasive optical spectroscopy and imaging of human brain function , 1997, Trends in Neurosciences.

[42]  A. T. Welford,et al.  Forty years of experimental psychology in relation to age: Retrospect and prospect , 1986, Experimental Gerontology.