Learning-by-Doing: Using Near Infrared Spectroscopy to Detect Habituation and Adaptation in Automated Driving

The advent of automated features in modern vehicles requires human factors researchers to find measures other than driving behavior to anticipate the response of drivers in various contexts. Functional near-infrared spectroscopy (fNIRS) is one research tool that allows us to quantify the driver's mental state. However, the underlying mechanisms of fNIRS technology can limit the possible contexts for its application. The pervasive question arises, whether the measurement device at hand is suitable for the research topic in question and is it capable of detecting the phenomenon under investigation? We provide a proof of concept study demonstrating that significant habituation is present when drivers operate new automated driving systems and that fNIRS technology is suitable to detect said driver habituation effects. The study presented here was conducted in a driving simulator and investigated the drivers' cortical activation in three different modes of automation: manual, partially autonomous, and fully autonomous modes.

[1]  T. Sejnowski,et al.  Estimating alertness from the EEG power spectrum , 1997, IEEE Transactions on Biomedical Engineering.

[2]  Allison M. Okamura,et al.  Motor learning affects car-to-driver handover in automated vehicles , 2016, Science Robotics.

[3]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[4]  Isao Nambu,et al.  Evaluation of motor learning by fNIRS signal analysis using a general linear model , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[5]  D. Boas,et al.  HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain. , 2009, Applied optics.

[6]  Gerhard Tröster,et al.  Discriminating Stress From Cognitive Load Using a Wearable EDA Device , 2010, IEEE Transactions on Information Technology in Biomedicine.

[7]  Wendy Ju,et al.  Tunneled In: Drivers with Active Secondary Tasks Need More Time to Transition from Automation , 2017, CHI.

[8]  Wendy Ju,et al.  Monitoring driver cognitive load using functional near infrared spectroscopy in partially autonomous cars , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[9]  R. Parasuraman,et al.  Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development , 2013, Front. Hum. Neurosci..

[10]  Yoshitaka Marumo,et al.  Measurement of Frontal Cortex Brain Activity Attributable to the Driving Workload and Increased Attention , 2009 .

[11]  Pavlo D. Antonenko,et al.  Using Electroencephalography to Measure Cognitive Load , 2010 .

[12]  Bryan Reimer,et al.  Classifying driver workload using physiological and driving performance data: two field studies , 2014, CHI.

[13]  G. McArthur,et al.  Validation of the Emotiv EPOC® EEG gaming system for measuring research quality auditory ERPs , 2013, PeerJ.

[14]  J. Grafman,et al.  Dorsolateral prefrontal contributions to human working memory , 2013, Cortex.

[15]  Vince D. Calhoun,et al.  Group Study of Simulated Driving fMRI Data by Multiset Canonical Correlation Analysis , 2009, NeuroImage.

[16]  Ning Liu,et al.  Corrigendum: Sex differences in neural and behavioral signatures of cooperation revealed by fNIRS hyperscanning , 2016, Scientific reports.

[17]  Toshinori Kato,et al.  Functional brain imaging using near-infrared spectroscopy during actual driving on an expressway , 2013, Front. Hum. Neurosci..

[18]  A. Reiss,et al.  Sex differences in neural and behavioral signatures of cooperation revealed by fNIRS hyperscanning , 2016, Scientific reports.

[19]  Mark D'Esposito,et al.  Rapid Prefrontal-Hippocampal Habituation to Novel Events , 2004, The Journal of Neuroscience.

[20]  Raja Parasuraman,et al.  Into the Wild: Neuroergonomic Differentiation of Hand-Held and Augmented Reality Wearable Displays during Outdoor Navigation with Functional Near Infrared Spectroscopy , 2016, Front. Hum. Neurosci..

[21]  A. Owen,et al.  Anterior prefrontal cortex: insights into function from anatomy and neuroimaging , 2004, Nature Reviews Neuroscience.

[22]  Robert J. K. Jacob,et al.  From Brain Signals to Adaptive Interfaces: Using fNIRS in HCI , 2010, Brain-Computer Interfaces.

[23]  C. Vaidya,et al.  Sensitivity of fNIRS to cognitive state and load , 2014, Front. Hum. Neurosci..

[24]  Hitoshi Tsunashima,et al.  Measurement of Brain Function of Car Driver Using Functional Near-Infrared Spectroscopy (fNIRS) , 2009, Comput. Intell. Neurosci..

[25]  Wendy Ju,et al.  Distraction Becomes Engagement in Automated Driving , 2015 .

[26]  J Aasman,et al.  Operator Effort and the Measurement of Heart-Rate Variability , 1987, Human factors.

[27]  R. Engle,et al.  The role of prefrontal cortex in working-memory capacity, executive attention, and general fluid intelligence: An individual-differences perspective , 2002, Psychonomic bulletin & review.

[28]  Takahiro Imai,et al.  Estimating Brain Activity of Motor Learning by Using fNIRS-GLM Analysis , 2012, ICONIP.

[29]  J. Tanaka,et al.  Workload of using a driver assistance system , 2000, ITSC2000. 2000 IEEE Intelligent Transportation Systems. Proceedings (Cat. No.00TH8493).

[30]  Cécile Issard,et al.  Adult-like processing of time-compressed speech by newborns: A NIRS study , 2016, Developmental Cognitive Neuroscience.

[31]  Robert J. K. Jacob,et al.  Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines , 2009, UIST '09.

[32]  Stephane Champely,et al.  Assessing Workload through Physiological Measurements in Bus Drivers Using an Automated System during Docking , 2003, Hum. Factors.

[33]  Hasan Ayaz,et al.  Implementation of fNIRS for Monitoring Levels of Expertise and Mental Workload , 2011, HCI.

[34]  M. Bouton.  Learning and Behavior: A Contemporary Synthesis , 2006 .

[35]  Robert J. K. Jacob,et al.  This is your brain on interfaces: enhancing usability testing with functional near-infrared spectroscopy , 2011, CHI.