Evaluating Mental State of Drivers in Automated Driving Using Heart Rate Variability towards Feasible Request-to-Intervene

As Intelligent Transport Systems (ITS) advances, more and more people will have the opportunities to drive vehicles with autonomous capabilities. This rise in number of semi-autonomous vehicles also gives rise to several challenges with regards on how human factors come into play in interacting with the vehicle’s Automated Driving System (ADS). One important interaction of an ADS with Level 3 Conditional Automated Driving capabilities is Request-to-Intervene (RTI), which alerts drivers to takeover the vehicle during an automated driving session, however, the driver is not necessarily ready to receive the authority. To see whether an ADS can detect the readiness of the user for RTI, in this preliminary study we evaluated the mental states of ADS users in naturalistic driving conditions by comparing them with those of drivers and passengers. The mental states were evaluated by measuring their heart rate and by calculating specific features of Heart Rate Variability (HRV), specifically NN50 and pNN50 indices, during driving events (turning, lane changing, and stopping) and no-events. The results showed the NN50 and pNN50 values of manual driving were significantly different from those of ADS driving and passenger, suggesting that ADS driving has a higher level of relaxed state. In addition, events such as lane-changing in the ADS driving did not induce significantly different NN50 and pNN50 from nonevent situation, which may imply the participants did not pay attention to such events.

[1]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[2]  Stefan Feuerriegel,et al.  Improving heart rate variability measurements from consumer smartwatches with machine learning , 2019, UbiComp/ISWC Adjunct.

[3]  Robert Karasek,et al.  The perception of work stressors is related to reduced parasympathetic activity , 2011, International archives of occupational and environmental health.

[4]  Koichi Fujiwara,et al.  Real-time heart rate variability monitoring employing a wearable telemeter and a smartphone , 2014, Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific.

[5]  Peter B. Nagy,et al.  HRV-based Stress Level Assessment Using Very Short Recordings , 2017, Period. Polytech. Electr. Eng. Comput. Sci..

[6]  D. Bai,et al.  Stress and Heart Rate Variability: A Meta-Analysis and Review of the Literature , 2018, Psychiatry investigation.

[7]  Stephan Getzmann,et al.  Evaluating Pro- and Re-Active Driving Behavior by Means of the EEG , 2018, Front. Hum. Neurosci..

[8]  Omid Dehzangi,et al.  Wearable Driver Distraction Identification On-The-Road via Continuous Decomposition of Galvanic Skin Responses , 2018, Sensors.

[9]  Rubén Usamentiaga,et al.  Driver Distraction Using Visual-Based Sensors and Algorithms , 2016, Sensors.

[10]  Dick de Waard,et al.  Behavioural impacts of Advanced Driver Assistance Systems–an overview , 2019, European Journal of Transport and Infrastructure Research.

[11]  Oscal T.-C. Chen,et al.  Real-time physiological and facial monitoring for safe driving , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[12]  Paolo Napoletano,et al.  Combining heart and breathing rate for car driver stress recognition , 2018, 2018 IEEE 8th International Conference on Consumer Electronics - Berlin (ICCE-Berlin).

[13]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[14]  Susanne Boll,et al.  Assisting Drivers with Ambient Take-Over Requests in Highly Automated Driving , 2016, AutomotiveUI.

[15]  A. Boyd.  The United States department of transportation , 1968 .

[16]  Santokh Singh,et al.  Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey , 2015 .

[17]  Klaus Bengler,et al.  Evaluation of a Contact Analog Head-Up Display for Highly Automated Driving , 2012 .

[18]  Toshihiro Hiraoka,et al.  Development of Drowsiness Detection Method by Integrating Heart Rate Variability Analysis and Multivariate Statistical Process Control , 2016 .

[19]  Marilyn A. Uy,et al.  The Body and the Brain: Measuring Skin Conductance Responses to Understand the Emotional Experience , 2019 .

[20]  Mohamad Khalil,et al.  Driver stress level detection using HRV analysis , 2015, 2015 International Conference on Advances in Biomedical Engineering (ICABME).

[21]  E. Petridou,et al.  Human factors in the causation of road traffic crashes , 2004, European Journal of Epidemiology.

[22]  S. Lewandowsky,et al.  A connectionist model of complacency and adaptive recovery under automation. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[23]  Thomas A. Ranney,et al.  The effects of in-vehicle distraction on driver response during a crucial driving maneuver , 1999 .

[24]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[25]  D Ruscio,et al.  Distraction or cognitive overload? Using modulations of the autonomic nervous system to discriminate the possible negative effects of advanced assistance system. , 2017, Accident; analysis and prevention.

[26]  Fokie Cnossen,et al.  Heart rate variability as a measure of mental stress in surgery: a systematic review , 2020, International Archives of Occupational and Environmental Health.

[27]  Rossana Castaldo,et al.  Ultra-short term HRV features as surrogates of short term HRV: a case study on mental stress detection in real life , 2019, BMC Medical Informatics and Decision Making.

[28]  Hirofumi Aoki,et al.  A Novel Method for Classifying Driver Mental Workload Under Naturalistic Conditions With Information From Near-Infrared Spectroscopy , 2018, Front. Hum. Neurosci..

[29]  Oren Musicant,et al.  Associating Vehicles Automation With Drivers Functional State Assessment Systems: A Challenge for Road Safety in the Future , 2019, Front. Hum. Neurosci..