Detecting and influencing driver emotions using psycho- physiological sensors and ambient light

Driving is a sensitive task that is strongly affected by the driver’s emotions. Negative emotions, such as anger, can evidently lead to more driving errors. In this work, we introduce a concept of detecting and influencing driver emotions using psycho-physiological sensing for emotion classification and ambient light for feedback. We detect arousal and valence of emotional responses from wearable bio-electric sensors, namely brain-computer interfaces and heart rate sensors. We evaluated our concept in a static driving simulator with a fully equipped car with 12 participants. Before the rides, we elicit negative emotions and evaluate driving performance and physiological data while driving under stressful conditions. We use three ambient lighting conditions (no light, blue, orange). Using a subject-dependent random forests classifier with 40 features collected from physiological data we achieve an average accuracy of 78.9% for classifying valence and 68.7% for arousal. Driving performance was enhanced in conditions where ambient lighting was introduced. Both blue and orange light helped drivers to improve lane keeping. We discuss insights from our study and provide design recommendations for designing emotion sensing and feedback systems in the car.

[1]  Lee Skrypchuk,et al.  Developing a Body Sensor Network to Detect Emotions During Driving , 2014, IEEE Transactions on Intelligent Transportation Systems.

[2]  D. O. Bos,et al.  EEG-based Emotion Recognition The Influence of Visual and Auditory Stimuli , 2007 .

[3]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[4]  Joris C Verster,et al.  Standard operation procedures for conducting the on-the-road driving test, and measurement of the standard deviation of lateral position (SDLP) , 2011, International journal of general medicine.

[5]  Ignacio Alvarez,et al.  Driver in the Loop: Best Practices in Automotive Sensing and Feedback Mechanisms , 2017 .

[6]  Bruce N. Walker,et al.  Comparing heart rate and pupil size as objective measures of workload in the driving context: initial look , 2015, AutomotiveUI.

[7]  Susanne Boll,et al.  "Should I stay or should I go?": Different designs to support drivers' decision making , 2014, NordiCHI.

[8]  Alois Ferscha,et al.  Driver Activity Recognition from Sitting Postures , 2007, Mensch & Computer Workshopband.

[9]  Ioannis Arapakis,et al.  Theories, methods and current research on emotions in library and information science, information retrieval and human-computer interaction , 2011, Inf. Process. Manag..

[10]  Bryan Reimer,et al.  Classifying driver workload using physiological and driving performance data: two field studies , 2014, CHI.

[11]  Ing-Marie Jonsson,et al.  Automatic recognition of affective cues in the speech of car drivers to allow appropriate responses , 2005, OZCHI.

[12]  Qiang Ji,et al.  Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance , 2002, Real Time Imaging.

[13]  Florian Alt,et al.  Improving Driver Emotions with Affective Strategies , 2019, Multimodal Technol. Interact..

[14]  Myounghoon Jeon,et al.  If You're Angry, Turn the Music on: Music Can Mitigate Anger Effects on Driving Performance , 2014, AutomotiveUI.

[15]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[16]  Jennifer Healey,et al.  Affective wearables , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[17]  Robin Johnson,et al.  Identifying periods of drowsy driving using EEG. , 2013, Annals of advances in automotive medicine. Association for the Advancement of Automotive Medicine. Annual Scientific Conference.

[18]  M Jessurun,et al.  Effect of road layout and road environment on driving performance, drivers' physiology and road appreciation. , 1995, Ergonomics.

[19]  Clifford Nass,et al.  Emotion regulation for frustrating driving contexts , 2011, CHI.

[20]  E. Goffman.  Presentation of Self , 2006 .

[21]  D. I. Fotiadis,et al.  ASSESSMENT OF MUSCLE FATIGUE DURING DRIVING USING SURFACE EMG , 2003 .

[22]  Chris S Dula,et al.  Risky, aggressive, or emotional driving: addressing the need for consistent communication in research. , 2003, Journal of safety research.

[23]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[24]  B. Halpern-Felsher,et al.  Developmental sources of crash risk in young drivers , 2002 .

[25]  Moushumi Sharmin,et al.  Estimating Drivers' Stress from GPS Traces , 2014, AutomotiveUI.

[26]  R. Yerkes,et al.  The relation of strength of stimulus to rapidity of habit‐formation , 1908 .

[27]  Heiner Bubb.  Fahrerassistenz - primär ein Beitrag zum Komfort oder für die Sicherheit? , 2003 .

[28]  G. Tonello,et al.  The Influence of Light on Mood , 2015 .

[29]  J. Russell.  A circumplex model of affect. , 1980 .

[30]  Elisabeth Wells-Parker,et al.  An exploratory study of the relationship between road rage and crash experience in a representative sample of US drivers. , 2002, Accident; analysis and prevention.

[31]  Lisa Dorn,et al.  Driver Stress and Performance on a Driving Simulator , 1998, Hum. Factors.

[32]  Chin-Teng Lin,et al.  Nonparametric Single-Trial EEG Feature Extraction and Classification of Driver's Cognitive Responses , 2008, EURASIP J. Adv. Signal Process..

[33]  Rahul Banerjee,et al.  A comparative evaluation of neural network classifiers for stress level analysis of automotive drivers using physiological signals , 2013, Biomed. Signal Process. Control..

[34]  Bao-Liang Lu,et al.  Identifying Stable Patterns over Time for Emotion Recognition from EEG , 2016, IEEE Transactions on Affective Computing.

[35]  Michael Braun,et al.  ASAM: an Emotion Sampling Method for the Automotive Industry , 2017, AutomotiveUI.

[36]  Susanne Boll,et al.  Enlightening Drivers: A Survey on In-Vehicle Light Displays , 2016, AutomotiveUI.

[37]  John D. Lee,et al.  Warn me now or inform me later: Drivers' acceptance of real-time and post-drive distraction mitigation systems , 2012, Int. J. Hum. Comput. Stud..

[38]  Linda Ng Boyle,et al.  The influence of driver distraction on the severity of injuries sustained by teenage drivers and their passengers. , 2008, Accident; analysis and prevention.

[39]  Michael E Rakauskas,et al.  Effects of naturalistic cell phone conversations on driving performance. , 2004, Journal of safety research.

[40]  Anind K. Dey,et al.  Sensors Know When to Interrupt You in the Car: Detecting Driver Interruptibility Through Monitoring of Peripheral Interactions , 2015, CHI.

[41]  Daniel McDuff,et al.  AutoEmotive: bringing empathy to the driving experience to manage stress , 2014, DIS Companion '14.

[42]  J. Groeger.  Understanding Driving: Applying Cognitive Psychology to a Complex Everyday Task , 2000 .

[43]  Björn W. Schuller,et al.  Emotion on the Road - Necessity, Acceptance, and Feasibility of Affective Computing in the Car , 2010, Adv. Hum. Comput. Interact..

[44]  Wendy Ju,et al.  Distraction Becomes Engagement in Automated Driving , 2015 .

[45]  Gerhard Rigoll,et al.  A large-scale LED array to support anticipatory driving , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[46]  Rosalind W. Picard.  Affective computing: challenges , 2003, Int. J. Hum. Comput. Stud..

[47]  Stefan Schneegaß,et al.  A data set of real world driving to assess driver workload , 2013, AutomotiveUI.

[48]  Ioannis Konstas,et al.  Using facial expressions and peripheral physiological signals as implicit indicators of topical relevance , 2009, ACM Multimedia.

[49]  Elisabeth Schmidt,et al.  Correlation between subjective driver state measures and psychophysiological and vehicular data in simulated driving , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[50]  Ying Wang,et al.  Recognizing and Releasing Drivers' Negative Emotions by Using Music: Evidence from Driver Anger , 2016, AutomotiveUI.

[51]  Desney S. Tan,et al.  Brain-Computer Interfaces and Human-Computer Interaction , 2010, Brain-Computer Interfaces.

[52]  C. Collet,et al.  Physiological and behavioural changes associated to the management of secondary tasks while driving. , 2009, Applied ergonomics.

[53]  Andreas Butz,et al.  Heartbeat: Experience the Pulse of an Electric Vehicle , 2014, AutomotiveUI.

[54]  Paul A. Green.  Standard definitions for driving measures and statistics: overview and status of recommended practice J2944 , 2013, AutomotiveUI.

[55]  Bryan Reimer,et al.  Monitoring, managing, and motivating driver safety and well-being , 2011, IEEE Pervasive Computing.

[56]  Björn W. Schuller,et al.  On-line emotion recognition in a 3-D activation-valence-time continuum using acoustic and linguistic cues , 2009, Journal on Multimodal User Interfaces.

[57]  Georg Jahn,et al.  Peripheral detection as a workload measure in driving: Effects of traffic complexity and route guidance system use in a driving study , 2005 .

[58]  Josef F. Krems,et al.  Evaluation of night vision enhancement systems : Driver needs and acceptance , 2007 .

[59]  J. Oxley.  What to Detect , 2006 .

[60]  Ji Hyun Yang,et al.  Validity Analysis of Vehicle and Physiological Data for Detecting Driver Drowsiness, Distraction, and Workload , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.