Detecting Pedestrian Situation Awareness in Real-Time: Algorithm Development Using Heart Rate Variability and Phone Position

To help automated vehicles learn surrounding environments via V2X communications, it is important to detect and transfer pedestrian situation awareness to the related vehicles. Based on the characteristics of pedestrians, a real-time algorithm was developed to detect pedestrian situation awareness. In the study, the heart rate variability (HRV) and phone position were used to understand the mental state and distractions of pedestrians. The HRV analysis was used to detect the fatigue and alert state of the pedestrian, and the phone position was used to define the phone distractions of the pedestrian. A Support Vector Machine algorithm was used to classify the pedestrian’s mental state. The results indicated a good performance with 86% prediction accuracy. The developed algorithm shows high applicability to detect the pedestrian’s situation awareness in real-time, which would further extend our understanding on V2X employment and automated vehicle design.

[1]  Prabir Bhattacharya,et al.  A driver fatigue recognition model based on information fusion and dynamic Bayesian network , 2010, Inf. Sci..

[2]  E. Salas,et al.  Human Factors : The Journal of the Human Factors and Ergonomics Society , 2012 .

[3]  Leah L Thompson,et al.  Impact of social and technological distraction on pedestrian crossing behaviour: an observational study , 2012, Injury Prevention.

[4]  David B. Kaber,et al.  Situation awareness in driving while using adaptive cruise control and a cell phone: (577392012-036) , 2005 .

[5]  Paul M. Salmon,et al.  Compatible cognition amongst road users: The compatibility of driver, motorcyclist, and cyclist situation awareness , 2013 .

[6]  Mi-Kyeong Byeon,et al.  A study of HRV analysis to detect drowsiness states of drivers , 2006 .

[7]  J. Thayer,et al.  Vagal influence on working memory and attention. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[8]  Richard Wener,et al.  Mobile telephones, distracted attention, and pedestrian safety. , 2008, Accident; analysis and prevention.

[9]  H.T. Nguyen,et al.  Early Driver Fatigue Detection from Electroencephalography Signals using Artificial Neural Networks , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[10]  N Egelund,et al.  Spectral analysis of heart rate variability as an indicator of driver fatigue. , 1982, Ergonomics.

[11]  Jarle Eid,et al.  Who benefits from simulator training: Personality and heart rate variability in relation to situation awareness during navigation training , 2012, Comput. Hum. Behav..

[12]  Derek Troyer,et al.  Pedestrian injuries due to mobile phone use in public places. , 2013, Accident; analysis and prevention.

[13]  Neville A Stanton,et al.  Cognitive compatibility of motorcyclists and car drivers. , 2011, Accident; analysis and prevention.

[14]  Claudia J. Stanny,et al.  Effects of distraction and experience on situation awareness and simulated driving , 2007 .

[15]  Zhiwei Zhu,et al.  Real-time nonintrusive monitoring and prediction of driver fatigue , 2004, IEEE Transactions on Vehicular Technology.

[16]  Chong Zhang,et al.  Estimating mental fatigue based on electroencephalogram and heart rate variability , 2010 .

[17]  G. Breithardt,et al.  Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. , 1996 .

[18]  David B. Kaber,et al.  Situation awareness and workload in driving while using adaptive cruise control and a cell phone , 2005 .

[19]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[20]  Qiang Ji,et al.  Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance , 2002, Real Time Imaging.

[21]  Robert J. Kosinski,et al.  A Literature Review on Reaction Time Kinds of Reaction Time Experiments , 2012 .

[22]  U. Vollmer-Conna,et al.  Reduced heart rate variability predicts poor sleep quality in a case–control study of chronic fatigue syndrome , 2010, Experimental Brain Research.

[23]  Ashley Craig,et al.  Development of an algorithm for an EEG-based driver fatigue countermeasure. , 2003, Journal of safety research.

[24]  A. Peters,et al.  Short-Term Heart Rate Variability—Influence of Gender and Age in Healthy Subjects , 2015, PloS one.

[25]  J. Thayer,et al.  The Effect of Brief Situational Awareness Training in a Police Shooting Simulator: An Experimental Study , 2006 .

[26]  Tzyy-Ping Jung,et al.  EEG-based drowsiness estimation for safety driving using independent component analysis , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[27]  Neville Stanton,et al.  Situation awareness measurement: a review of applicability for C4i environments. , 2006, Applied ergonomics.

[28]  W. Zareba,et al.  Heart rate variability. , 2013, Handbook of clinical neurology.

[29]  Albert Kircher,et al.  Using mobile telephones: cognitive workload and attention resource allocation. , 2004, Accident; analysis and prevention.

[30]  Despina Stavrinos,et al.  Distraction and pedestrian safety: how talking on the phone, texting, and listening to music impact crossing the street. , 2012, Accident; analysis and prevention.

[31]  A. Craig,et al.  A critical review of the psychophysiology of driver fatigue , 2001, Biological Psychology.

[32]  Tapio Seppänen,et al.  Short‐term correlation properties of R–R interval dynamics at different exercise intensity levels , 2003, Clinical physiology and functional imaging.

[33]  Giuseppe Baselli,et al.  The influence of exercise intensity on the power spectrum of heart rate variability , 2004, European Journal of Applied Physiology and Occupational Physiology.

[34]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[35]  Mica R. Endsley,et al.  Measurement of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[36]  Wan-Young Chung,et al.  Driver fatigue and drowsiness monitoring system with embedded electrocardiogram sensor on steering wheel , 2014 .

[37]  Guy H. Walker,et al.  Awesome Foursome? The Compatibility of Driver, Cyclist, Motorcyclist, and Pedestrian Situation Awareness at Intersections , 2013, HCI.