Recognising safety critical events: can automatic video processing improve naturalistic data analyses?

New trends in research on traffic accidents include Naturalistic Driving Studies (NDS). NDS are based on large scale data collection of driver, vehicle, and environment information in real world. NDS data sets have proven to be extremely valuable for the analysis of safety critical events such as crashes and near crashes. However, finding safety critical events in NDS data is often difficult and time consuming. Safety critical events are currently identified using kinematic triggers, for instance searching for deceleration below a certain threshold signifying harsh braking. Due to the low sensitivity and specificity of this filtering procedure, manual review of video data is currently necessary to decide whether the events identified by the triggers are actually safety critical. Such reviewing procedure is based on subjective decisions, is expensive and time consuming, and often tedious for the analysts. Furthermore, since NDS data is exponentially growing over time, this reviewing procedure may not be viable anymore in the very near future. This study tested the hypothesis that automatic processing of driver video information could increase the correct classification of safety critical events from kinematic triggers in naturalistic driving data. Review of about 400 video sequences recorded from the events, collected by 100 Volvo cars in the euroFOT project, suggested that drivers' individual reaction may be the key to recognize safety critical events. In fact, whether an event is safety critical or not often depends on the individual driver. A few algorithms, able to automatically classify driver reaction from video data, have been compared. The results presented in this paper show that the state of the art subjective review procedures to identify safety critical events from NDS can benefit from automated objective video processing. In addition, this paper discusses the major challenges in making such video analysis viable for future NDS and new potential applications for NDS video processing. As new NDS such as SHRP2 are now providing the equivalent of five years of one vehicle data each day, the development of new methods, such as the one proposed in this paper, seems necessary to guarantee that these data can actually be analysed.

[1]  Roberto Tadei,et al.  Deliverable D5.3: Final delivery of data and answers to questionnaires , 2012 .

[2]  David Geer.  Digital rights technology sparks interoperability concerns , 2004, Computer.

[3]  J. Hanley,et al.  A method of comparing the areas under receiver operating characteristic curves derived from the same cases. , 1983, Radiology.

[4]  Tanaya Guha,et al.  Learning Sparse Representations for Human Action Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Trent Victor,et al.  Sweden-Michigan Naturalistic Field Operational Test (SeMiFOT), Phase 1: WP5 Evaluation of Methodology. Final Report , 2010 .

[6]  W W Wierwille,et al.  Evaluation of driver drowsiness by trained raters. , 1994, Accident; analysis and prevention.

[7]  Alan Kirk,et al.  Accident causation and pre-accidental driving situations. Part 2. In-depth accident causation analysis , 2008 .

[8]  Thomas A Dingus,et al.  Naturalistic assessment of novice teenage crash experience. , 2011, Accident; analysis and prevention.

[9]  Peter A. Hancock,et al.  The distraction effects of phone use during a crucial driving maneuver. , 2003 .

[10]  J. Swets.  ROC analysis applied to the evaluation of medical imaging techniques. , 1979, Investigative radiology.

[11]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Johnny Ng,et al.  Dynamic Local Feature Analysis for Face Recognition , 2004, ICBA.

[13]  Yongdong Zhang,et al.  Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Richard J Hanowski,et al.  A descriptive analysis of light vehicle-heavy vehicle interactions using in situ driving data. , 2007, Accident; analysis and prevention.

[15]  Richard J. Hanowski,et al.  Driver Distraction in Commercial Vehicle Operations , 2009 .

[16]  Maria E. Jabon,et al.  Facial expression analysis for predicting unsafe driving behavior , 2011, IEEE Pervasive Computing.

[17]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[18]  David Geer.  Will Gesture-Recognition Technology Point the Way? , 2004, Computer.

[19]  Myra Blanco,et al.  The impact of secondary task cognitive processing demand on driving performance. , 2006, Accident; analysis and prevention.

[20]  Katherine M White,et al.  Dialling and driving: factors influencing intentions to use a mobile phone while driving. , 2008, Accident; analysis and prevention.

[21]  B A Jonah,et al.  Sensation seeking, risky driving and behavioral adaptation. , 2001, Accident; analysis and prevention.

[22]  Carlo S. Regazzoni,et al.  Real-time video-shot detection for scene surveillance applications , 2000, IEEE Trans. Image Process..

[23]  Marco Dozza,et al.  What factors influence drivers' response time for evasive maneuvers in real traffic? , 2013, Accident; analysis and prevention.

[24]  Thomas A. Dingus,et al.  Evaluating the Relationship Between Near-Crashes and Crashes: Can Near-Crashes Serve as a Surrogate Safety Metric for Crashes? , 2010 .

[25]  Mohammed Yeasin,et al.  From facial expression to level of interest: a spatio-temporal approach , 2004, CVPR 2004.

[26]  Yuichi Kobayashi,et al.  The EMOSIGN - analyzing the emotion signature in human motion , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[27]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[28]  Anastasios Tefas,et al.  Combining Fuzzy Vector Quantization With Linear Discriminant Analysis for Continuous Human Movement Recognition , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Pascal Drazetic,et al.  Determination of pre-impact occupant postures and analysis of consequences on injury outcome--part II: biomechanical study. , 2011, Accident; analysis and prevention.

[30]  Lai Tu,et al.  Forwards: A Map-Free Intersection Collision-Warning System for All Road Patterns , 2010, IEEE Transactions on Vehicular Technology.

[31]  Terrance E. Boult,et al.  Into the woods: visual surveillance of noncooperative and camouflaged targets in complex outdoor settings , 2001, Proc. IEEE.

[32]  Richard Bishop,et al.  Intelligent Vehicle Technology and Trends , 2005 .

[33]  James R. Sayer,et al.  Integrated vehicle-based safety systems: light vehicle field operational test, key findings report. , 2011, Annals of emergency medicine.

[34]  Robert J Carroll,et al.  The development of a naturalistic data collection system to perform critical incident analysis: an investigation of safety and fatigue issues in long-haul trucking. , 2006, Accident; analysis and prevention.

[35]  Maria E. Jabon,et al.  Automatically Analyzing Facial-Feature Movements to Identify Human Errors , 2011, IEEE Intelligent Systems.

[36]  Thomas Haslwanter,et al.  Improving calibration of 3-D video oculography systems , 2004, IEEE Transactions on Biomedical Engineering.

[37]  Thomas A. Dingus,et al.  The 100-Car Naturalistic Driving Study Phase II – Results of the 100-Car Field Experiment , 2006 .

[38]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.