Denoising Recurrent Neural Networks for Classifying Crash-Related Events

With detailed sensor and visual data from automobiles, a data-driven model can learn to classify crash-related events during a drive. We propose a neural network model accepting time-series vehicle sensor data and forward-facing videos as input for learning classification of crash-related events and varying types of such events. To elaborate, a novel recurrent neural network structure is introduced, namely, denoising gated recurrent unit with decay, in order to deal with time-series automobile sensor data with missing value and noises. Our model detects crash and near-crash events based on a large set of time-series data collected from naturalistic driving behavior. Furthermore, the model classifies those events involving pedestrians, a vehicle in front, or a vehicle on either side. The effectiveness of our model is evaluated with more than two thousand 30-s clips from naturalistic driving behavior data. The results show that the model, including sensory encoder with denoising gated recurrent unit with decay, visual encoder, and attention mechanism, outperforms gated recurrent unit with decay, gated CNN, and other baselines not only in event classification and but also in event-type classification.

[1]  Yann Dauphin,et al.  Language Modeling with Gated Convolutional Networks , 2016, ICML.

[2]  Qing Wu,et al.  A confabulation model for abnormal vehicle events detection in wide-area traffic monitoring , 2014, 2014 IEEE International Inter-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA).

[3]  Khaled Daqrouq,et al.  ECG Signal Denoising By Wavelet Transform Thresholding , 2008 .

[4]  Y. Raiwani,et al.  Vehicular Accident Analysis Using Neural Network , 2014 .

[5]  Rangachar Kasturi,et al.  Autonomous Driving Challenge: To Infer the Property of a Dynamic Object Based on Its Motion Pattern , 2016, ECCV Workshops.

[6]  S. Vijayarani,et al.  Analysis of road accidents in India using data mining classification algorithms , 2017, 2017 International Conference on Inventive Computing and Informatics (ICICI).

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[9]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.

[10]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[11]  Mykel J. Kochenderfer,et al.  Analysis of Recurrent Neural Networks for Probabilistic Modeling of Driver Behavior , 2017, IEEE Transactions on Intelligent Transportation Systems.

[12]  Lane F Burgette,et al.  Symmetric Bayesian Multinomial Probit Models , 2010 .

[13]  Quoc V. Le,et al.  Learning Longer-term Dependencies in RNNs with Auxiliary Losses , 2018, ICML.

[14]  Toshio Tsuji,et al.  Bioelectric signal classification using a recurrent probabilistic neural network with time-series discriminant component analysis , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[15]  Washington Leite Junger,et al.  Imputation of missing data in time series for air pollutants , 2015 .

[16]  J. Yosinski,et al.  Time-series Extreme Event Forecasting with Neural Networks at Uber , 2017 .

[17]  Peter E.D. Love,et al.  A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory , 2018 .

[18]  Wilfried N. Gansterer,et al.  Classification of Vehicle Collision Patterns in Road Accidents using Data Mining Algorithms , 2016 .

[19]  Weigong Zhang,et al.  Application of Wavelet Transform in Vehicle Wheel Speed Signal Denoising , 2009, 2009 International Conference on Measuring Technology and Mechatronics Automation.

[20]  Johan A. K. Suykens,et al.  Handling missing values in support vector machine classifiers , 2005, Neural Networks.

[21]  Chia-Hung Yeh,et al.  Vision-based vehicle event detection through visual rhythm analysis , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[22]  Bingru Yang,et al.  A SVM Regression Based Approach to Filling in Missing Values , 2005, KES.

[23]  Yu Wang,et al.  Application of Wavelet Method with Nonlinear Threshold Control in Vehicle Wheel Force Signal Denoising , 2011, 2011 Fourth International Conference on Intelligent Computation Technology and Automation.

[24]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[25]  Ed H. Chi,et al.  Factorized Recurrent Neural Architectures for Longer Range Dependence , 2018, AISTATS.

[26]  Jérôme Louradour,et al.  Audio Events Detection in Public Transport Vehicle , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[27]  Sachin Kumar,et al.  A data mining approach to characterize road accident locations , 2016, Journal of Modern Transportation.

[28]  Tami Toroyan,et al.  Global Status Report on Road Safety: Time for Action , 2009 .

[29]  Tomoaki Yano,et al.  Traffic Light Recognition and Dangerous Driving Events Detection from Surveillance Video of Vehicle Camera , 2017 .

[30]  Naina Mahajan,et al.  Analysis of Factors of Road Traffic Accidents using Enhanced Decision Tree Algorithm , 2016 .

[31]  S. Czesla,et al.  A posteriori noise estimation in variable data sets: With applications to spectra and light curves , 2017, 1712.02226.

[32]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  John R. Hershey,et al.  Driver confusion status detection using recurrent neural networks , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[34]  Jimeng Sun,et al.  Using recurrent neural network models for early detection of heart failure onset , 2016, J. Am. Medical Informatics Assoc..

[35]  Vijender Kumar Solanki,et al.  Analysis of Datamining Technique for Traffic Accident Severity Problem: A Review , 2017, RICE.

[36]  Jianzhong Li,et al.  K-Nearest Neighbor Based Missing Data Estimation Algorithm in Wireless Sensor Networks , 2010, Wirel. Sens. Netw..

[37]  Darccedil,et al.  A neural network (NN) model to predict intersection crashes based upon driver, vehicle and roadway surface characteristics 1 , 2010 .

[38]  Survey of Image Denoising using Different Filters , 2013 .

[39]  Mojtaba Ziyadi,et al.  Prediction of accident severity using artificial neural networks , 2011 .

[40]  Qiang Fu,et al.  Chemical substance classification using long short-term memory recurrent neural network , 2017, 2017 IEEE 17th International Conference on Communication Technology (ICCT).

[41]  A.R. Weerasinghe,et al.  Simulation of real-time vehicle speed violation detection using complex event processing , 2016, 2016 IEEE International Conference on Information and Automation for Sustainability (ICIAfS).

[42]  V. Wadley,et al.  Visual Sensory and Visual-Cognitive Function and Rate of Crash and Near-Crash Involvement Among Older Drivers Using Naturalistic Driving Data , 2017, Investigative ophthalmology & visual science.

[43]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[44]  Sei-Wang Chen,et al.  Automatic change detection of driving environments in a vision-based driver assistance system , 2003, IEEE Trans. Neural Networks.

[45]  Alexander Katalinic,et al.  Imputation of missing values of tumour stage in population-based cancer registration , 2011, BMC medical research methodology.

[46]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[47]  Jonardo R. Asor,et al.  A study on the road accidents using data investigation and visualization in Los Baños, Laguna, Philippines , 2018, 2018 International Conference on Information and Communications Technology (ICOIACT).

[48]  Chao Li,et al.  Real-time Detection of Abnormal Vehicle Events with Multi-Feature over Highway Surveillance Video , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[49]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[50]  Sabrina Eberhart,et al.  Applied Missing Data Analysis , 2016 .

[51]  Suttipong Thajchayapong,et al.  Detection of Driving Events using Sensory Data on Smartphone , 2017, Int. J. Intell. Transp. Syst. Res..

[52]  Jun Wang,et al.  Forecasting energy market indices with recurrent neural networks: Case study of crude oil price fluctuations , 2016 .

[53]  Yoshihiko Suhara,et al.  Driver behavior profiling: An investigation with different smartphone sensors and machine learning , 2017, PloS one.

[54]  Li Jie,et al.  Research on accident prediction of intersection and identification method of prominent accident form based on back propagation neural network , 2010, 2010 International Conference on Computer Application and System Modeling (ICCASM 2010).

[55]  D. Narmadha,et al.  A Survey on Image Denoising Techniques , 2012 .

[56]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[57]  S. L. Joshi,et al.  A Survey on ECG Signal Denoising Techniques , 2013, 2013 International Conference on Communication Systems and Network Technologies.

[58]  Sowmya,et al.  Analyzing the Road Traffic and Accidents with Classification Techniques , 2013 .

[59]  Vitor Augusto Ozaki,et al.  Missing data imputation of climate datasets: implications to modeling extreme drought events , 2014 .