A Deep Learning Framework for Driving Behavior Identification on In-Vehicle CAN-BUS Sensor Data

Human driving behaviors are personalized and unique, and the automobile fingerprint of drivers could be helpful to automatically identify different driving behaviors and further be applied in fields such as auto-theft systems. Current research suggests that in-vehicle Controller Area Network-BUS (CAN-BUS) data can be used as an effective representation of driving behavior for recognizing different drivers. However, it is difficult to capture complex temporal features of driving behaviors in traditional methods. This paper proposes an end-to-end deep learning framework by fusing convolutional neural networks and recurrent neural networks with an attention mechanism, which is more suitable for time series CAN-BUS sensor data. The proposed method can automatically learn features of driving behaviors and model temporal features without professional knowledge in features modeling. Moreover, the method can capture salient structure features of high-dimensional sensor data and explore the correlations among multi-sensor data for rich feature representations of driving behaviors. Experimental results show that the proposed framework performs well in the real world driving behavior identification task, outperforming the state-of-the-art methods.

[1]  Hiok Chai Quek,et al.  Driving Profile Modeling and Recognition Based on Soft Computing Approach , 2009, IEEE Transactions on Neural Networks.

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Jinsong Bao,et al.  A Hybrid CNN–LSTM Algorithm for Online Defect Recognition of CO2 Welding , 2018, Sensors.

[4]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  John H. L. Hansen,et al.  Analysis and Classification of Driver Behavior using In-Vehicle CAN-Bus Information , 2007 .

[6]  Juan-Carlos Cano,et al.  Providing accident detection in vehicular networks through OBD-II devices and Android-based smartphones , 2011, 2011 IEEE 36th Conference on Local Computer Networks.

[7]  M. Amaç Güvensan,et al.  Driver Behavior Analysis for Safe Driving: A Survey , 2015, IEEE Transactions on Intelligent Transportation Systems.

[8]  Samuel Berlemont,et al.  3D gesture classification with convolutional neural networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  H. Ishida,et al.  Application of Convolutional Long Short-Term Memory Neural Networks to Signals Collected from a Sensor Network for Autonomous Gas Source Localization in Outdoor Environments , 2018, Sensors.

[10]  Ram Dantu,et al.  Safe Driving Using Mobile Phones , 2012, IEEE Transactions on Intelligent Transportation Systems.

[11]  K. Itou,et al.  Driver Identification Based on Spectral Analysis of Driving Behavioral Signals , 2007 .

[12]  Yoshihiko Suhara,et al.  Exploiting the use of recurrent neural networks for driver behavior profiling , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[13]  Kazuya Takeda,et al.  Driver Modeling Based on Driving Behavior and Its Evaluation in Driver Identification , 2007, Proceedings of the IEEE.

[14]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[15]  Manohar Das,et al.  Driver Classification for Optimization of Energy Usage in a Vehicle , 2012, CSER.

[16]  Mohan M. Trivedi,et al.  Driver classification and driving style recognition using inertial sensors , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[17]  Feng Guo,et al.  Individual driver risk assessment using naturalistic driving data. , 2013, Accident; analysis and prevention.

[18]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[19]  Tadahiro Taniguchi,et al.  Visualization of driving behavior using deep sparse autoencoder , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[20]  Marco Dozza,et al.  How does glance behavior influence crash and injury risk? A ‘what-if’ counterfactual simulation using crashes and near-crashes from SHRP2 , 2015 .

[21]  Yangsheng Xu,et al.  Human Driving Behavior Recognition Based on Hidden Markov Models , 2006, 2006 IEEE International Conference on Robotics and Biomimetics.

[22]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[23]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[24]  Dong Xuan,et al.  Mobile phone based drunk driving detection , 2010, 2010 4th International Conference on Pervasive Computing Technologies for Healthcare.

[25]  Tadayoshi Kohno,et al.  Automobile Driver Fingerprinting , 2016, Proc. Priv. Enhancing Technol..

[26]  Seungjin Choi,et al.  Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[27]  Yuhao Wang,et al.  Joint Deep Neural Network Modelling and Statistical Analysis on Characterizing Driving Behaviors , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[28]  Federico Alvarez,et al.  Modelling the Effect of Driving Events on Electrical Vehicle Energy Consumption Using Inertial Sensors in Smartphones , 2018 .

[29]  Tadahiro Taniguchi,et al.  Visualization of Driving Behavior Based on Hidden Feature Extraction by Using Deep Learning , 2017, IEEE Transactions on Intelligent Transportation Systems.

[30]  Xingjian Zhang,et al.  A Study of Individual Characteristics of Driving Behavior based on Hidden Markov Model , 2012 .

[31]  Thomas A. Dingus,et al.  Design of the In-Vehicle Driving Behavior and Crash Risk Study: In Support of the SHRP 2 Naturalistic Driving Study , 2011 .

[32]  Tara N. Sainath,et al.  Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Jeongsik Choi,et al.  NLOS Identification in WLANs Using Deep LSTM with CNN Features , 2018, Sensors.

[34]  James H. Martin,et al.  CU-NLP at SemEval-2016 Task 8: AMR Parsing using LSTM-based Recurrent Neural Networks , 2016, SemEval@NAACL-HLT.

[35]  Huy Kang Kim,et al.  Know your master: Driver profiling-based anti-theft method , 2016, 2016 14th Annual Conference on Privacy, Security and Trust (PST).

[36]  Kazuya Takeda,et al.  Driver identification using driving behavior signals , 2005, Proceedings. 2005 IEEE Intelligent Transportation Systems, 2005..

[37]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.