Fast and low-power behavior analysis on vehicles using smartphones

More and more deep learning methods are applied in unmanned or assisted driving, and have achieved very excellent performance. This paper describes long short-term memory recurrent neural networks used in assisted driving, which can capture the long temporal dependencies of multiple vehicles sensors' data, supporting drivers' behavior analysis on vehicles. Some optimization methods, such as model compression, weight quantization, adaptive window segmentation, are applied to make the deep network faster and less power. Therefore, it can be easily deployed on smart-phones and other embedded devices due to its moderate energy consumption and low latency. The architecture was trained in a sequence-to-sequence prediction manner, and it explicitly learns to predict the driving patterns given the temporal context. The experiment is executed on the smart-phone. Experimental results for different parameters are also presented in the paper. At last, we reduce the model size to 77 KB, the processing time to 4.27 ms, and the power overhead is 7.7 mW, the percentage of improved performance by our optimizations is over 60%.

[1]  Mohan M. Trivedi,et al.  On-road prediction of driver's intent with multimodal sensory cues , 2011, IEEE Pervasive Computing.

[2]  慧 金成.  Asia-Pacific Conference on Vision 2015 , 2015 .

[3]  Sebastian Ramos,et al.  Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling , 2016, 2017 IEEE Intelligent Vehicles Symposium (IV).

[4]  Mo Li,et al.  IODetector: a generic service for indoor outdoor detection , 2012, SenSys '12.

[5]  Nicholas D. Lane,et al.  From smart to deep: Robust activity recognition on smartwatches using deep learning , 2016, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[6]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[7]  Rohit Prabhavalkar,et al.  On the Efficient Representation and Execution of Deep Acoustic Models , 2016, INTERSPEECH.

[8]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Jian Ma,et al.  Accelerometer Based Transportation Mode Recognition on Mobile Phones , 2010, 2010 Asia-Pacific Conference on Wearable Computing Systems.

[10]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[11]  Shaohan Hu,et al.  DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing , 2016, WWW.

[12]  Ruzena Bajcsy,et al.  Improved driver modeling for human-in-the-loop vehicular control , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[14]  Wojciech Zaremba,et al.  Learning to Execute , 2014, ArXiv.

[15]  Yong Du,et al.  Hierarchical recurrent neural network for skeleton based action recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Mohan M. Trivedi,et al.  Driving style recognition using a smartphone as a sensor platform , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[17]  Reinhard Klette,et al.  Look at the Driver, Look at the Road: No Distraction! No Accident! , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Florian Michahelles,et al.  Driving behavior analysis with smartphones: insights from a controlled field study , 2012, MUM.

[19]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.