LaNet: Real-time Lane Identification by Learning Road SurfaceCharacteristics from Accelerometer Data

The resolution of GPS measurements, especially in urban areas, is insufficient for identifying a vehicle's lane. In this work, we develop a deep LSTM neural network model LaNet that determines the lane vehicles are on by periodically classifying accelerometer samples collected by vehicles as they drive in real time. Our key finding is that even adjacent patches of road surfaces contain characteristics that are sufficiently unique to differentiate between lanes, i.e., roads inherently exhibit differing bumps, cracks, potholes, and surface unevenness. Cars can capture this road surface information as they drive using inexpensive, easy-to-install accelerometers that increasingly come fitted in cars and can be accessed via the CAN-bus. We collect an aggregate of 60 km driving data and synthesize more based on this that capture factors such as variable driving speed, vehicle suspensions, and accelerometer noise. Our formulated LSTM-based deep learning model, LaNet, learns lane-specific sequences of road surface events (bumps, cracks etc.) and yields 100% lane classification accuracy with 200 meters of driving data, achieving over 90% with just 100 m (correspondingly to roughly one minute of driving). We design the LaNet model to be practical for use in real-time lane classification and show with extensive experiments that LaNet yields high classification accuracy even on smooth roads, on large multi-lane roads, and on drives with frequent lane changes. Since different road surfaces have different inherent characteristics or entropy, we excavate our neural network model and discover a mechanism to easily characterize the achievable classification accuracies in a road over various driving distances by training the model just once. We present LaNet as a low-cost, easily deployable and highly accurate way to achieve fine-grained lane identification.

[1]  Kang G. Shin,et al.  Invisible Sensing of Vehicle Steering with Smartphones , 2015, MobiSys.

[2]  Zhaozheng Yin,et al.  Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks , 2015, ACM Multimedia.

[3]  Stephen Marshall,et al.  Activation Functions: Comparison of trends in Practice and Research for Deep Learning , 2018, ArXiv.

[4]  Nitesh V. Chawla,et al.  Proceedings of the sixth international workshop on knowledge discovery from sensor data (SensorKDD '12) , 2012 .

[5]  Xiao Wang,et al.  Convoy: Physical Context Verification for Vehicle Platoon Admission , 2017, HotMobile.

[6]  Minglu Li,et al.  L3: Sensing driving conditions for vehicle lane-level localization on highways , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[7]  Moustafa Youssef,et al.  semMatch: road semantics-based accurate map matching for challenging positioning data , 2015, SIGSPATIAL/GIS.

[8]  Xiang-Yang Li,et al.  SmartLoc: push the limit of the inertial sensor based metropolitan localization using smartphone , 2013, MobiCom.

[9]  C. Hegarty,et al.  Modeling the effects of ionospheric scintillation on GPS/Satellite‐Based Augmentation System availability , 2003 .

[10]  Matt Smith,et al.  Pavement Condition Monitoring with Crowdsourced Connected Vehicle Data , 2014 .

[11]  Alex Sherstinsky,et al.  Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network , 2018, Physica D: Nonlinear Phenomena.

[12]  John Krumm,et al.  Probabilistic modeling of traffic lanes from GPS traces , 2010, GIS '10.

[13]  Ralf Birken,et al.  VOTERS: design of a mobile multi-modal multi-sensor system , 2012, SensorKDD '12.

[14]  Guy Pujolle,et al.  A Long Short-Term Memory Recurrent Neural Network Framework for Network Traffic Matrix Prediction , 2017, ArXiv.

[15]  John Krumm,et al.  Hidden Markov map matching through noise and sparseness , 2009, GIS.

[16]  Sang Min Yoon,et al.  Divide and Conquer-Based 1D CNN Human Activity Recognition Using Test Data Sharpening † , 2018, Sensors.

[17]  I. Elamvazuthi,et al.  Voice Recognition Algorithms using Mel Frequency Cepstral Coefficient (MFCC) and Dynamic Time Warping (DTW) Techniques , 2010, ArXiv.

[18]  Steve Renals,et al.  Learning hidden unit contributions for unsupervised speaker adaptation of neural network acoustic models , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[19]  Victor Carbune,et al.  Fast multi-language LSTM-based online handwriting recognition , 2020, International Journal on Document Analysis and Recognition (IJDAR).

[20]  Luca Calderoni,et al.  Deploying a network of smart cameras for traffic monitoring on a "city kernel" , 2014, Expert Syst. Appl..

[21]  Kyu J. Han,et al.  The CAPIO 2017 Conversational Speech Recognition System , 2017, ArXiv.

[22]  Hossam S. Hassanein,et al.  Monitoring road surface anomalies towards dynamic road mapping for future smart cities , 2017, 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[23]  Amzari Zhahir,et al.  The Suitability of GPS Receivers Update Rates for Navigation Applications , 2013 .

[24]  Shuzhi Sam Ge,et al.  Autonomous vehicle positioning with GPS in urban canyon environments , 2001, IEEE Trans. Robotics Autom..

[25]  Ciro Martins,et al.  Speaker-adaptation for hybrid HMM-ANN continuous speech recognition system , 1995, EUROSPEECH.

[26]  Ryan Newton,et al.  The pothole patrol: using a mobile sensor network for road surface monitoring , 2008, MobiSys '08.

[27]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Christian Früh,et al.  Google Street View: Capturing the World at Street Level , 2010, Computer.

[29]  Larry P. Heck,et al.  Contextual LSTM (CLSTM) models for Large scale NLP tasks , 2016, ArXiv.

[30]  Pietro Laface,et al.  Linear hidden transformations for adaptation of hybrid ANN/HMM models , 2007, Speech Commun..

[31]  Sebastian Thrun,et al.  Robust vehicle localization in urban environments using probabilistic maps , 2010, 2010 IEEE International Conference on Robotics and Automation.

[32]  Moustafa Youssef,et al.  LaneQuest: An accurate and energy-efficient lane detection system , 2015, 2015 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[33]  Khe Chai Sim,et al.  Comparison of discriminative input and output transformations for speaker adaptation in the hybrid NN/HMM systems , 2010, INTERSPEECH.

[34]  Sung-Bae Cho,et al.  Human activity recognition with smartphone sensors using deep learning neural networks , 2016, Expert Syst. Appl..

[35]  Luc Van Gool,et al.  Towards End-to-End Lane Detection: an Instance Segmentation Approach , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[36]  Andreas Vlachos,et al.  Dependency Recurrent Neural Language Models for Sentence Completion , 2015, ACL.

[37]  Edward D. Lemaire,et al.  Correcting Smartphone orientation for accelerometer-based analysis , 2013, 2013 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[38]  Patrick Weber,et al.  OpenStreetMap: User-Generated Street Maps , 2008, IEEE Pervasive Computing.

[39]  Ke Wang,et al.  Empirical Evaluation of Speaker Adaptation on DNN based Acoustic Model , 2018, INTERSPEECH.

[40]  Andrew W. Senior,et al.  Long short-term memory recurrent neural network architectures for large scale acoustic modeling , 2014, INTERSPEECH.

[41]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[42]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.