Lane Position Detection Based on Long Short-Term Memory (LSTM)

Accurate detection of lane lines is of great significance for improving vehicle driving safety. In our previous research, by improving the horizontal and vertical density of the detection grid in the YOLO v3 (You Only Look Once, the 3th version) model, the obtained lane line (LL) algorithm, YOLO v3 (S × 2S), has high accuracy. However, like the traditional LL detection algorithms, they do not use spatial information and have low detection accuracy under occlusion, deformation, worn, poor lighting, and other non-ideal environmental conditions. After studying the spatial information between LLs and learning the distribution law of LLs, an LL prediction model based on long short-term memory (LSTM) and recursive neural network (RcNN) was established; the method can predict the future LL position by using historical LL position information. Moreover, by combining the LL information predicted with YOLO v3 (S × 2S) detection results using Dempster Shafer (D-S) evidence theory, the LL detection accuracy can be improved effectively, and the uncertainty of this system be reduced correspondingly. The results show that the accuracy of LL detection can be significantly improved in rainy, snowy weather, and obstacle scenes.

[1]  Dacheng Tao,et al.  Deep Neural Network for Structural Prediction and Lane Detection in Traffic Scene , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Z. Papp,et al.  Traffic control and intelligent vehicle highway systems: a survey , 2011 .

[3]  Wei Yang,et al.  Estimation of the Lateral Distance between Vehicle and Lanes Using Convolutional Neural Network and Vehicle Dynamics , 2018 .

[4]  Shun-Feng Su,et al.  Towards self-driving car using convolutional neural network and road lane detector , 2017, 2017 2nd International Conference on Automation, Cognitive Science, Optics, Micro Electro-­Mechanical System, and Information Technology (ICACOMIT).

[5]  Yaoqi Zhou,et al.  RNA secondary structure prediction using an ensemble of two-dimensional deep neural networks and transfer learning , 2019, Nature Communications.

[6]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[7]  Witold Pedrycz,et al.  Online Feature Transformation Learning for Cross-Domain Object Category Recognition , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[8]  George M. Mohay,et al.  A Novel Sliding Window Based Change Detection Algorithm for Asymmetric Traffic , 2008, 2008 IFIP International Conference on Network and Parallel Computing.

[9]  In So Kweon,et al.  VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  Minho Lee,et al.  Fast learning method for convolutional neural networks using extreme learning machine and its application to lane detection , 2017, Neural Networks.

[11]  Muwei Jian,et al.  A Synthetic Fusion Rule for Salient Region Detection under the Framework of DS-Evidence Theory , 2018, Symmetry.

[12]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[13]  Juan E. Gilbert,et al.  An Open Road Evaluation of a Self-Driving Vehicle Human–Machine Interface Designed for Visually Impaired Users , 2019, Int. J. Hum. Comput. Interact..

[14]  Yefu Wu,et al.  Double Lane Line Edge Detection Method Based on Constraint Conditions Hough Transform , 2018, 2018 17th International Symposium on Distributed Computing and Applications for Business Engineering and Science (DCABES).

[15]  Mathias Schulze,et al.  An Accurate and Computational Efficient System for Detecting and Classifying Ego and Sides Lanes Using LiDAR , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[16]  Yong Wang,et al.  Analyzing the influence of automatic steering system on the trajectory tracking accuracy of intelligent vehicle , 2018, Adv. Eng. Softw..

[17]  Xiaolin Tang,et al.  A novel simplified model for torsional vibration analysis of a series-parallel hybrid electric vehicle , 2017 .

[18]  Haisheng Yu,et al.  Research on the energy control of a dual-motor hybrid vehicle during engine start-stop process , 2019, Energy.

[19]  Masayoshi Tomizuka,et al.  Adaptive vehicle traction force control for intelligent vehicle highway systems (IVHSs) , 1996, IEEE Trans. Ind. Electron..

[20]  Heng Tao Shen,et al.  Video Captioning With Attention-Based LSTM and Semantic Consistency , 2017, IEEE Transactions on Multimedia.

[21]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[22]  Dhiraj Manohar Dhane,et al.  A review of recent advances in lane detection and departure warning system , 2018, Pattern Recognit..

[23]  Noel E. O'Connor,et al.  Shallow and Deep Convolutional Networks for Saliency Prediction , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Yang Yang Ye,et al.  Lane detection method based on lane structural analysis and CNNs , 2018 .

[25]  Long Chen,et al.  Robust Lane Detection From Continuous Driving Scenes Using Deep Neural Networks , 2019, IEEE Transactions on Vehicular Technology.

[26]  Petr Hurtík,et al.  Lane departure warning for mobile devices based on a fuzzy representation of images , 2016, Fuzzy Sets Syst..

[27]  Minho Lee,et al.  Robust Lane Detection Based On Convolutional Neural Network and Random Sample Consensus , 2014, ICONIP.

[28]  Kwanghoon Sohn,et al.  Real-time illumination invariant lane detection for lane departure warning system , 2015, Expert Syst. Appl..

[29]  Xiang Zhang,et al.  An Improved Algorithm for Image Synthesis based on Gradient and Database , 2018, ICRCA '18.

[30]  Shaohua Wang,et al.  Optimal control of intelligent vehicle longitudinal dynamics via hybrid model predictive control , 2019, Robotics Auton. Syst..

[31]  Azzedine Boukerche,et al.  A real-time lane marking localization, tracking and communication system , 2016, Comput. Commun..

[32]  In-So Kweon,et al.  Combinatorial approach for lane detection using image and LIDAR reflectance , 2015, 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[33]  Haisheng Yu,et al.  Novel Torsional Vibration Modeling and Assessment of a Power-Split Hybrid Electric Vehicle Equipped With a Dual-Mass Flywheel , 2018, IEEE Transactions on Vehicular Technology.

[34]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[35]  Yang Yan,et al.  Accurate and robust lane detection based on Dual-View Convolutional Neutral Network , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[36]  A. Mishra,et al.  Drought forecasting using feed-forward recursive neural network , 2006 .

[37]  Wei Yang,et al.  Lateral distance detection model based on convolutional neural network , 2019 .

[38]  Petros A. Ioannou,et al.  Personalized Driver/Vehicle Lane Change Models for ADAS , 2015, IEEE Transactions on Vehicular Technology.

[39]  Wei Li,et al.  Entropic pressure between fluctuating membranes in multilayer systems , 2018, Science China Physics, Mechanics & Astronomy.

[40]  Jie Liu,et al.  A Fast Learning Method for Accurate and Robust Lane Detection Using Two-Stage Feature Extraction with YOLO v3 , 2018, Sensors.

[41]  Christopher Rose,et al.  An Integrated Vehicle Navigation System Utilizing Lane-Detection and Lateral Position Estimation Systems in Difficult Environments for GPS , 2014, IEEE Transactions on Intelligent Transportation Systems.

[42]  Klaus Werner Schmidt,et al.  A lane detection algorithm based on reliable lane markings , 2018, 2018 26th Signal Processing and Communications Applications Conference (SIU).

[43]  Qingxiang Wu,et al.  Lane detection based on spiking neural network and hough transform , 2015, 2015 8th International Congress on Image and Signal Processing (CISP).

[44]  Deepak Khosla,et al.  Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition , 2014, International Journal of Computer Vision.

[45]  Vijay Gaikwad,et al.  Lane Departure Identification for Advanced Driver Assistance , 2015, IEEE Transactions on Intelligent Transportation Systems.

[46]  Xun Wang,et al.  Lane marking detection via deep convolutional neural network , 2017, Neurocomputing.

[47]  Jiyoung Jung,et al.  Real-Time Road Lane Detection in Urban Areas Using LiDAR Data , 2018, Electronics.