Surround Vehicle Motion Prediction Using LSTM-RNN for Motion Planning of Autonomous Vehicles at Multi-Lane Turn Intersections

This paper presents a surround vehicle motion prediction algorithm for multi-lane turn intersections using a Long Short-Term Memory (LSTM)-based Recurrent Neural Network (RNN). The motion predictor is trained using the states of subject and surrounding vehicles, which are collected by sensors mounted on an autonomous vehicle. Data on 484 vehicle trajectories were collected from real traffic situations at multi-lane turn intersections. 11,662 and 4,998 samples acquired from the vehicle trajectories were used to train and evaluate the networks, respectively. A motion planner based on Model Predictive Control (MPC) is designed to determine the longitudinal acceleration command based on the predicted states of surrounding vehicles. The future states of the subject vehicle derived by MPC is used as an input feature to reflect the interaction of subject and target vehicles in LSTM-RNN based motion predictor. The proposed algorithm was evaluated in terms of its accuracy and its effects on the motion planning algorithm based on the driving data sets. The improved prediction accuracy substantially increased safety by bounding the prediction error within the safety margin. The application results of the proposed predictor demonstrate the improved recognition timing of the preceding vehicle and the similarity of longitudinal acceleration with drivers.

[1]  Mykel J. Kochenderfer,et al.  Generalizable intention prediction of human drivers at intersections , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[2]  Angelos Amditis,et al.  Sensor Fusion for Predicting Vehicles' Path for Collision Avoidance Systems , 2007, IEEE Transactions on Intelligent Transportation Systems.

[3]  Chung Choo Chung,et al.  Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[4]  Karl-Heinz Hoffmann,et al.  Prediction of driver intended path at intersections , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[5]  Chang Wang,et al.  Driving intention recognition and behaviour prediction based on a double-layer hidden Markov model , 2012, Journal of Zhejiang University SCIENCE C.

[6]  Francesco Borrelli,et al.  Kinematic and dynamic vehicle models for autonomous driving control design , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[7]  Yonghwan Jeong,et al.  AVM / LiDAR sensor based lane marking detection method for automated driving on complex urban roads , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[8]  Angelos Amditis,et al.  Cooperative Path Prediction in Vehicular Environments , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[9]  Matthias Althoff,et al.  Comparison of Markov Chain Abstraction and Monte Carlo Simulation for the Safety Assessment of Autonomous Cars , 2011, IEEE Transactions on Intelligent Transportation Systems.

[10]  Kyongsu Yi,et al.  Stochastic Model-Predictive Control for Lane Change Decision of Automated Driving Vehicles , 2018, IEEE Transactions on Vehicular Technology.

[11]  Klaus C. J. Dietmayer,et al.  Probabilistic long-term prediction for autonomous vehicles , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[12]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[13]  Björn W. Schuller,et al.  Route and Stopping Intent Prediction at Intersections From Car Fleet Data , 2016, IEEE Transactions on Intelligent Vehicles.

[14]  Eun-Ha Choi,et al.  Crash Factors in Intersection-Related Crashes: An On-Scene Perspective , 2010 .

[15]  Martin Buss,et al.  Interactive scene prediction for automotive applications , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[16]  A. Barth,et al.  Where will the oncoming vehicle be the next second? , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[17]  Kyongsu Yi,et al.  Probabilistic and Holistic Prediction of Vehicle States Using Sensor Fusion for Application to Integrated Vehicle Safety Systems , 2014, IEEE Transactions on Intelligent Transportation Systems.

[18]  Pan Zhao,et al.  Intent-Estimation- and Motion-Model-Based Collision Avoidance Method for Autonomous Vehicles in Urban Environments , 2017 .

[19]  Volker Willert,et al.  Bayesian, maneuver-based, long-term trajectory prediction and criticality assessment for driver assistance systems , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[20]  Ulrich Kressel,et al.  Probabilistic trajectory prediction with Gaussian mixture models , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[21]  Mohan M. Trivedi,et al.  Surround vehicles trajectory analysis with recurrent neural networks , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[22]  Fawzi Nashashibi,et al.  Real time trajectory prediction for collision risk estimation between vehicles , 2009, 2009 IEEE 5th International Conference on Intelligent Computer Communication and Processing.

[23]  Horst-Michael Groß,et al.  Combining behavior and situation information for reliably estimating multiple intentions , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[24]  Stewart Worrall,et al.  Long short term memory for driver intent prediction , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[25]  Krzysztof Czarnecki,et al.  Trajectory prediction of traffic agents at urban intersections through learned interactions , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[26]  T. Kanade,et al.  Monte Carlo road safety reasoning , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[27]  Ruzena Bajcsy,et al.  Integrating Intuitive Driver Models in Autonomous Planning for Interactive Maneuvers , 2017, IEEE Transactions on Intelligent Transportation Systems.

[28]  Han-Shue Tan,et al.  Vehicle future trajectory prediction with a DGPS/INS-based positioning system , 2006, 2006 American Control Conference.

[29]  Franz Kummert,et al.  Behavior prediction at multiple time-scales in inner-city scenarios , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[30]  Olaf Stursberg,et al.  Safety assessment of driving behavior in multi-lane traffic for autonomous vehicles , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[31]  Christoph Stiller,et al.  Driver intent inference at urban intersections using the intelligent driver model , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[32]  Ümit Özgüner,et al.  Trajectory prediction of a lane changing vehicle based on driver behavior estimation and classification , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[33]  Martin Heckmann,et al.  Predicting driver left-turn behavior from few training samples using a maximum a posteriori method , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[34]  Klaus C. J. Dietmayer,et al.  The Ko-PER intersection laserscanner and video dataset , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[35]  Stewart Worrall,et al.  A Recurrent Neural Network Solution for Predicting Driver Intention at Unsignalized Intersections , 2018, IEEE Robotics and Automation Letters.

[36]  K.C.J. Dietmayer,et al.  IMM object tracking for high dynamic driving maneuvers , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[37]  Julian Eggert,et al.  Spatio-temporal trajectory similarity and its application to predicting lack of interaction in traffic situations , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[38]  Marcelo H. Ang,et al.  Vehicle motion intention reasoning using cooperative perception on urban road , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[39]  Rüdiger Dillmann,et al.  A probabilistic model for estimating driver behaviors and vehicle trajectories in traffic environments , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.