Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs

To safely and efficiently navigate through complex traffic scenarios, autonomous vehicles need to have the ability to predict the future motion of surrounding vehicles. Multiple interacting agents, the multi-modal nature of driver behavior, and the inherent uncertainty involved in the task make motion prediction of surrounding vehicles a challenging problem. In this paper, we present an LSTM model for interaction aware motion prediction of surrounding vehicles on freeways. Our model assigns confidence values to maneuvers being performed by vehicles and outputs a multi-modal distribution over future motion based on them. We compare our approach with the prior art for vehicle motion prediction on the publicly available NGSIM US-101 and I-80 datasets. Our results show an improvement in terms of RMS values of prediction error. We also present an ablative analysis of the components of our proposed model and analyze the predictions made by the model in complex traffic scenarios.

[1]  Angelos Amditis,et al.  Sensor Fusion for Predicting Vehicles' Path for Collision Avoidance Systems , 2007, IEEE Transactions on Intelligent Transportation Systems.

[2]  Mohan M. Trivedi,et al.  Learning, Modeling, and Classification of Vehicle Track Patterns from Live Video , 2008, IEEE Transactions on Intelligent Transportation Systems.

[3]  Gerd Wanielik,et al.  Comparison and evaluation of advanced motion models for vehicle tracking , 2008, 2008 11th International Conference on Information Fusion.

[4]  A. Barth,et al.  Where will the oncoming vehicle be the next second? , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[5]  Rafael Toledo-Moreo,et al.  IMM-Based Lane-Change Prediction in Highways With Low-Cost GPS/INS , 2009, IEEE Transactions on Intelligent Transportation Systems.

[6]  Helge J. Ritter,et al.  Recognition of situation classes at road intersections , 2010, 2010 IEEE International Conference on Robotics and Automation.

[7]  Amaury Nègre,et al.  Probabilistic Analysis of Dynamic Scenes and Collision Risks Assessment to Improve Driving Safety , 2011, IEEE Intelligent Transportation Systems Magazine.

[8]  Mohan M. Trivedi,et al.  Learning multi-lane trajectories using vehicle-based vision , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[9]  Ulrich Kressel,et al.  Probabilistic trajectory prediction with Gaussian mixture models , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[10]  Mohan M. Trivedi,et al.  Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis , 2013, IEEE Transactions on Intelligent Transportation Systems.

[11]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[12]  Véronique Berge-Cherfaoui,et al.  Vehicle trajectory prediction based on motion model and maneuver recognition , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[14]  Jonas Firl,et al.  Online maneuver recognition and multimodal trajectory prediction for intersection assistance using non-parametric regression , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[15]  Dizan Vasquez,et al.  A survey on motion prediction and risk assessment for intelligent vehicles , 2014, ROBOMECH Journal.

[16]  Mohan M. Trivedi,et al.  Dynamic Probabilistic Drivability Maps for Lane Change and Merge Driver Assistance , 2014, IEEE Transactions on Intelligent Transportation Systems.

[17]  Volker Willert,et al.  Bayesian, maneuver-based, long-term trajectory prediction and criticality assessment for driver assistance systems , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[18]  Markus Maurer,et al.  Towards Tactical Lane Change Behavior Planning for Automated Vehicles , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[19]  Klaus-Dieter Kuhnert,et al.  When will it change the lane? A probabilistic regression approach for rarely occurring events , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  Mohan M. Trivedi,et al.  Trajectories and Maneuvers of Surrounding Vehicles With Panoramic Camera Arrays , 2016, IEEE Transactions on Intelligent Vehicles.

[22]  Mohan M. Trivedi,et al.  Surround vehicles trajectory analysis with recurrent neural networks , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[23]  Jonas Fredriksson,et al.  If, When, and How to Perform Lane Change Maneuvers on Highways , 2016, IEEE Intelligent Transportation Systems Magazine.

[24]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Andreas Lawitzky,et al.  A Combined Model- and Learning-Based Framework for Interaction-Aware Maneuver Prediction , 2016, IEEE Transactions on Intelligent Transportation Systems.

[26]  Mykel J. Kochenderfer,et al.  Generalizable intention prediction of human drivers at intersections , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[27]  Chung Choo Chung,et al.  Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[28]  Philip H. S. Torr,et al.  DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Mykel J. Kochenderfer,et al.  Imitating driver behavior with generative adversarial networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[30]  Mohan M. Trivedi,et al.  How Would Surround Vehicles Move? A Unified Framework for Maneuver Classification and Motion Prediction , 2018, IEEE Transactions on Intelligent Vehicles.