TPNet: Trajectory Proposal Network for Motion Prediction

Making accurate motion prediction of the surrounding traffic agents such as pedestrians, vehicles, and cyclists is crucial for autonomous driving. Recent data-driven motion prediction methods have attempted to learn to directly regress the exact future position or its distribution from massive amount of trajectory data. However, it remains difficult for these methods to provide multimodal predictions as well as integrate physical constraints such as traffic rules and movable areas. In this work we propose a novel two-stage motion prediction framework, Trajectory Proposal Network (TPNet). TPNet first generates a candidate set of future trajectories as hypothesis proposals, then makes the final predictions by classifying and refining the proposals which meets the physical constraints. By steering the proposal generation process, safe and multimodal predictions are realized. Thus this framework effectively mitigates the complexity of motion prediction problem while ensuring the multimodal output. Experiments on four large-scale trajectory prediction datasets, i.e. the ETH, UCY, Apollo and Argoverse datasets, show that TPNet achieves the state-of-the-art results both quantitatively and qualitatively.

[1]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Marco Pavone,et al.  Distributional Prediction of Human Driving Behaviours using Mixture Density Networks , 2016 .

[3]  Sara McMains,et al.  Convolution neural network-based lane change intention prediction of surrounding vehicles for ACC , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Dongsuk Kum,et al.  The multilayer perceptron approach to lateral motion prediction of surrounding vehicles for autonomous vehicles , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[6]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[7]  Yu Zhao,et al.  Applying Deep Bidirectional LSTM and Mixture Density Network for Basketball Trajectory Prediction , 2017, ArXiv.

[8]  Yuke Li,et al.  Which Way Are You Going? Imitative Decision Learning for Path Forecasting in Dynamic Scenes , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Mohan M. Trivedi,et al.  How Would Surround Vehicles Move? A Unified Framework for Maneuver Classification and Motion Prediction , 2018, IEEE Transactions on Intelligent Vehicles.

[10]  Philip H. S. Torr,et al.  DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Mayank Bansal,et al.  ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst , 2018, Robotics: Science and Systems.

[12]  Dizan Vasquez,et al.  A survey on motion prediction and risk assessment for intelligent vehicles , 2014, ROBOMECH Journal.

[13]  Bo Zhang,et al.  Forecast the Plausible Paths in Crowd Scenes , 2017, IJCAI.

[14]  Amaury Nègre,et al.  Probabilistic Analysis of Dynamic Scenes and Collision Risks Assessment to Improve Driving Safety , 2011, IEEE Intelligent Transportation Systems Magazine.

[15]  Véronique Berge-Cherfaoui,et al.  Vehicle trajectory prediction based on motion model and maneuver recognition , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Yoichi Sato,et al.  Future Person Localization in First-Person Videos , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Hiren M. Mandalia,et al.  Using Support Vector Machines for Lane-Change Detection , 2005 .

[18]  Zhaoxin Li,et al.  STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[20]  Henggang Cui,et al.  Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving , 2018, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[21]  Mohan M. Trivedi,et al.  Surround vehicles trajectory analysis with recurrent neural networks , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[22]  Simon Lucey,et al.  Argoverse: 3D Tracking and Forecasting With Rich Maps , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Klaus-Dieter Kuhnert,et al.  When will it change the lane? A probabilistic regression approach for rarely occurring events , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[24]  Alessio Del Bue,et al.  MX-LSTM: Mixing Tracklets and Vislets to Jointly Forecast Trajectories and Head Poses , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Kym Watson,et al.  Recognition of dangerous situations within a cooperative group of vehicles , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[26]  Jonathan P. How,et al.  Threat assessment design for driver assistance system at intersections , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[27]  Dongchun Ren,et al.  StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topology , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  Silvio Savarese,et al.  Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Martial Hebert,et al.  Activity Forecasting , 2012, ECCV.

[30]  Volker Willert,et al.  Bayesian, maneuver-based, long-term trajectory prediction and criticality assessment for driver assistance systems , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[31]  G. Srinivasaraghavan,et al.  Human Trajectory Prediction using Spatially aware Deep Attention Models , 2017, ArXiv.

[32]  Dani Lischinski,et al.  Crowds by Example , 2007, Comput. Graph. Forum.

[33]  A. Barth,et al.  Where will the oncoming vehicle be the next second? , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[34]  Dinesh Manocha,et al.  TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents , 2018, AAAI.

[35]  Mathias Perrollaz,et al.  Learning-based approach for online lane change intention prediction , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[36]  Dariu Gavrila,et al.  Using road topology to improve cyclist path prediction , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[37]  Nanning Zheng,et al.  SR-LSTM: State Refinement for LSTM Towards Pedestrian Trajectory Prediction , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Dariu M. Gavrila,et al.  Human motion trajectory prediction: a survey , 2019, Int. J. Robotics Res..

[39]  Chung Choo Chung,et al.  Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[40]  Juan Carlos Niebles,et al.  Peeking Into the Future: Predicting Future Person Activities and Locations in Videos , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Florent Altché,et al.  An LSTM network for highway trajectory prediction , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[42]  Jonas Firl,et al.  Online maneuver recognition and multimodal trajectory prediction for intersection assistance using non-parametric regression , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[44]  Silvio Savarese,et al.  SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Jianyu Chen,et al.  Intention-aware Long Horizon Trajectory Prediction of Surrounding Vehicles using Dual LSTM Networks , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[46]  Henggang Cui,et al.  Motion Prediction of Traffic Actors for Autonomous Driving using Deep Convolutional Networks , 2018, ArXiv.