GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Driving

Despite the advancement in the technology of autonomous driving cars, the safety of a self-driving car is still a challenging problem that has not been well studied. Motion prediction is one of the core functions of an autonomous driving car. Previously, we propose a novel scheme called GRIP which is designed to predict trajectories for traffic agents around an autonomous car efficiently. GRIP uses a graph to represent the interactions of close objects, applies several graph convolutional blocks to extract features, and subsequently uses an encoder-decoder long short-term memory (LSTM) model to make predictions. Even though our experimental results show that GRIP improves the prediction accuracy of the state-of-the-art solution by 30%, GRIP still has some limitations. GRIP uses a fixed graph to describe the relationships between different traffic agents and hence may suffer some performance degradations when it is being used in urban traffic scenarios. Hence, in this paper, we describe an improved scheme called GRIP++ where we use both fixed and dynamic graphs for trajectory predictions of different types of traffic agents. Such an improvement can help autonomous driving cars avoid many traffic accidents. Our evaluations using a recently released urban traffic dataset, namely ApolloScape showed that GRIP++ achieves better prediction accuracy than state-of-the-art schemes. GRIP++ ranked #1 on the leaderboard of the ApolloScape trajectory competition in October 2019. In addition, GRIP++ runs 21.7 times faster than a state-of-the-art scheme, CS-LSTM.

[1]  Mooi Choo Chuah,et al.  GRIP: Graph-based Interaction-aware Trajectory Prediction , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[2]  Dongchun Ren,et al.  StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topology , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Mooi Choo Chuah,et al.  DAC: Data-Free Automatic Acceleration of Convolutional Networks , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[4]  Dinesh Manocha,et al.  TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Xin Li,et al.  LiveFace: A Multi-task CNN for Fast Face-Authentication , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[6]  Dinesh Manocha,et al.  TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents , 2018, AAAI.

[7]  Bin Yang,et al.  Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Mohan M. Trivedi,et al.  Convolutional Social Pooling for Vehicle Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Mohan M. Trivedi,et al.  Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[10]  Dinesh Manocha,et al.  AutoRVO: Local Navigation with Dynamic Constraints in Dense Heterogeneous Traffic , 2018, ArXiv.

[11]  Ruigang Yang,et al.  The ApolloScape Open Dataset for Autonomous Driving and Its Application , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Xin Li,et al.  ReHAR: Robust and Efficient Human Activity Recognition , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[13]  Chung Choo Chung,et al.  Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[14]  Mohan M. Trivedi,et al.  How Would Surround Vehicles Move? A Unified Framework for Maneuver Classification and Motion Prediction , 2018, IEEE Transactions on Intelligent Vehicles.

[15]  Bernt Schiele,et al.  Long-Term On-board Prediction of People in Traffic Scenes Under Uncertainty , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Yi Yang,et al.  Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[18]  Florent Altché,et al.  An LSTM network for highway trajectory prediction , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[19]  Xin Li,et al.  SBGAR: Semantics Based Group Activity Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Mooi Choo Chuah,et al.  UAV assisted smart parking solution , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[21]  Hao Wang,et al.  Depth-Projection-Map-Based Bag of Contour Fragments for Robust Hand Gesture Recognition , 2017, IEEE Transactions on Human-Machine Systems.

[22]  Xin Li,et al.  CASHEIRS: Cloud assisted scalable hierarchical encrypted based image retrieval system , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[23]  Liang Zheng,et al.  Improving Person Re-identification by Attribute and Identity Learning , 2017, Pattern Recognit..

[24]  Juan Song,et al.  Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM , 2017, IEEE Access.

[25]  Mykel J. Kochenderfer,et al.  Imitating driver behavior with generative adversarial networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[26]  Anil K. Jain,et al.  Secure Face Unlock: Spoof Detection on Smartphones , 2016, IEEE Transactions on Information Forensics and Security.

[27]  Jonathan P. How,et al.  Augmented dictionary learning for motion prediction , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Dinesh Manocha,et al.  GLMP- realtime pedestrian path prediction using global and local movement patterns , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[29]  Stefano Soatto,et al.  Intent-aware long-term prediction of pedestrian motion , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Eunhyeok Park,et al.  Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications , 2015, ICLR.

[31]  Klaus-Dieter Kuhnert,et al.  When will it change the lane? A probabilistic regression approach for rarely occurring events , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[32]  Jian Sun,et al.  Accelerating Very Deep Convolutional Networks for Classification and Detection , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Ivan V. Oseledets,et al.  Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition , 2014, ICLR.

[34]  Volker Willert,et al.  Bayesian, maneuver-based, long-term trajectory prediction and criticality assessment for driver assistance systems , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[35]  Jonas Firl,et al.  Online maneuver recognition and multimodal trajectory prediction for intersection assistance using non-parametric regression , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[36]  Jonathan P. How,et al.  Real-Time Predictive Modeling and Robust Avoidance of Pedestrians with Uncertain, Changing Intentions , 2014, WAFR.

[37]  Andrew Zisserman,et al.  Speeding up Convolutional Neural Networks with Low Rank Expansions , 2014, BMVC.

[38]  Wen Hu,et al.  Face recognition on smartphones via optimised Sparse Representation Classification , 2014, IPSN-14 Proceedings of the 13th International Symposium on Information Processing in Sensor Networks.

[39]  Véronique Berge-Cherfaoui,et al.  Vehicle trajectory prediction based on motion model and maneuver recognition , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Sorin A. Huss,et al.  Predictive maneuver evaluation for enhancement of Car-to-X mobility data , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[41]  Siddharth Swarup Rautaray,et al.  Real Time Hand Gesture Recognition System for Dynamic Applications , 2012 .

[42]  Christian Laugier,et al.  Exploiting map information for driver intention estimation at road intersections , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[43]  Rafael Toledo-Moreo,et al.  IMM-Based Lane-Change Prediction in Highways With Low-Cost GPS/INS , 2009, IEEE Transactions on Intelligent Transportation Systems.

[44]  L. Petersson,et al.  Monte Carlo based Threat Assessment: Analysis and Improvements , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[45]  Yuexin Ma,et al.  TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents , 2018 .

[46]  Humberto Loaiza,et al.  Identifying facial gestures to emulate a mouse: navigation application on Facebook. , 2017, IEEE Latin America Transactions.

[47]  Aida Khosroshahi,et al.  Learning, Classification and Prediction of Maneuvers of Surround Vehicles at Intersections using LSTMs , 2017 .