Agent Prioritization for Autonomous Navigation

In autonomous navigation, a planning system reasons about other agents to plan a safe and plausible trajectory. Before planning starts, agents are typically processed with computationally intensive models for recognition, tracking, motion estimation and prediction. With limited computational resources and a large number of agents to process in real time, it becomes important to efficiently rank agents according to their impact on the decision making process. This allows spending more time processing the most important agents. We propose a system to rank agents around an autonomous vehicle (AV) in real time. We automatically generate a ranking data set by running the planner in simulation on real-world logged data, where we can afford to run more accurate and expensive models on all the agents. The causes of various planner actions are logged and used for assigning ground truth importance scores. The generated data set can be used to learn ranking models. In particular, we show the utility of combining learned features, via a convolutional neural network, with engineered features designed to capture domain knowledge. We show the benefits of various design choices experimentally. When tested on real AVs, our system demonstrates the capability of understanding complex driving situations.

[1]  Christopher J. C. Burges,et al.  From RankNet to LambdaRank to LambdaMART: An Overview , 2010 .

[2]  Mayank Bansal,et al.  ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst , 2018, Robotics: Science and Systems.

[3]  Silvio Savarese,et al.  Learning to Track at 100 FPS with Deep Regression Networks , 2016, ECCV.

[4]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[5]  Jaana Kekäläinen,et al.  IR evaluation methods for retrieving highly relevant documents , 2000, SIGIR Forum.

[6]  Yoshua Bengio,et al.  Object Recognition with Gradient-Based Learning , 1999, Shape, Contour and Grouping in Computer Vision.

[7]  Xiaogang Wang,et al.  Multi-context Attention for Human Pose Estimation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[9]  Tie-Yan Liu,et al.  A Theoretical Analysis of NDCG Type Ranking Measures , 2013, COLT.

[10]  Ming-Hsuan Yang,et al.  Object Tracking Benchmark , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  S. V. N. Vishwanathan,et al.  Ranking via Robust Binary Classification , 2014, NIPS.

[12]  Thomas Colthurst,et al.  Compact multi-class boosted trees , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[13]  William Whittaker,et al.  Self-Driving Cars and the Urban Challenge , 2008, IEEE Intelligent Systems.

[14]  Sujitha Martin,et al.  Learning to Attend to Salient Targets in Driving Videos Using Fully Convolutional RNN , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[15]  Rong Jin,et al.  Learning to Rank by Optimizing NDCG Measure , 2009, NIPS.

[16]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[17]  Qiang Wu,et al.  McRank: Learning to Rank Using Multiple Classification and Gradient Boosting , 2007, NIPS.

[18]  Thomas Hofmann,et al.  Learning to Rank with Nonsmooth Cost Functions , 2006, NIPS.

[19]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[20]  Luc Van Gool,et al.  End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners , 2018, ECCV.

[21]  Mohan M. Trivedi,et al.  Are all objects equal? Deep spatio-temporal importance prediction in driving videos , 2017, Pattern Recognit..

[22]  Greg Ridgeway,et al.  Generalized Boosted Models: A guide to the gbm package , 2006 .

[23]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[24]  Robert D. Nowak,et al.  Active Ranking using Pairwise Comparisons , 2011, NIPS.

[25]  Thierry Fraichard,et al.  Dynamic trajectory planning with dynamic constraints: A 'state-time space' approach , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[26]  Kai Oliver Arras,et al.  Joint Long-Term Prediction of Human Motion Using a Planning-Based Social Force Approach , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Francisco Facchinei,et al.  On the Accurate Identification of Active Constraints , 1998, SIAM J. Optim..

[28]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[29]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Patrick van der Smagt,et al.  Two-stream RNN/CNN for action recognition in 3D videos , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[31]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[32]  Yoram Singer,et al.  Learning to Order Things , 1997, NIPS.

[33]  Thomas Colthurst,et al.  TF Boosted Trees: A Scalable TensorFlow Based Framework for Gradient Boosting , 2017, ECML/PKDD.