Multi-Step Prediction of Occupancy Grid Maps With Recurrent Neural Networks

We investigate the multi-step prediction of the drivable space, represented by Occupancy Grid Maps (OGMs), for autonomous vehicles. Our motivation is that accurate multi-step prediction of the drivable space can efficiently improve path planning and navigation resulting in safe, comfortable and optimum paths in autonomous driving. We train a variety of Recurrent Neural Network (RNN) based architectures on the OGM sequences from the KITTI dataset. The results demonstrate significant improvement of the prediction accuracy using our proposed difference learning method, incorporating motion related features, over the state of the art. We remove the egomotion from the OGM sequences by transforming them into a common frame. Although in the transformed sequences the KITTI dataset is heavily biased toward static objects, by learning the difference between consecutive OGMs, our proposed method provides accurate prediction over both the static and moving objects.

[1]  Gabriel Kreiman,et al.  Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning , 2016, ICLR.

[2]  Sebastian Thrun,et al.  Model based vehicle detection and tracking for autonomous urban driving , 2009, Auton. Robots.

[3]  Steven Lake Waslander,et al.  State initialization for recurrent neural network modeling of time-series data , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[4]  Kazuya Yoshida,et al.  Collision avoidance method for mobile robot considering motion and personal spaces of evacuees , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Sen Wang,et al.  End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks , 2018, Int. J. Robotics Res..

[6]  Klaus C. J. Dietmayer,et al.  A random finite set approach for dynamic occupancy grid maps with real-time application , 2016, Int. J. Robotics Res..

[7]  Klaus Dietmayer,et al.  Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Yann LeCun,et al.  Deep multi-scale video prediction beyond mean square error , 2015, ICLR.

[9]  Dushyant Rao,et al.  Deep tracking in the wild: End-to-end tracking using recurrent neural networks , 2018, Int. J. Robotics Res..

[10]  A. Elfes,et al.  Occupancy Grids: A Stochastic Spatial Representation for Active Robot Perception , 2013, ArXiv.

[11]  Kelson Rômulo Teixeira Aires,et al.  An Approach for 2D Visual Occupancy Grid Map Using Monocular Vision , 2011, CLEI Selected Papers.

[12]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[13]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[14]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[15]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[16]  Nitish Srivastava,et al.  Unsupervised Learning of Video Representations using LSTMs , 2015, ICML.

[17]  Ingmar Posner,et al.  Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks , 2016, AAAI.

[18]  Mark E. Campbell,et al.  A mixture‐model based algorithm for real‐time terrain estimation , 2006, J. Field Robotics.

[19]  Tomomasa Sato,et al.  Mobile robot path planning using human prediction model based on massive trajectories , 2012, 2012 Ninth International Conference on Networked Sensing (INSS).

[20]  Luc Van Gool,et al.  Object Detection and Tracking for Autonomous Navigation in Dynamic Environments , 2010, Int. J. Robotics Res..

[21]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[22]  Ray A. Jarvis,et al.  Optimal Global Path Planning in Time Varying Environments Based on a Cost Evaluation Function , 2008, Australasian Conference on Artificial Intelligence.

[23]  Klaus C. J. Dietmayer,et al.  Deep Object Tracking on Dynamic Occupancy Grid Maps Using RNNs , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[24]  Sergey Levine,et al.  Unsupervised Learning for Physical Interaction through Video Prediction , 2016, NIPS.

[25]  Emmanouil Tsardoulias,et al.  A Review of Global Path Planning Methods for Occupancy Grid Maps Regardless of Obstacle Density , 2016, J. Intell. Robotic Syst..

[26]  Steven Lake Waslander,et al.  Deep Learning a Quadrotor Dynamic Model for Multi-Step Prediction , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.