Deep Active Localization

Active localization consists of generating robot actions that allow it to maximally disambiguate its pose within a reference map. Traditional approaches use an information-theoretic criterion for action selection and hand-crafted perceptual models. In this work we propose an end-to-end differentiable method for learning to take informative actions that is trainable entirely in simulation and then transferable to real robot hardware with zero refinement. The system is composed of two learned modules: a convolutional neural network for perception, and a deep reinforcement learned planning module. We leverage a multi-scale approach in the perceptual model since the accuracy needed to take actions using reinforcement learning is much less than the accuracy needed for robot control. We demonstrate that the resulting system outperforms traditional approach for either perception or planning. We also demonstrate our approach's robustness to different map configurations and other nuisance parameters through the use of domain randomization in training. The code has been released: https://github.com/montrealrobotics/dal and is compatible with the OpenAI gym framework, as well as the Gazebo simulator.

[1]  Peter I. Corke,et al.  Visual Place Recognition: A Survey , 2016, IEEE Transactions on Robotics.

[2]  Edwin Olson,et al.  Real-time correlative scan matching , 2009, 2009 IEEE International Conference on Robotics and Automation.

[3]  Roberto Cipolla,et al.  Convolutional networks for real-time 6-DOF camera relocalization , 2015, ArXiv.

[4]  Wolfram Burgard,et al.  Deep Auxiliary Learning for Visual Localization and Odometry , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Alberto Elfes,et al.  Using occupancy grids for mobile robot perception and navigation , 1989, Computer.

[6]  Wolfram Burgard,et al.  Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters , 2007, IEEE Transactions on Robotics.

[7]  Wolfram Burgard,et al.  Monte Carlo localization for mobile robots , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[8]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[9]  Wolfram Burgard,et al.  Active Mobile Robot Localization , 1997, IJCAI.

[10]  Elman Mansimov,et al.  Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation , 2017, NIPS.

[11]  John J. Leonard,et al.  Adaptive Mobile Robot Navigation and Mapping , 1999, Int. J. Robotics Res..

[12]  Stergios I. Roumeliotis,et al.  Active vision-based robot localization and navigation in a visual memory , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Frank P. Ferrie,et al.  Viewpoint selection by navigation through entropy maps , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  Wolfram Burgard,et al.  Monte Carlo Localization in Outdoor Terrains Using Multi-Level Surface Maps , 2007, FSR.

[15]  Geoffrey E. Hinton,et al.  A Mobile Robot That Learns Its Place , 1997, Neural Computation.

[16]  Jaime Valls Miró,et al.  Active Pose SLAM , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Fabio Viola,et al.  Learning models for visual 3D localization with implicit mapping , 2018, ArXiv.

[18]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[19]  Wolfram Burgard,et al.  Active Markov localization for mobile robots , 1998, Robotics Auton. Syst..

[20]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Ryan M. Eustice,et al.  Active visual SLAM for robotic area coverage: Theory and experiment , 2015, Int. J. Robotics Res..

[22]  Ali Farhadi,et al.  Target-driven visual navigation in indoor scenes using deep reinforcement learning , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[23]  David Hsu,et al.  Particle Filter Networks: End-to-End Probabilistic Localization From Visual Observations , 2018, ArXiv.

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Wolfram Burgard,et al.  Monte Carlo localization in outdoor terrains using multilevel surface maps , 2008 .

[26]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[27]  Ruslan Salakhutdinov,et al.  Active Neural Localization , 2018, ICLR.

[28]  Oliver Brock,et al.  Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors , 2018, Robotics: Science and Systems.

[29]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[30]  Wolfram Burgard,et al.  Coastal navigation-mobile robot navigation with uncertainty in dynamic environments , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[31]  Sebastian Thrun.  Finding landmarks for mobile robot navigation , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[32]  Richard S. Sutton,et al.  On the Significance of Markov Decision Processes , 1997, ICANN.

[33]  Davide Scaramuzza,et al.  Appearance-based Active, Monocular, Dense Reconstruction for Micro Aerial Vehicles , 2014, Robotics: Science and Systems.

[34]  W. Burgard,et al.  Markov Localization for Mobile Robots in Dynamic Environments , 1999, J. Artif. Intell. Res..

[35]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).