NavTuner: Learning a Scene-Sensitive Family of Navigation Policies

The advent of deep learning has inspired research into end-to-end learning for a variety of problem domains in robotics. For navigation, the resulting methods may not have the generalization properties desired let alone match the performance of traditional methods. Instead of learning a navigation policy, we explore learning an adaptive policy in the parameter space of an existing navigation module. Having adaptive parameters provides the navigation module with a family of policies that can be dynamically reconfigured based on the local scene structure and addresses the common assertion in machine learning that engineered solutions are inflexible. Of the methods tested, reinforcement learning (RL) is shown to provide a significant performance boost to a modern navigation method through reduced sensitivity of its success rate to environmental clutter. The outcomes indicate that RL as a meta-policy learner, or dynamic parameter tuner, effectively robustifies algorithms sensitive to external, measurable nuisance factors.

[1]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[2]  Silvio Savarese,et al.  Learning to Navigate Using Mid-Level Visual Priors , 2019, CoRL.

[3]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[4]  Sergey Levine,et al.  Self-Supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Patricio A. Vela,et al.  Real-Time Egocentric Navigation Using 3D Sensing , 2019, Machine Vision and Navigation.

[6]  Ruslan Salakhutdinov,et al.  Learning to Explore using Active Neural SLAM , 2020, ICLR.

[7]  Justin S. Smith,et al.  egoTEB: Egocentric, Perception Space Navigation Using Timed-Elastic-Bands , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Roland Siegwart,et al.  From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Peter Stone,et al.  APPLD: Adaptive Planner Parameter Learning From Demonstration , 2020, IEEE Robotics and Automation Letters.

[10]  Ali Farhadi,et al.  Target-driven visual navigation in indoor scenes using deep reinforcement learning , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Jitendra Malik,et al.  Combining Optimal Control and Learning for Visual Navigation in Novel Environments , 2019, CoRL.

[12]  Yiming Yang,et al.  Automatic Parameter Tuning of Motion Planning Algorithms , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[13]  Wolfram Burgard,et al.  Deep reinforcement learning with successor features for navigation across similar environments , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[14]  Byron Boots,et al.  Differentiable Gaussian Process Motion Planning , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[16]  Aleksandra Faust,et al.  Learning Navigation Behaviors End-to-End With AutoRL , 2018, IEEE Robotics and Automation Letters.

[17]  Ming Liu,et al.  Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Jitendra Malik,et al.  On Evaluation of Embodied Navigation Agents , 2018, ArXiv.

[19]  Ye-Hoon Kim,et al.  End-to-end deep learning for autonomous navigation of mobile robot , 2018, 2018 IEEE International Conference on Consumer Electronics (ICCE).

[20]  Arash Tavakoli,et al.  Action Branching Architectures for Deep Reinforcement Learning , 2017, AAAI.

[21]  Max Q.-H. Meng,et al.  Autonomous mobile robot navigation in uneven and unstructured indoor environments , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Michael Milford,et al.  One-Shot Reinforcement Learning for Robot Navigation with Interactive Replay , 2017, ArXiv.

[23]  Joelle Pineau,et al.  Socially Adaptive Path Planning in Human Environments Using Inverse Reinforcement Learning , 2016, Int. J. Soc. Robotics.

[24]  Bo Liu,et al.  APPLI: Adaptive Planner Parameter Learning From Interventions , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Andreas Krause,et al.  Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics , 2016, Machine Learning.

[26]  Robert Babuska,et al.  Automated tuning and configuration of path planning algorithms , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Patricio A. Vela,et al.  Learning to Navigate: Exploiting Deep Networks to Inform Sample-Based Planning During Vision-Based Navigation , 2018, ArXiv.

[28]  Wei Gao,et al.  Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation , 2017, CoRL.

[29]  Jitendra Malik,et al.  Learning Navigation Subroutines from Egocentric Videos , 2019, CoRL.

[30]  Razvan Pascanu,et al.  Learning to Navigate in Complex Environments , 2016, ICLR.

[31]  Aaron Klein,et al.  Hyperparameter Optimization , 2017, Encyclopedia of Machine Learning and Data Mining.

[32]  Xin Yao,et al.  Automatic Parameter Tuning using Bayesian Optimization Method , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[33]  Hong Zhu,et al.  Hyper-Parameter Optimization: A Review of Algorithms and Applications , 2020, ArXiv.

[34]  Torsten Bertram,et al.  Trajectory modification considering dynamic constraints of autonomous robots , 2012, ROBOTIK.

[35]  Lydia Tapia,et al.  PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-Based Planning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[37]  Peter Stone,et al.  Motion Control for Mobile Robot Navigation Using Machine Learning: a Survey , 2020, ArXiv.

[38]  Torsten Bertram,et al.  Efficient trajectory optimization using a sparse model , 2013, 2013 European Conference on Mobile Robots.

[39]  Dhruv Batra,et al.  Auxiliary Tasks Speed Up Learning PointGoal Navigation , 2020, CoRL.

[40]  Sang-Jin Lee,et al.  Self-learning navigation algorithm for vision-based mobile robots using machine learning algorithms , 2011 .

[41]  Bo Liu,et al.  APPLR: Adaptive Planner Parameter Learning from Reinforcement , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).