Improving Autonomous Robotic Navigation Using Imitation Learning

Autonomous navigation to a specified waypoint is traditionally accomplished with a layered stack of global path planning and local motion planning modules that generate feasible and obstacle-free trajectories. While these modules can be modified to meet task-specific constraints and user preferences, current modification procedures require substantial effort on the part of an expert roboticist with a great deal of technical training. In this paper, we simplify this process by inserting a Machine Learning module between the global path planning and local motion planning modules of an off-the shelf navigation stack. This model can be trained with human demonstrations of the preferred navigation behavior, using a training procedure based on Behavioral Cloning, allowing for an intuitive modification of the navigation policy by non-technical users to suit task-specific constraints. We find that our approach can successfully adapt a robot’s navigation behavior to become more like that of a demonstrator. Moreover, for a fixed amount of demonstration data, we find that the proposed technique compares favorably to recent baselines with respect to both navigation success rate and trajectory similarity to the demonstrator.

[1]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[2]  Goldie Nejat,et al.  Learning based semi-autonomous control for robots in urban search and rescue , 2012, 2012 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[3]  Stefan Schaal,et al.  Learning, planning, and control for quadruped locomotion over challenging terrain , 2011, Int. J. Robotics Res..

[4]  Lydia E. Kavraki,et al.  Probabilistic roadmaps for path planning in high-dimensional configuration spaces , 1996, IEEE Trans. Robotics Autom..

[5]  Maggie Wigness,et al.  Hao Zhang: Robot Adaptation to Unstructured Terrains by Joint Representation and Apprenticeship Learning , 2019, Robotics: Science and Systems.

[6]  Luis E. Navarro-Serment,et al.  Robot Navigation from Human Demonstration: Learning Control Behaviors , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Geoffrey J. Gordon,et al.  No-Regret Reductions for Imitation Learning and Structured Prediction , 2010, ArXiv.

[8]  Bernard Ghanem,et al.  Driving Policy Transfer via Modularity and Abstraction , 2018, CoRL.

[9]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[10]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[11]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[12]  Jitendra Malik,et al.  Combining Optimal Control and Learning for Visual Navigation in Novel Environments , 2019, CoRL.

[13]  Sebastian Thrun,et al.  Learning Metric-Topological Maps for Indoor Mobile Robot Navigation , 1998, Artif. Intell..

[14]  Silvio Savarese,et al.  Deep Local Trajectory Replanning and Control for Robot Navigation , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[15]  Richard M. Murray,et al.  A motion planner for nonholonomic mobile robots , 1994, IEEE Trans. Robotics Autom..

[16]  S. LaValle,et al.  Randomized Kinodynamic Planning , 2001 .

[17]  Goldie Nejat,et al.  A Learning-Based Semi-Autonomous Controller for Robotic Exploration of Unknown Disaster Scenes While Searching for Victims , 2014, IEEE Transactions on Cybernetics.

[18]  Wei Gao,et al.  Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation , 2017, CoRL.

[19]  Aleksandra Faust,et al.  Learning Navigation Behaviors End to End , 2018, ArXiv.

[20]  Nicholas Roy,et al.  Learning over Subgoals for Efficient Navigation of Structured, Unknown Environments , 2018, CoRL.

[21]  Andreas Krause,et al.  Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Mapless Navigation by Leveraging Prior Demonstrations , 2018, IEEE Robotics and Automation Letters.

[22]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[23]  Luka Petrovic,et al.  Motion planning in high-dimensional spaces , 2018, ArXiv.

[24]  Csaba Szepesvári,et al.  Apprenticeship Learning using Inverse Reinforcement Learning and Gradient Methods , 2007, UAI.

[25]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.