Deep Local Trajectory Replanning and Control for Robot Navigation

We present a navigation system that combines ideas from hierarchical planning and machine learning. The system uses a traditional global planner to compute optimal paths towards a goal, and a deep local trajectory planner and velocity controller to compute motion commands. The latter components of the system adjust the behavior of the robot through attention mechanisms such that it moves towards the goal, avoids obstacles, and respects the space of nearby pedestrians. Both the structure of the proposed deep models and the use of attention mechanisms make the system’s execution interpretable. Our simulation experiments suggest that the proposed architecture outperforms baselines that try to map global plan information and sensor data directly to velocity commands. In comparison to a hand-designed traditional navigation system, the proposed approach showed more consistent performance.

[1]  L. Steels The Biology and Technology of Intelligent Autonomous Agents , 1995, NATO ASI Series.

[2]  Rahul Sukthankar,et al.  Cognitive Mapping and Planning for Visual Navigation , 2017, International Journal of Computer Vision.

[3]  S. R. Searle,et al.  Restricted Maximum Likelihood (REML) Estimation of Variance Components in the Mixed Model , 1976 .

[4]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[5]  Jonathan P. How,et al.  Socially aware motion planning with deep reinforcement learning , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[7]  Roland Siegwart,et al.  From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Sebastian Thrun,et al.  Lifelong robot learning , 1993, Robotics Auton. Syst..

[9]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[10]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[12]  Alonzo Kelly,et al.  Mobile Robotics: Mathematics, Models, and Methods , 2013 .

[13]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[14]  Lakhmi C. Jain,et al.  Path Planning and Obstacle Avoidance for Autonomous Mobile Robots: A Review , 2006, KES.

[15]  Shimon Whiteson,et al.  Acquiring social interaction behaviours for telepresence robots via deep learning from demonstration , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[17]  Glen Berseth,et al.  DeepLoco: dynamic locomotion skills using hierarchical deep reinforcement learning , 2017, ACM Trans. Graph..

[18]  Matthieu Cord,et al.  MUTAN: Multimodal Tucker Fusion for Visual Question Answering , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Silvio Savarese,et al.  Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Bernard Ghanem,et al.  Driving Policy Transfer via Modularity and Abstraction , 2018, CoRL.

[21]  Silvio Savarese,et al.  Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Wolfram Burgard,et al.  Improving Grid-based SLAM with Rao-Blackwellized Particle Filters by Adaptive Proposals and Selective Resampling , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[23]  Kai Oliver Arras,et al.  Learning socially normative robot navigation behaviors with Bayesian inverse reinforcement learning , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Juan Carlos Niebles,et al.  A Deep Learning Based Behavioral Approach to Indoor Autonomous Navigation , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Wolfram Burgard,et al.  Socially compliant mobile robot navigation via inverse reinforcement learning , 2016, Int. J. Robotics Res..

[26]  Trevor Darrell,et al.  Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding , 2016, EMNLP.

[27]  J A Bagnell,et al.  An Invitation to Imitation , 2015 .

[28]  William D. Smart,et al.  Tuning Cost Functions for Social Navigation , 2013, ICSR.

[29]  A. Kendon Conducting Interaction: Patterns of Behavior in Focused Encounters , 1990 .

[30]  Benjamin Kuipers,et al.  Socially-Aware Navigation Using Topological Maps and Social Norm Learning , 2018, AIES.

[31]  Andreas Krause,et al.  Robot navigation in dense human crowds: the case for cooperation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[32]  Wolfram Burgard,et al.  Time dependent planning on a layered social cost map for human-aware robot navigation , 2015, 2015 European Conference on Mobile Robots (ECMR).

[33]  Gonzalo Ferrer Mínguez,et al.  SOCIAL ROBOT NAVIGATION , 2013 .

[34]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[35]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[36]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Pieter Abbeel,et al.  Value Iteration Networks , 2016, NIPS.

[38]  David M. Bradley,et al.  Boosting Structured Prediction for Imitation Learning , 2006, NIPS.

[39]  Hannes Sommer,et al.  Predicting actions to act predictably: Cooperative partial motion planning with maximum entropy models , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[40]  E. Hall,et al.  The Hidden Dimension , 1970 .

[41]  Byron Boots,et al.  Agile Autonomous Driving using End-to-End Deep Imitation Learning , 2017, Robotics: Science and Systems.

[42]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[43]  Jean Oh,et al.  Social Attention: Modeling Attention in Human Crowds , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[44]  William D. Smart,et al.  Layered costmaps for context-sensitive navigation , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[45]  Oussama Khatib,et al.  Elastic bands: connecting path planning and control , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[46]  S. Tang,et al.  A review of control architectures for autonomous navigation of mobile robots , 2011 .

[47]  Carlos R. del-Blanco,et al.  DroNet: Learning to Fly by Driving , 2018, IEEE Robotics and Automation Letters.

[48]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[49]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[50]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[51]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[52]  Jodi Forlizzi,et al.  Social Robot Navigation , 2010 .

[53]  Wolfram Burgard,et al.  Socially Compliant Navigation Through Raw Depth Inputs with Generative Adversarial Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[54]  Yann LeCun,et al.  Off-Road Obstacle Avoidance through End-to-End Learning , 2005, NIPS.

[55]  Christian Vollmer,et al.  Learning to navigate through crowded environments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[56]  Kai Oliver Arras,et al.  Socially-aware robot navigation: A learning approach , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[57]  Wei Gao,et al.  Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation , 2017, CoRL.

[58]  Brian P. Gerkey Planning and Control in Unstructured Terrain , 2008 .

[59]  Andrew Y. Ng,et al.  A control architecture for quadruped locomotion over rough terrain , 2008, 2008 IEEE International Conference on Robotics and Automation.

[60]  Michael W. Otte A Survey of Machine Learning Approaches to Robotic Path-Planning , 2009 .

[61]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[62]  Dieter Fox,et al.  KLD-Sampling: Adaptive Particle Filters , 2001, NIPS.

[63]  Zvi Shiller,et al.  Dynamic motion planning of autonomous vehicles , 1991, IEEE Trans. Robotics Autom..

[64]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[65]  Oussama Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Autonomous Robot Vehicles.

[66]  Stefano Ermon,et al.  Model-Free Imitation Learning with Policy Optimization , 2016, ICML.

[67]  Christian Laugier,et al.  From Proxemics Theory to Socially-Aware Navigation: A Survey , 2015, Int. J. Soc. Robotics.

[68]  Richard M. Murray,et al.  A motion planner for nonholonomic mobile robots , 1994, IEEE Trans. Robotics Autom..

[69]  P. Molnár Social Force Model for Pedestrian Dynamics Typeset Using Revt E X 1 , 1995 .

[70]  P. Dayan,et al.  Flexible shaping: How learning in small steps helps , 2009, Cognition.

[71]  Martial Hebert,et al.  Improved Learning of Dynamics Models for Control , 2016, ISER.