AdaptiveON: Adaptive Outdoor Navigation Method For Stable and Reliable Actions

—We present a novel outdoor navigation algorithm to generate stable and efficient actions to navigate a robot to reach a goal. We use a multi-stage training pipeline and show that our approach produces policies that result in stable and reliable robot navigation on complex terrains. Based on the Proximal Policy Optimization (PPO) algorithm, we developed a novel method to achieve multiple capabilities for outdoor navi- gation tasks, namely alleviating the robot’s drifting, keeping the robot stable on bumpy terrains, avoiding climbing on hills with steep elevation changes, and avoiding collisions. Our training process mitigates the reality (sim-to-real) gap by introducing generalized environmental and robotic parameters and training with rich features captured from light detection and ranging (Lidar) sensor in a high-fidelity Unity simulator. We evaluate our method in both simulation and real-world environments using Clearpath Husky and Jackal robots. Further, we compare our method against the state-of-the-art approaches and observe that, in the real world, our method improves stability by at least 30 . 7% on uneven terrains, reduces drifting by 8 . 08% , and decreases the elevation changes by 14 . 75% . Code is available at: https://github.com/jingGM/adaptiveON.git

[1]  Utsav Patel,et al.  TERP: Reliable Planning in Uneven Outdoor Environments using Deep Reinforcement Learning , 2022, ICRA.

[2]  Jing Liang,et al.  TerraPN: Unstructured terrain navigation through Online Self-Supervised Learning , 2022, ArXiv.

[3]  Goldie Nejat,et al.  A Sim-to-Real Pipeline for Deep Reinforcement Learning for Autonomous Robot Navigation in Cluttered Rough Terrain , 2021, IEEE Robotics and Automation Letters.

[4]  Gregory Dudek,et al.  Trajectory-Constrained Deep Latent Visual Attention for Improved Local Planning in Presence of Heterogeneous Terrain , 2021, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Qing Zhao,et al.  Polynomial-based smooth trajectory planning for fruit-picking robot manipulator , 2021, Information Processing in Agriculture.

[6]  Jitendra Malik,et al.  RMA: Rapid Motor Adaptation for Legged Robots , 2021, Robotics: Science and Systems.

[7]  Daniel S. Drew,et al.  Multi-Agent Systems for Search and Rescue Applications , 2021, Current Robotics Reports.

[8]  P. B. Sujit,et al.  OFFSEG: A Semantic Segmentation Framework For Off-Road Driving , 2021, 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE).

[9]  Sergey Levine,et al.  How to train your robot with deep reinforcement learning: lessons we have learned , 2021, Int. J. Robotics Res..

[10]  D. Skočaj,et al.  Deep reinforcement learning for map-less goal-driven robot navigation , 2021, International Journal of Advanced Robotic Systems.

[11]  Dinesh Manocha,et al.  OF-VO: Efficient Navigation Among Pedestrians Using Commodity Sensors , 2020, IEEE Robotics and Automation Letters.

[12]  S. Levine,et al.  BADGR: An Autonomous Self-Supervised Learning-Based Navigation System , 2020, IEEE Robotics and Automation Letters.

[13]  Dinesh Manocha,et al.  GANav: Group-wise Attention Network for Classifying Navigable Regions in Unstructured Outdoor Environments , 2021, ArXiv.

[14]  Aleksey Staroverov,et al.  Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning , 2020, IEEE Access.

[15]  Lorenz Wellhausen,et al.  Learning quadrupedal locomotion over challenging terrain , 2020, Science Robotics.

[16]  Shirel Josef,et al.  Deep Reinforcement Learning for Safe Local Planning of a Ground Vehicle in Unknown Rough Terrain , 2020, IEEE Robotics and Automation Letters.

[17]  Srikanth Saripalli,et al.  An Iterative LQR Controller for Off-Road and On-Road Vehicles using a Neural Network Dynamics Model , 2020, 2020 IEEE Intelligent Vehicles Symposium (IV).

[18]  Dinesh Manocha,et al.  Crowd-Steer: Realtime Smooth and Collision-Free Robot Navigation in Densely Crowded Scenarios Trained using High-Fidelity Simulation , 2020, IJCAI.

[19]  S. Levine,et al.  Learning Agile Robotic Locomotion Skills by Imitating Animals , 2020, Robotics: Science and Systems.

[20]  Dinesh Manocha,et al.  DenseCAvoid: Real-time Navigation in Dense Crowds using Anticipatory Behaviors , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Jing Liang,et al.  CrowdSteer: Realtime Smooth and Collision-Free Robot Navigation in Dense Crowd Scenarios Trained using High-Fidelity Simulation , 2020 .

[22]  Krzysztof Walas,et al.  Where Should I Walk? Predicting Terrain Properties From Images Via Self-Supervised Learning , 2019, IEEE Robotics and Automation Letters.

[23]  Joonho Lee,et al.  Learning agile and dynamic motor skills for legged robots , 2019, Science Robotics.

[24]  Zendai Kashino,et al.  Deep Reinforcement Learning Robot for Search and Rescue Applications: Exploration in Unknown Cluttered Environments , 2019, IEEE Robotics and Automation Letters.

[25]  Brendan Englot,et al.  LeGO-LOAM: Lightweight and Ground-Optimized Lidar Odometry and Mapping on Variable Terrain , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Goldie Nejat,et al.  Robot Navigation of Environments with Unknown Rough Terrain Using deep Reinforcement Learning , 2018, 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[27]  Marco Hutter,et al.  Probabilistic Terrain Mapping for Mobile Robots With Uncertain Localization , 2018, IEEE Robotics and Automation Letters.

[28]  Andreas Krause,et al.  Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Mapless Navigation by Leveraging Prior Demonstrations , 2018, IEEE Robotics and Automation Letters.

[29]  Silvio Savarese,et al.  GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimation , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[30]  Marco Hutter,et al.  Per-Contact Iteration Method for Solving Contact Dynamics , 2018, IEEE Robotics and Automation Letters.

[31]  Roger Wattenhofer,et al.  Teaching a Machine to Read Maps with Deep Reinforcement Learning , 2017, AAAI.

[32]  Marcin Andrychowicz,et al.  Sim-to-Real Transfer of Robotic Control with Dynamics Randomization , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[34]  Sergey Levine,et al.  Uncertainty-Aware Reinforcement Learning for Collision Avoidance , 2017, ArXiv.

[35]  Lino Marques,et al.  Robots for Environmental Monitoring: Significant Advancements and Applications , 2012, IEEE Robotics & Automation Magazine.

[36]  G. Tang,et al.  Motion Planning for Vibration Reducing of Free-floating Redundant Manipulators Based on Hybrid Optimization Approach , 2011 .

[37]  Dinesh Manocha,et al.  Reciprocal collision avoidance with acceleration-velocity obstacles , 2011, 2011 IEEE International Conference on Robotics and Automation.

[38]  Joachim Hertzberg,et al.  Evaluation of 3D registration reliability and speed - A comparison of ICP and NDT , 2009, 2009 IEEE International Conference on Robotics and Automation.

[39]  Tom Duckett,et al.  Scan registration for autonomous mining vehicles using 3D‐NDT , 2007, J. Field Robotics.

[40]  K. Murphy,et al.  Path planning for autonomous vehicles driving over rough terrain , 1998, Proceedings of the 1998 IEEE International Symposium on Intelligent Control (ISIC) held jointly with IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) Intell.

[41]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[42]  W. Lovejoy.  A survey of algorithmic methods for partially observed Markov decision processes , 1991 .