Vision-Based Goal-Conditioned Policies for Underwater Navigation in the Presence of Obstacles

We present Nav2Goal, a data-efficient and end-to-end learning method for goal-conditioned visual navigation. Our technique is used to train a navigation policy that enables a robot to navigate close to sparse geographic waypoints provided by a user without any prior map, all while avoiding obstacles and choosing paths that cover user-informed regions of interest. Our approach is based on recent advances in conditional imitation learning. General-purpose, safe and informative actions are demonstrated by a human expert. The learned policy is subsequently extended to be goal-conditioned by training with hindsight relabelling, guided by the robot's relative localization system, which requires no additional manual annotation. We deployed our method on an underwater vehicle in the open ocean to collect scientifically relevant data of coral reefs, which allowed our robot to operate safely and autonomously, even at very close proximity to the coral. Our field deployments have demonstrated over a kilometer of autonomous visual navigation, where the robot reaches on the order of 40 waypoints, while collecting scientifically relevant data. This is done while travelling within 0.5 m altitude from sensitive corals and exhibiting significant learned agility to overcome turbulent ocean conditions and to actively avoid collisions.

[1]  David Held,et al.  Adaptive Variance for Changing Sparse-Reward Environments , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[2]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[3]  Stefan B. Williams,et al.  Towards terrain-aided navigation for underwater robotics , 2001, Adv. Robotics.

[4]  Andrew Hogue,et al.  A visually guided swimming robot , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Marcin Andrychowicz,et al.  Hindsight Experience Replay , 2017, NIPS.

[6]  Paul Timothy Furgale,et al.  Exploiting Reusable Paths in Mobile Robotics: Benefits and Challenges for Long-term Autonomy , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[7]  Stefan B. Williams,et al.  Autonomous underwater navigation and control , 2001, Robotica.

[8]  Leslie Pack Kaelbling,et al.  Learning to Achieve Goals , 1993, IJCAI.

[9]  Alex Kendall,et al.  Concrete Dropout , 2017, NIPS.

[10]  Daniel Cremers,et al.  Direct Sparse Odometry , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Sergey Levine,et al.  Deep Imitative Models for Flexible Inference, Planning, and Control , 2018, ICLR.

[12]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[13]  Pieter Abbeel,et al.  Goal-conditioned Imitation Learning , 2019, NeurIPS.

[14]  Anca D. Dragan,et al.  DART: Noise Injection for Robust Imitation Learning , 2017, CoRL.

[15]  Gaurav S. Sukhatme,et al.  Informative path planning for an autonomous underwater vehicle , 2010, 2010 IEEE International Conference on Robotics and Automation.

[16]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[17]  Carlos R. del-Blanco,et al.  DroNet: Learning to Fly by Driving , 2018, IEEE Robotics and Automation Letters.

[18]  Gregory Dudek,et al.  Autonomous adaptive exploration using realtime online spatiotemporal topic modeling , 2014, Int. J. Robotics Res..

[19]  Andreas Krause,et al.  Nonmyopic Informative Path Planning in Spatio-Temporal Models , 2007, AAAI.

[20]  Jürgen Schmidhuber,et al.  A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots , 2016, IEEE Robotics and Automation Letters.

[21]  Alexey Dosovitskiy,et al.  End-to-End Driving Via Conditional Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Martial Hebert,et al.  Learning monocular reactive UAV control in cluttered natural environments , 2012, 2013 IEEE International Conference on Robotics and Automation.

[23]  Terrence Fong,et al.  Multi-modal active perception for information gathering in science missions , 2017, Autonomous Robots.

[24]  Mac Schwager,et al.  SACBP: Belief space planning for continuous-time dynamical systems via stochastic sequential action control , 2018, WAFR.

[25]  Nikolai Smolyanskiy,et al.  Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[27]  O. Pizarro,et al.  Visually Augmented Navigation for Autonomous Underwater Vehicles , 2008, IEEE Journal of Oceanic Engineering.

[28]  Steven M. LaValle,et al.  Randomized Kinodynamic Planning , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[29]  François Michaud,et al.  Long-term online multi-session graph-based SPLAM with memory management , 2017, Autonomous Robots.

[30]  Sergey Levine,et al.  End-to-End Robotic Reinforcement Learning without Reward Engineering , 2019, Robotics: Science and Systems.

[31]  Tom Schaul,et al.  Universal Value Function Approximators , 2015, ICML.

[32]  Vijay Kumar,et al.  Information-Theoretic Planning with Trajectory Optimization for Dense 3D Mapping , 2015, Robotics: Science and Systems.

[33]  Hanumant Singh,et al.  Towards High-resolution Imaging from Underwater Vehicles , 2007, Int. J. Robotics Res..

[34]  Roland Siegwart,et al.  From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[36]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Jason M. O'Kane,et al.  Navigation in the Presence of Obstacles for an Agile Autonomous Underwater Vehicle , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[38]  Liam Paull,et al.  An information gain based adaptive path planning method for an autonomous underwater vehicle using sidescan sonar , 2010, 2010 IEEE International Conference on Automation Science and Engineering.

[39]  Gregory Dudek,et al.  Vision-Based Autonomous Underwater Swimming in Dense Coral for Combined Collision Avoidance and Target Selection , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[40]  C. Roman,et al.  Imaging Coral I: Imaging Coral Habitats with the SeaBED AUV , 2004 .