Robot Navigation of Environments with Unknown Rough Terrain Using deep Reinforcement Learning

In Urban Search and Rescue (USAR) missions, mobile rescue robots need to search cluttered disaster environments in order to find victims. However, these environments can be very challenging due to the unknown rough terrain that the robots must be able to navigate. In this paper, we uniquely explore the first use of deep reinforcement learning (DRL) to address the robot navigation problem in such cluttered environments with unknown rough terrain. We have developed and trained a DRL network that uses raw sensory data from the robot's onboard sensors to determine a series of local navigation actions for a mobile robot to execute. The performance of our approach was successfully tested in several unique 3D simulated environments with varying sizes and levels of traversability.

[1]  David Silver,et al.  Learning from Demonstration for Autonomous Navigation in Complex Unstructured Terrain , 2010, Int. J. Robotics Res..

[2]  Anthony Stentz,et al.  Optimal and efficient path planning for partially-known environments , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[3]  Goldie Nejat,et al.  A Learning-Based Semi-Autonomous Controller for Robotic Exploration of Unknown Disaster Scenes While Searching for Victims , 2014, IEEE Transactions on Cybernetics.

[4]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[5]  Wolfram Burgard,et al.  Deep reinforcement learning with successor features for navigation across similar environments , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Goldie Nejat,et al.  Robot exploration in unknown cluttered environments when dealing with uncertainty , 2017, 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS).

[7]  Goldie Nejat,et al.  A hierarchical reinforcement learning based control architecture for semi-autonomous rescue robots in cluttered environments , 2010, 2010 IEEE International Conference on Automation Science and Engineering.

[8]  Jean-Claude Latombe,et al.  Numerical potential field techniques for robot path planning , 1991, Fifth International Conference on Advanced Robotics 'Robots in Unstructured Environments.

[9]  Sergey Levine,et al.  Self-Supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Steven M. LaValle,et al.  RRT-connect: An efficient approach to single-query path planning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[11]  James M. Rehg,et al.  Traversability classification using unsupervised on-line visual learning for outdoor robot navigation , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[12]  Meng Wang,et al.  Fuzzy logic-based real-time robot navigation in unknown environment with dead ends , 2008, Robotics Auton. Syst..

[13]  Peter Stone,et al.  Deep Recurrent Q-Learning for Partially Observable MDPs , 2015, AAAI Fall Symposia.

[14]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[15]  Michael Happold,et al.  Enhancing Supervised Terrain Classification with Predictive Unsupervised Learning , 2006, Robotics: Science and Systems.

[16]  Lentin Joseph.  Robot Operating System (ROS) for Absolute Beginners , 2018, Apress.

[17]  Simon Lacroix,et al.  Motion and Perception Strategies for Outdoor Mobile Robot Navigation in Unknown Environments , 1995, ISER.

[18]  Ming Liu,et al.  Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Goldie Nejat,et al.  Robotic Urban Search and Rescue: A Survey from the Control Perspective , 2013, J. Intell. Robotic Syst..

[20]  Heiko Hirschmüller,et al.  Stereo camera based navigation of mobile robots on rough terrain , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Aaron C. Courville,et al.  A Generative Model of Terrain for Autonomous Navigation in Vegetation , 2006, Int. J. Robotics Res..

[22]  Alma Y. Alanis,et al.  Rough Terrain Perception Through Geometric Entities for Robot Navigation , 2013, CSE 2013.

[23]  Goldie Nejat,et al.  Learning based semi-autonomous control for robots in urban search and rescue , 2012, 2012 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[24]  Junqiang Xi,et al.  Self‐supervised learning to visually detect terrain surfaces for autonomous robots operating in forested terrain , 2012, J. Field Robotics.

[25]  Marc G. Bellemare,et al.  The Reactor: A fast and sample-efficient Actor-Critic agent for Reinforcement Learning , 2017, ICLR.

[26]  Goldie Nejat,et al.  A victim identification methodology for rescue robots operating in cluttered USAR environments , 2013, Adv. Robotics.

[27]  Anil A. Bharath,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[28]  Goldie Nejat,et al.  Semi-autonomous exploration with robot teams in urban search and rescue , 2013, 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[29]  Wolfram Burgard,et al.  OctoMap: an efficient probabilistic 3D mapping framework based on octrees , 2013, Autonomous Robots.

[30]  Roger Wattenhofer,et al.  Teaching a Machine to Read Maps with Deep Reinforcement Learning , 2017, AAAI.

[31]  Andrey V. Savkin,et al.  Seeking a path through the crowd: Robot navigation in unknown dynamic environments with moving obstacles based on an integrated environment representation , 2014, Robotics Auton. Syst..

[32]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[33]  J. Andrew Bagnell,et al.  Improving Robot Navigation Through Self-Supervised Online Learning , 2006, Robotics: Science and Systems.

[34]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.