Mapless Navigation: Learning UAVs Motion forExploration of Unknown Environments

This study presents a new methodology for learning-based motion planning for autonomous exploration using aerial robots. Through the reinforcement learning method of learning through trial and error, the action policy is derived that can guide autonomous exploration of underground and tunnel environments. A new Markov decision process state is designed to learn the robot’s action policy by using simulation only, and the results is applied to the real-world environment without further learning. Reduce the need for precision map in grid-based path planner and achieve map-less navigation. The proposed method can have a path with less computing cost than the grid-based planner but has similar performance. The trained action policy is broadly evaluated in both simulation and field trials related to autonomous exploration of underground mines or indoor spaces.

[1]  Yi Lin,et al.  Gradient-based online safe trajectory generation for quadrotor flight in complex environments , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[2]  Sven Koenig,et al.  A Deep Reinforcement Learning Framework for UAV Navigation in Indoor Environments , 2019, 2019 IEEE Aerospace Conference.

[3]  Ben M. Chen,et al.  A robust online path planning approach in cluttered environments for micro rotorcraft drones , 2016 .

[4]  H. Jin Kim,et al.  Autonomous flight with robust visual odometry under dynamic lighting conditions , 2018, Autonomous Robots.

[5]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[6]  Roland Siegwart,et al.  Online Informative Path Planning for Active Classification on UAVs , 2016, ISER 2016.

[7]  Demin Xu,et al.  Intelligent Online Path Planning for UAVs in Adversarial Environments , 2012 .

[8]  Christos Papachristos,et al.  Learning-based Path Planning for Autonomous Exploration of Subterranean Environments , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[9]  S. Sastry,et al.  Autonomous Exploration in Unknown Urban Environments for Unmanned Aerial Vehicles , 2005 .

[10]  Qichao Zhang,et al.  Deep Reinforcement Learning-Based Automatic Exploration for Navigation in Unknown Environment , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Roland Siegwart,et al.  Obstacle-aware Adaptive Informative Path Planning for UAV-based Target Search , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[12]  Zendai Kashino,et al.  Deep Reinforcement Learning Robot for Search and Rescue Applications: Exploration in Unknown Cluttered Environments , 2019, IEEE Robotics and Automation Letters.

[13]  Roland Siegwart,et al.  Receding Horizon "Next-Best-View" Planner for 3D Exploration , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[15]  Mohsen Guizani,et al.  Unmanned Aerial Vehicles (UAVs): A Survey on Civil Applications and Key Research Challenges , 2018, IEEE Access.

[16]  Roland Siegwart,et al.  Three-dimensional coverage path planning via viewpoint resampling and tour optimization for aerial robots , 2015, Autonomous Robots.

[17]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[18]  Vijay Kumar,et al.  Search-Based Motion Planning for Aggressive Flight in SE(3) , 2017, IEEE Robotics and Automation Letters.

[19]  Davide Scaramuzza,et al.  Aggressive quadrotor flight through narrow gaps with onboard sensing and computing using active vision , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Christos Papachristos,et al.  Autonomous Navigation and Mapping in Underground Mines Using Aerial Robots , 2019, 2019 IEEE Aerospace Conference.

[21]  Marco Pavone,et al.  A Real-Time Framework for Kinodynamic Planning with Application to Quadrotor Obstacle Avoidance , 2016 .

[22]  Ming Liu,et al.  Mobile robots exploration through cnn-based reinforcement learning , 2016, Robotics and biomimetics.

[23]  Wolfgang Hess,et al.  Real-time loop closure in 2D LIDAR SLAM , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Arturo Gil,et al.  A comparison of path planning strategies for autonomous exploration and mapping of unknown environments , 2012, Auton. Robots.

[25]  Benjamin Kuipers,et al.  A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations , 1991, Robotics Auton. Syst..

[26]  Brian Yamauchi,et al.  A frontier-based approach for autonomous exploration , 1997, Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation'.