Deep learning based semantic situation awareness system for multirotor aerial robots using LIDAR

In this work, we present a semantic situation awareness system for multirotor aerial robots, based on 2D LIDAR measurements, targeting the understanding of the environment and assuming to have a precise robot localization as an input of our algorithm. Our proposed situation awareness system calculates a semantic map of the objects of the environment as a list of circles represented by their radius, and the position and the velocity of their center in world coordinates. Our proposed algorithm includes three main parts. First, the LIDAR measurements are preprocessed and an object segmentation clusters the candidate objects present in the environment. Secondly, a Convolutional Neural Network (CNN) that has been designed and trained using an artificially generated dataset, computes the radius and the position of the center of individual circles in sensor coordinates. Finally, an indirect-EKF provides the estimate of the semantic map in world coordinates, including the velocity of the center of the circles in world coordinates.We have quantitative and qualitative evaluated the performance of our proposed situation awareness system by means of Software-In-The-Loop simulations using VRep with one and multiple static and moving cylindrical objects in the scene, obtaining results that support our proposed algorithm. In addition, we have demonstrated that our proposed algorithm is capable of handling real environments thanks to real laboratory experiments with non-cylindrical static (i.e. a barrel) and moving (i.e. a person) objects.

[1]  Fernando García,et al.  Obstacle Detection and Avoidance System Based on Monocular Camera and Size Expansion Algorithm for UAVs , 2017, Sensors.

[2]  A. Janowski.  The circle object detection with the use of Msplit estimation , 2018 .

[3]  Jose Luis Sanchez-Lopez,et al.  Visual Marker based Multi-Sensor Fusion State Estimation , 2017 .

[4]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[5]  Abdul Nurunnabi,et al.  Robust statistical approaches for circle fitting in laser scanning three-dimensional point cloud data , 2018, Pattern Recognit..

[6]  Jane Labadin,et al.  Feature selection based on mutual information , 2015, 2015 9th International Conference on IT in Asia (CITA).

[7]  K.B. Ariyur,et al.  Reactive inflight obstacle avoidance via radar feedback , 2005, Proceedings of the 2005, American Control Conference, 2005..

[8]  Xiangjing An,et al.  A novel setup method of 3D LIDAR for negative obstacle detection in field environment , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[9]  André Dias,et al.  Collision avoidance for safe structure inspection with multirotor UAV , 2017, 2017 European Conference on Mobile Robots (ECMR).

[10]  Hriday Bavle,et al.  Laser-Based Reactive Navigation for Multirotor Aerial Robots using Deep Reinforcement Learning , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Sven Behnke,et al.  A high-performance MAV for autonomous navigation in complex 3D environments , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Q. Guo,et al.  A geometric method for wood-leaf separation using terrestrial and simulated Lidar data , 2015 .

[15]  Seyed Mohammad Mavaei,et al.  Line Segmentation and Slam For Rescue Robots In Unknown Environments , 2013 .

[16]  S. Wender,et al.  Classification of laserscanner measurements at intersection scenarios with automatic parameter optimization , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[17]  Jing Liu,et al.  LIDAR obstacle warning and avoidance system for unmanned aerial vehicle sense-and-avoid , 2016 .

[18]  Miguel A. Olivares-Méndez,et al.  Model Predictive Control for Aerial Collision Avoidance in Dynamic Environments , 2018, 2018 26th Mediterranean Conference on Control and Automation (MED).

[19]  Sergio Montenegro,et al.  Obstacle Detection and Collision Avoidance for a UAV With Complementary Low-Cost Sensors , 2015, IEEE Access.

[20]  Jose Luis Sanchez-Lopez,et al.  A robust real-time path planner for the collision-free navigation of multirotor aerial robots in dynamic environments , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[21]  Qing Zhu,et al.  Circular Object Detection in Polar Coordinates for 2D LIDAR Data , 2016, CCPR.

[22]  Stefan Hrabar,et al.  3D path planning and stereo-based obstacle avoidance for rotorcraft UAVs , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  F. Tarsha-Kurdi,et al.  EXTENDED RANSAC ALGORITHM FOR AUTOMATIC DETECTION OF BUILDING ROOF PLANES FROM LIDAR DATA , 2008 .

[24]  Min Wang,et al.  A Real-Time 3D Path Planning Solution for Collision-Free Navigation of Multirotor Aerial Robots in Dynamic Environments , 2019, J. Intell. Robotic Syst..

[25]  Ruibin Zhao,et al.  Robust shape extraction for automatically segmenting raw LiDAR data of outdoor scenes , 2018, International Journal of Remote Sensing.

[26]  Xiao Zhang,et al.  Efficient L-shape fitting for vehicle detection using laser scanners , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[27]  Cristiano Premebida,et al.  Exploiting LIDAR-based features on pedestrian detection in urban scenarios , 2009, 2009 12th International IEEE Conference on Intelligent Transportation Systems.

[28]  Tong Heng Lee,et al.  A Comprehensive UAV Indoor Navigation System Based on Vision Optical Flow and Laser FastSLAM , 2013 .

[29]  Sebastian Scherer,et al.  Flying Fast and Low Among Obstacles , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[30]  Vicente Matellán Olivera,et al.  Tracking People in a Mobile Robot From 2D LIDAR Scans Using Full Convolutional Neural Networks for Security in Cluttered Environments , 2019, Front. Neurorobot..

[31]  Milton C. P. Santos,et al.  UAV obstacle avoidance using RGB-D system , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).