Three-Dimensional Reconstruction Based on Visual SLAM of Mobile Robot in Search and Rescue Disaster Scenarios

Conventional simultaneous localization and mapping (SLAM) has concentrated on two-dimensional (2D) map building. To adapt it to urgent search and rescue (SAR) environments, it is necessary to combine the fast and simple global 2D SLAM and three-dimensional (3D) objects of interest (OOIs) local sub-maps. The main novelty of the present work is a method for 3D OOI reconstruction based on a 2D map, thereby retaining the fast performances of the latter. A theory is established that is adapted to a SAR environment, including the object identification, exploration area coverage (AC), and loop closure detection of revisited spots. Proposed for the first is image optical flow calculation with a 2D/3D fusion method and RGB-D (red, green, blue + depth) transformation based on Joblove–Greenberg mathematics and OpenCV processing. The mathematical theories of optical flow calculation and wavelet transformation are used for the first time to solve the robotic SAR SLAM problem. The present contributions indicate two aspects: (i) mobile robots depend on planar distance estimation to build 2D maps quickly and to provide SAR exploration AC; (ii) 3D OOIs are reconstructed using the proposed innovative methods of RGB-D iterative closest points (RGB-ICPs) and 2D/3D principle of wavelet transformation. Different mobile robots are used to conduct indoor and outdoor SAR SLAM. Both the SLAM and the SAR OOIs detection are implemented by simulations and ground-truth experiments, which provide strong evidence for the proposed 2D/3D reconstruction SAR SLAM approaches adapted to post-disaster environments.

[1]  Rodrigo Ventura,et al.  Combining monocular and stereo vision in 6D-SLAM for the localization of a tracked wheel robot , 2012, 2012 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[2]  Andrey V. Savkin,et al.  Reactive and the shortest path navigation of a wheeled mobile robot in cluttered environments , 2012, Robotica.

[3]  Luis Miguel Bergasa,et al.  Real-Time Hierarchical Outdoor SLAM Based on Stereovision and GPS Fusion , 2009, IEEE Transactions on Intelligent Transportation Systems.

[4]  Tsuneo Yoshikawa,et al.  Constructing a 3-D map of rubble by teleoperated mobile robots with a motion canceling camera system , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[5]  Muhamad Risqi U. Saputra,et al.  Visual SLAM and Structure from Motion in Dynamic Environments , 2018, ACM Comput. Surv..

[6]  Jaime Valls Miró,et al.  Information-Efficient 3-D Visual SLAM for Unstructured Domains , 2008, IEEE Transactions on Robotics.

[7]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[8]  Zhe Zhang,et al.  Finding Disaster Victims: Robot-Assisted 3D Mapping of Urban Search and Rescue Environments via Landmark Identification , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[9]  David Portugal,et al.  An evaluation of 2D SLAM techniques available in Robot Operating System , 2013, 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[10]  Keiji Nagatani,et al.  Three-dimensional localization and mapping for mobile robot in disaster environments , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[11]  Akio Gofuku,et al.  Stereoscopic presentation of 3D scan data obtained by mobile robot , 2011, 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics.

[12]  Luis Miguel Bergasa,et al.  Real-time hierarchical stereo Visual SLAM in large-scale environments , 2010, Robotics Auton. Syst..

[13]  Javier Civera,et al.  Real-time dense map fusion for stereo SLAM , 2018, Robotica.

[14]  Jaime Valls Miró,et al.  Dense 3D Map Construction for Indoor Search and Rescue , 2007, J. Field Robotics.

[15]  Joachim Hertzberg,et al.  6D SLAM—3D mapping outdoor environments , 2007, J. Field Robotics.

[16]  Werner von Seelen,et al.  Image processing and behavior planning for intelligent vehicles , 2003, IEEE Trans. Ind. Electron..

[17]  Yin-Tien Wang,et al.  Improvement of speeded-up robust features for robot visual simultaneous localization and mapping , 2014, Robotica.

[18]  Tobias Deissler,et al.  UWB SLAM with Rao-Blackwellized Monte Carlo data association , 2010, 2010 International Conference on Indoor Positioning and Indoor Navigation.

[19]  Tim Murphy,et al.  Trial by Fire , 1995 .

[20]  Isabelle Bloch,et al.  Information Fusion in Signal and Image Processing , 2008 .

[21]  Farhad Aghili.  3D simultaneous localization and mapping using IMU and its observability analysis , 2011, Robotica.

[22]  Roland Siegwart,et al.  Incremental topological segmentation for semi-structured environments using discretized GVG , 2015, Auton. Robots.

[23]  Hong Guo,et al.  Finding Disaster Victims: A Sensory System for Robot-Assisted 3D Mapping of Urban Search and Rescue Environments , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[24]  Meir Pachter,et al.  Anti-Temporal-Aliasing Constraints for Image-Based Feature Tracking Applications With and Without Inertial Aiding , 2010, IEEE Transactions on Vehicular Technology.

[25]  B. Day,et al.  A Depth Sensing Display for Bomb Disposal Robots , 2008, 2008 IEEE International Workshop on Safety, Security and Rescue Robotics.

[26]  Roland Siegwart,et al.  Monocular‐SLAM–based navigation for autonomous micro helicopters in GPS‐denied environments , 2011, J. Field Robotics.

[27]  Jaime Valls Miró,et al.  A strategy for efficient observation pruning in multi-objective 3D SLAM , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Sven Behnke,et al.  Efficient Transmission and Rendering of RGB-D Views , 2013, ISVC.

[29]  Ian Oakley,et al.  Design and evaluation of a head-mounted display for immersive 3D teleoperation of field robots , 2014, Robotica.

[30]  Zhe Zhang,et al.  Intelligent Sensing Systems for Rescue Robots: Landmark Identification and Three-Dimensional Mapping of Unknown Cluttered Urban Search and Rescue Environments , 2009, Adv. Robotics.

[31]  Rick S. Blum,et al.  Multi-sensor image fusion and its applications , 2005 .

[32]  Kenji Ueno.  Mathematical Reserch and Mathematical Literature , 2005 .

[33]  Robin R. Murphy,et al.  A case study of fuzzy-logic-based robot navigation , 2006, IEEE Robotics & Automation Magazine.

[34]  Prabir Barooah,et al.  Distributed collaborative 3D pose estimation of robots from heterogeneous relative measurements: an optimization on manifold approach , 2014, Robotica.

[35]  Jizhong Xiao,et al.  An adaptive data association for robotic SLAM in search and rescue operation , 2011, 2011 IEEE International Conference on Mechatronics and Automation.

[36]  Robin R. Murphy,et al.  Trial by fire [rescue robots] , 2004, IEEE Robotics & Automation Magazine.

[37]  Dietrich Paulus,et al.  Robbie: A Fully Autonomous Robot for RoboCupRescue , 2009, Adv. Robotics.

[38]  Lindsay Kleeman,et al.  Simultaneous landmark classification, localization and map building for an advanced sonar ring , 2006, Robotica.

[39]  Peter Stone,et al.  Selective Visual Attention for Object Detection on a Legged Robot , 2006, RoboCup.

[40]  Joachim Hertzberg,et al.  An autonomous mobile robot with a 3D laser range finder for 3D exploration and digitalization of indoor environments , 2003, Robotics Auton. Syst..

[41]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[42]  Alberto Finzi,et al.  Autonomous Attentive Exploration in Search and Rescue Scenarios , 2008, WAPCV.

[43]  Kazunori Ohno,et al.  Real-Time Robot Trajectory Estimation and 3D Map Construction using 3D Camera , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[44]  Georgios Chliveros,et al.  A system for reconstruction from point clouds in 3D: Simplification and mesh representation , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[45]  Janusz Bedkowski,et al.  Towards terrestrial 3D data registration improved by parallel programming and evaluated with geodetic precision , 2014 .

[46]  Ryan M. Eustice,et al.  Perception-driven navigation: Active visual SLAM for robotic area coverage , 2013, 2013 IEEE International Conference on Robotics and Automation.

[47]  Michael Happold,et al.  Using Learned Features from 3D Data for Robot Navigation , 2007 .

[48]  Zhe Zhang,et al.  Robot-assisted intelligent 3D mapping of unknown cluttered search and rescue environments , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[49]  Robin R. Murphy,et al.  Mobile robots in mine rescue and recovery , 2009, IEEE Robotics & Automation Magazine.

[50]  Claude Sammut,et al.  Virtual reconstruction using an autonomous robot , 2012, 2012 International Conference on Indoor Positioning and Indoor Navigation (IPIN).