Semantic Histogram Based Graph Matching for Real-Time Multi-Robot Global Localization in Large Scale Environment

The core problem of visual multi-robot simultaneous localization and mapping (MR-SLAM) is how to efficiently and accurately perform multi-robot global localization (MR-GL). The difficulties are two-fold. The first is the difficulty of global localization for significant viewpoint difference. Appearance-based localization methods tend to fail under large viewpoint changes. Recently, semantic graphs have been utilized to overcome the viewpoint variation problem. However, the methods are highly time-consuming, especially in large-scale environments. This leads to the second difficulty, which is how to perform real-time global localization. In this paper, we propose a semantic histogram-based graph matching method that is robust to viewpoint variation and can achieve real-time global localization. Based on that, we develop a system that can accurately and efficiently perform MR-GL for both homogeneous and heterogeneous robots. The experimental results show that our approach is about 30 times faster than Random Walk based semantic descriptors. Moreover, it achieves an accuracy of 95% for global localization, while the accuracy of the state-of-the-art method is 85%.

[1]  Abel Gawel,et al.  X-View: Graph-Based Semantic Multiview Localization , 2017, IEEE Robotics and Automation Letters.

[2]  Tim Fingscheidt,et al.  Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance , 2020, ECCV.

[3]  John J. Leonard,et al.  Toward Object-based Place Recognition in Dense RGB-D Maps , 2015 .

[4]  Beom-Hee Lee,et al.  A survey of map merging techniques for cooperative-SLAM , 2012, 2012 9th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[5]  Niko Sünderhauf,et al.  Look No Deeper: Recognizing Places from Opposing Viewpoints under Varying Scene Appearance using Single-View Depth Estimation , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[6]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[7]  Sen Wang,et al.  Global Localization with Object-Level Semantics and Topology , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[8]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[9]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[10]  Patrik Schmuck,et al.  Multi-UAV collaborative monocular SLAM , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Michael Milford,et al.  LoST? Appearance-Invariant Place Recognition for Opposite Viewpoints using Visual Semantics , 2018, Robotics: Science and Systems.

[12]  Junhao Xiao,et al.  Distributed Monocular Multi-Robot SLAM , 2018, 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER).

[13]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[14]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  James M. Conrad,et al.  A Survey of Implementation of Multi-Robot Simultaneous Localization and Mapping , 2018, SoutheastCon 2018.

[16]  Michael Milford,et al.  Place Recognition with ConvNet Landmarks: Viewpoint-Robust, Condition-Robust, Training-Free , 2015, Robotics: Science and Systems.

[17]  Richard Elvira,et al.  ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM , 2021, IEEE Transactions on Robotics.

[18]  Simon Lacroix,et al.  Location graphs for visual place recognition , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Jingjing Du,et al.  Rao-Blackwellized Particle Filters multi robot SLAM with unknown initial correspondences and limited communication , 2010, 2010 IEEE International Conference on Robotics and Automation.

[20]  Vladlen Koltun,et al.  MSeg: A Composite Dataset for Multi-Domain Semantic Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Roland Siegwart,et al.  Robust Visual Place Recognition with Graph Kernels , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Gim Hee Lee,et al.  PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[24]  Rares Ambrus,et al.  3D Packing for Self-Supervised Monocular Depth Estimation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Michael Milford,et al.  Deep learning features at scale for visual place recognition , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Paul Newman,et al.  FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance , 2008, Int. J. Robotics Res..

[27]  Il Hong Suh,et al.  From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation , 2019, ArXiv.

[28]  Antonio M. López,et al.  The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Jimmy Li,et al.  Semantic Mapping for View-Invariant Relocalization , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[31]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[32]  Shawn D. Newsam,et al.  Improving Semantic Segmentation via Video Propagation and Label Relaxation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Dorian Gálvez-López,et al.  Bags of Binary Words for Fast Place Recognition in Image Sequences , 2012, IEEE Transactions on Robotics.