Underwater place recognition in noisy stereo data using FAB-MAP with a multimodal vocabulary from 2D texture and 3D surface descriptors

One fundamental problem in Simultaneous Localization and Mapping is place recognition, used to create loop closures, necessary to improve map quality. Ideally, the sensor data alone should be used to detect loop closures, referred to as appearance-based methods. Such methods usually employ feature descriptors, especially popular in underwater research are 2D visual features, and the bag-of-words approach. Here we demonstrate the combined use of 2D visual and 3D surface features, which is specifically interesting in the context of 3D underwater mapping with stereo data. Different sensor and feature types excel at different situations, and thus would benefit from a joint treatment in a bag-of-words based place recognition method. Specifically, this paper presents a method to employ multiple dictionaries computed from different feature types, extracted from different raw data in a single FAB-MAP-based place recognition method. Experiments on an underwater dataset show an increase of performance of the method when using a combination of feature types instead of single feature types.

[1]  Jörg Stückler,et al.  SURE: Surface Entropy for Distinctive 3D Features , 2012, Spatial Cognition.

[2]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[3]  Yu Zhong,et al.  Intrinsic shape signatures: A shape descriptor for 3D object recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[4]  R. Eustice,et al.  Large area 3D reconstructions from underwater surveys , 2004, Oceans '04 MTS/IEEE Techno-Ocean '04 (IEEE Cat. No.04CH37600).

[5]  Frank Dellaert,et al.  3D reconstruction of underwater structures , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Barbara Caputo,et al.  Towards robust place recognition for robot localization , 2008, 2008 IEEE International Conference on Robotics and Automation.

[7]  Andreas Birk,et al.  Spectral registration of noisy sonar data for underwater 3D mapping , 2011, Auton. Robots.

[8]  Thomas B. Schön,et al.  Learning to close loops from range data , 2011, Int. J. Robotics Res..

[9]  Barbara Anne am Ende 3D Mapping of Underwater Caves , 2001, IEEE Computer Graphics and Applications.

[10]  Gérard G. Medioni,et al.  Structural Indexing: Efficient 3-D Object Recognition , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Andreas Birk,et al.  Uncertainty estimation for a 6-DoF spectral registration method as basis for sonar-based underwater 3D SLAM , 2012, 2012 IEEE International Conference on Robotics and Automation.

[12]  Andreas Birk,et al.  Representing and solving local and global ambiguities as multimodal and hyperedge constraints in a generalized graph SLAM framework , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Andreas Nüchter,et al.  Automatic appearance-based loop detection from three-dimensional laser data using the normal distributions transform , 2009 .

[14]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[15]  Shahriar Negahdaripour,et al.  Efficient three‐dimensional scene modeling and mosaicing , 2009, J. Field Robotics.

[16]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[17]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[18]  Hans Burkhardt,et al.  Fast Codebook Generation by Sequential Data Analysis for Object Classification , 2007, ISVC.

[19]  Andreas Birk,et al.  On behalf of: Multimedia Archives , 2012 .

[20]  Paul Newman,et al.  Appearance-only SLAM at large scale with FAB-MAP 2.0 , 2011, Int. J. Robotics Res..

[21]  David Wettergreen,et al.  Real‐Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels , 2007, J. Field Robotics.

[22]  Federico Tombari,et al.  A combined texture-shape descriptor for enhanced 3D feature matching , 2011, 2011 18th IEEE International Conference on Image Processing.

[23]  Libor Preucil,et al.  Comparison of shape matching techniques for place recognition , 2013, 2013 European Conference on Mobile Robots.

[24]  Wolfram Burgard,et al.  Point feature extraction on 3D range scans taking into account object boundaries , 2011, 2011 IEEE International Conference on Robotics and Automation.

[25]  Anton van den Hengel,et al.  Thrift: Local 3D Structure Recognition , 2007, 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007).

[26]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[27]  Gordon Wyeth,et al.  OpenFABMAP: An open source toolbox for appearance-based loop closure detection , 2012, 2012 IEEE International Conference on Robotics and Automation.

[28]  Chin Seng Chua,et al.  Point Signatures: A New Representation for 3D Object Recognition , 1997, International Journal of Computer Vision.

[29]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[30]  Donald E. Knuth,et al.  The art of computer programming, volume 3: (2nd ed.) sorting and searching , 1998 .

[31]  Henning Lategahn,et al.  How to learn an illumination robust image feature for place recognition , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[32]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[33]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[34]  Richard Bellman,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[35]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[36]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[37]  Andreas Birk,et al.  Large-scale image mosaicking using multimodal hyperedge constraints from multiple registration methods within the Generalized Graph SLAM framework , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  M. Hebert,et al.  Automatic three-dimensional modeling from reality , 2002 .

[39]  Andreas Birk,et al.  Underwater stereo data acquisition and 3D registration with a spectral method , 2013, 2013 MTS/IEEE OCEANS - Bergen.

[40]  Paul Newman,et al.  FAB-MAP 3D: Topological mapping with spatial and visual appearance , 2010, 2010 IEEE International Conference on Robotics and Automation.

[41]  Paul Newman,et al.  FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance , 2008, Int. J. Robotics Res..

[42]  Mohammed Bennamoun,et al.  On the Repeatability and Quality of Keypoints for Local Feature-based 3D Object Retrieval from Cluttered Scenes , 2009, International Journal of Computer Vision.

[43]  Michael Milford,et al.  Condition-invariant, top-down visual place recognition , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[44]  Huosheng Hu,et al.  3-D-Laser-Based Scene Measurement and Place Recognition for Mobile Robots in Dynamic Indoor Environments , 2013, IEEE Transactions on Instrumentation and Measurement.

[45]  Andrew Hogue,et al.  Underwater 3D SLAM through entropy minimization , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[46]  Wolfram Burgard,et al.  Geometrical FLIRT phrases for large scale place recognition in 2D range data , 2013, 2013 IEEE International Conference on Robotics and Automation.

[47]  Gary R. Bradski,et al.  Learning OpenCV - computer vision with the OpenCV library: software that sees , 2008 .

[48]  Stefan B. Williams,et al.  Simultaneous localisation and mapping on the Great Barrier Reef , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[49]  Reinhard Koch,et al.  3D reconstruction based on underwater video from ROV Kiel 6000 considering underwater imaging conditions , 2009, OCEANS 2009-EUROPE.

[50]  Wolfram Burgard,et al.  Place recognition in 3D scans using a combination of bag of words and point feature based relative pose estimation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[51]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[52]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[53]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[54]  Wolfram Burgard,et al.  Robust place recognition for 3D range data based on point features , 2010, 2010 IEEE International Conference on Robotics and Automation.

[55]  Jack Collier,et al.  Multi-sensor Appearance-Based Place Recognition , 2013, 2013 International Conference on Computer and Robot Vision.

[56]  Frédéric Jurie,et al.  Fast Discriminative Visual Codebooks using Randomized Clustering Forests , 2006, NIPS.

[57]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.