Autonomous Multisensor Calibration and Closed‐loop Fusion for SLAM

The vast majority of current robot mapping and navigation systems require specific well-characterized sensors that may require human-supervised calibration and are applicable only in one type of environment. Furthermore, if a sensor degrades in performance, either through damage to itself or changes in environmental conditions, the effect on the mapping system is usually catastrophic. In contrast, the natural world presents robust, reasonably well-characterized solutions to these problems. Using simple movement behaviors and neural learning mechanisms, rats calibrate their sensors for mapping and navigation in an incredibly diverse range of environments and then go on to adapt to sensor damage and changes in the environment over the course of their lifetimes. In this paper, we introduce similar movement-based autonomous calibration techniques that calibrate place recognition and self-motion processes as well as methods for online multisensor weighting and fusion. We present calibration and mapping results from multiple robot platforms and multisensory configurations in an office building, university campus, and forest. With moderate assumptions and almost no prior knowledge of the robot, sensor suite, or environment, the methods enable the bio-inspired RatSLAM system to generate topologically correct maps in the majority of experiments.

[1]  Gaurav S. Sukhatme,et al.  Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration , 2011, Int. J. Robotics Res..

[2]  J. O’Keefe,et al.  Boundary Vector Cells in the Subiculum of the Hippocampal Formation , 2009, The Journal of Neuroscience.

[3]  Stergios I. Roumeliotis,et al.  A Kalman Filter-Based Algorithm for IMU-Camera Calibration: Observability Analysis and Performance Evaluation , 2008, IEEE Transactions on Robotics.

[4]  Silvia Silva da Costa Botelho,et al.  Sensor fusion based on multi-self-organizing maps for SLAM , 2010, 2010 IEEE Conference on Multisensor Fusion and Integration.

[5]  Gordon Wyeth,et al.  Spatial cognition for robots , 2009, IEEE Robotics & Automation Magazine.

[6]  E. Nebot,et al.  Autonomous Navigation and Map building Using Laser Range Sensors in Outdoor Applications , 2000 .

[7]  Gordon Wyeth,et al.  Hybrid robot control and SLAM for persistent navigation and mapping , 2010, Robotics Auton. Syst..

[8]  Fabio Ramos,et al.  Semi-parametric learning for visual odometry , 2013, Int. J. Robotics Res..

[9]  Janet Wiles,et al.  Solving Navigational Uncertainty Using Grid Cells on Robots , 2010, PLoS Comput. Biol..

[10]  P. Teitelbaum,et al.  "Warm-up" along dimensions of movement in the ontogeny of exploration in rats and other infant mammals. , 1981, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Benjamin Kuipers,et al.  Local metrical and global topological maps in the hybrid spatial semantic hierarchy , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[12]  Tara N. Sainath,et al.  A voice-commandable robotic forklift working alongside humans in minimally-prepared outdoor environments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[13]  Tom Duckett,et al.  A Minimalistic Approach to Appearance-Based Visual SLAM , 2008, IEEE Transactions on Robotics.

[14]  Giovanni Muscato,et al.  Localization and self-calibration of a robot for volcano exploration , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[15]  Paul Newman,et al.  Outdoor SLAM using visual appearance and laser ranging , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[16]  P. Zhang,et al.  Navigation with IMU/GPS/digital compass with unscented Kalman filter , 2005, IEEE International Conference Mechatronics and Automation, 2005.

[17]  Gordon Wyeth,et al.  CAT-SLAM: probabilistic localisation and mapping using a continuous appearance-based trajectory , 2012, Int. J. Robotics Res..

[18]  Gordon Wyeth,et al.  Aerial SLAM with a single camera using visual expectation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[19]  Kurt Konolige,et al.  Incremental mapping of large cyclic environments , 1999, Proceedings 1999 IEEE International Symposium on Computational Intelligence in Robotics and Automation. CIRA'99 (Cat. No.99EX375).

[20]  Paul Newman,et al.  Appearance-only SLAM at large scale with FAB-MAP 2.0 , 2011, Int. J. Robotics Res..

[21]  Takeo Kanade,et al.  Factorization-based calibration method for MEMS inertial measurement unit , 2008, 2008 IEEE International Conference on Robotics and Automation.

[22]  David Raposo,et al.  Dynamic weighting of multisensory stimuli shapes decision-making in rats and humans. , 2013, Journal of vision.

[23]  Paul R. Cohen,et al.  Camera Calibration with Distortion Models and Accuracy Evaluation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Hauke Strasdat,et al.  Scale Drift-Aware Large Scale Monocular SLAM , 2010, Robotics: Science and Systems.

[25]  Michael Milford Robot Navigation from Nature - Simultaneous Localisation, Mapping, and Path Planning based on Hippocampal Models , 2008, Springer Tracts in Advanced Robotics.

[26]  Illah R. Nourbakhsh,et al.  Appearance-based place recognition for topological localization , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[27]  David Ball,et al.  Maintaining a Cognitive Map in Darkness: The Need to Fuse Boundary Knowledge with Path Integration , 2012, PLoS Comput. Biol..

[28]  Friedrich Fraundorfer,et al.  Topological mapping, localization and navigation using image collections , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Gordon Wyeth,et al.  Spatial Cognition for Robots Spatial Cognition for Robots Robot Navigation from Biological Inspiration , 2009 .

[30]  Lina María Paz,et al.  Large-Scale 6-DOF SLAM With Stereo-in-Hand , 2008, IEEE Transactions on Robotics.

[31]  Morton H. Lewin The Standard Library , 1986 .

[32]  Jack Collier,et al.  Multi-sensor Appearance-Based Place Recognition , 2013, 2013 International Conference on Computer and Robot Vision.

[33]  Gordon Wyeth,et al.  Feature-based visual odometry and featureless place recognition for SLAM in 2.5D environments , 2011, ICRA 2011.

[34]  Gerd Hirzinger,et al.  Extrinsic and depth calibration of ToF-cameras , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  E. Save,et al.  Contribution of multiple sensory information to place field stability in hippocampal place cells , 2000, Hippocampus.

[36]  Eduardo Nebot,et al.  Localization and map building using laser range sensors in outdoor applications , 2000, J. Field Robotics.

[37]  Michael Milford,et al.  Brain-inspired sensor fusion for navigating robots , 2013, 2013 IEEE International Conference on Robotics and Automation.

[38]  Giovanni Muscato,et al.  A Robotic System for Volcano Exploration , 2005 .

[39]  Michel Dhome,et al.  Outdoor autonomous navigation using monocular vision , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Sebastian Thrun,et al.  Online self-calibration for mobile robots , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[41]  Michael Milford,et al.  Autonomous movement-driven place recognition calibration for generic multi-sensor robot platforms , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[42]  Benjamin Kuipers,et al.  A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations , 1991, Robotics Auton. Syst..

[43]  Kurt Konolige,et al.  Navigation in hybrid metric-topological maps , 2011, 2011 IEEE International Conference on Robotics and Automation.

[44]  Paul Newman,et al.  Highly scalable appearance-only SLAM - FAB-MAP 2.0 , 2009, Robotics: Science and Systems.

[45]  Jorge Dias,et al.  Relative Pose Calibration Between Visual and Inertial Sensors , 2007, Int. J. Robotics Res..

[46]  Kurt Konolige,et al.  FrameSLAM: From Bundle Adjustment to Real-Time Visual Mapping , 2008, IEEE Transactions on Robotics.

[47]  Richard M. Murray,et al.  Fault detection and isolation from uninterpreted data in robotic sensorimotor cascades , 2012, 2012 IEEE International Conference on Robotics and Automation.

[48]  José A. Castellanos,et al.  Multisensor fusion for simultaneous localization and map building , 2001, IEEE Trans. Robotics Autom..

[49]  William P. Maddern,et al.  Towards robust night and day place recognition using visible and thermal imaging , 2012 .

[50]  Richard M. Murray,et al.  Learning diffeomorphism models of robotic sensorimotor cascades , 2012, 2012 IEEE International Conference on Robotics and Automation.

[51]  Zhanyi Hu,et al.  Camera calibration with moving one-dimensional objects , 2005, Pattern Recognit..

[52]  Gordon Wyeth,et al.  OpenFABMAP: An open source toolbox for appearance-based loop closure detection , 2012, 2012 IEEE International Conference on Robotics and Automation.

[53]  Gordon Wyeth,et al.  Persistent Navigation and Mapping using a Biologically Inspired SLAM System , 2010, Int. J. Robotics Res..

[54]  Kurt Konolige,et al.  The Office Marathon: Robust navigation in an indoor office environment , 2010, 2010 IEEE International Conference on Robotics and Automation.

[55]  Michael Milford,et al.  Towards Brain-based Sensor Fusion for Navigating Robots , 2012, ICRA 2012.

[56]  Hugh F. Durrant-Whyte,et al.  A solution to the simultaneous localization and map building (SLAM) problem , 2001, IEEE Trans. Robotics Autom..

[57]  R. Y. Tsai,et al.  An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision , 1986, CVPR 1986.

[58]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  Gordon Wyeth,et al.  Mapping a Suburb With a Single Camera Using a Biologically Inspired SLAM System , 2008, IEEE Transactions on Robotics.

[60]  Stefano Soatto,et al.  Visual-inertial navigation, mapping and localization: A scalable real-time causal approach , 2011, Int. J. Robotics Res..

[61]  Janet Wiles,et al.  OpenRatSLAM: an open source brain-based SLAM system , 2013, Autonomous Robots.

[62]  J. O’Keefe,et al.  Hippocampal place units in the freely moving rat: Why they fire where they fire , 1978, Experimental Brain Research.

[63]  Tom Duckett,et al.  An adaptive appearance-based map for long-term topological localization of mobile robots , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[64]  Hugh F. Durrant-Whyte,et al.  A computationally efficient solution to the simultaneous localisation and map building (SLAM) problem , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[65]  Zhengyou Zhang,et al.  Camera calibration with one-dimensional objects , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Sebastian Thrun,et al.  The Graph SLAM Algorithm with Applications to Large-Scale Mapping of Urban Structures , 2006, Int. J. Robotics Res..

[67]  R U Muller,et al.  Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[68]  T. Hafting,et al.  Microstructure of a spatial map in the entorhinal cortex , 2005, Nature.

[69]  Sebastian Thrun,et al.  Integrating Grid-Based and Topological Maps for Mobile Robot Navigation , 1996, AAAI/IAAI, Vol. 2.

[70]  Eric E. Thomson,et al.  Perceiving Invisible Light through a Somatosensory Cortical Prosthesis , 2013, Nature Communications.

[71]  Janet Wiles,et al.  The race to learn: Spike timing and STDP can coordinate learning and recall in CA3 , 2011, Hippocampus.

[72]  Janet Wiles,et al.  Using Strategic Movement to Calibrate a Neural Compass: A Spiking Network for Tracking Head Direction in Rats and Robots , 2011, PloS one.

[73]  Jorge Dias,et al.  ROBUST PLACE RECOGNITION WITHIN MULTI-SENSOR VIEW SEQUENCES USING BERNOULLI MIXTURE MODELS , 2007 .

[74]  Paul Newman,et al.  Using laser range data for 3D SLAM in outdoor environments , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[75]  Aram Kawewong,et al.  Online and Incremental Appearance-based SLAM in Highly Dynamic Environments , 2011, Int. J. Robotics Res..

[76]  Raymond Austin Jarvis,et al.  An active visual loop closure detection and validation system for topological SLAM , 2010, ICRA 2010.

[77]  Jana Kosecka,et al.  Vision based topological Markov localization , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[78]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  Paul Timothy Furgale,et al.  Visual teach and repeat for long‐range rover autonomy , 2010, J. Field Robotics.