Appearance-based mapping and localization for mobile robots using a feature stability histogram

Abstract The strength of appearance-based mapping models for mobile robots lies in their ability to represent the environment through high-level image features and to provide human-readable information. However, developing a mapping and a localization method using these kinds of models is very challenging, especially if robots must deal with long-term mapping, localization, navigation, occlusions, and dynamic environments. In other words, the mobile robot has to deal with environmental appearance change, which modifies its representation of the environment. This paper proposes an indoor appearance-based mapping and a localization method for mobile robots based on the human memory model, which was used to build a Feature Stability Histogram (FSH) at each node in the robot topological map. This FSH registers local feature stability over time through a voting scheme, and the most stable features were considered for mapping, for Bayesian localization and for incrementally updating the current appearance reference view in the topological map. The experimental results are presented using an omnidirectional images dataset acquired over the long-term and considering: illumination changes (time of day, different seasons), occlusions, random removal of features, and perceptual aliasing. The results include a comparison with the approach proposed by Dayoub and Duckett (2008)  [19] and the popular Bag-of-Words (Bazeille and Filliat, 2010)  [35] approach. The obtained results confirm the viability of our method and indicate that it can adapt the internal map representation over time to localize the robot both globally and locally.

[1]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[2]  Patrick Gros,et al.  3D navigation based on a visual memory , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[3]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[4]  Vivek A. Sujan,et al.  Information based indoor environment robotic exploration and modeling using 2-D images and graphs , 2006, Auton. Robots.

[5]  Patrick Rives,et al.  Indoor SLAM based on composite sensor mixing laser scans and omnidirectional images , 2010, 2010 IEEE International Conference on Robotics and Automation.

[6]  Hugh Durrant-Whyte,et al.  Simultaneous Localisation and Mapping ( SLAM ) : Part I The Essential Algorithms , 2006 .

[7]  Jean-Arcady Meyer,et al.  Incremental vision-based topological SLAM , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Tieniu Tan,et al.  Mobile robot self-localization based on global visual appearance features , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[9]  José Santos-Victor,et al.  Toward Robot Perception through Omnidirectional Vision , 2007, Innovations in Intelligent Machines.

[10]  Luc Van Gool,et al.  Omnidirectional Vision Based Topological Navigation , 2007, International Journal of Computer Vision.

[11]  Sinisa Segvic,et al.  A mapping and localization framework for scalable appearance-based navigation , 2009, Comput. Vis. Image Underst..

[12]  Paul Newman,et al.  Detecting Loop Closure with Scene Sequences , 2007, International Journal of Computer Vision.

[13]  A. Baddeley.  Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[14]  Andrew Zisserman,et al.  A Visual Vocabulary for Flower Classification , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Ben J. A. Kröse,et al.  Hierarchical map building and planning based on graph partitioning , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[16]  Tom Duckett,et al.  A Minimalistic Approach to Appearance-Based Visual SLAM , 2008, IEEE Transactions on Robotics.

[17]  Joaquim Salvi,et al.  Appearance-based mapping and localisation using feature stability histograms , 2010 .

[18]  Roland Siegwart,et al.  Real-time monocular visual odometry for on-road vehicles with 1-point RANSAC , 2009, 2009 IEEE International Conference on Robotics and Automation.

[19]  Ben J. A. Kröse,et al.  Appearance-based concurrent map building and localization using a multi-hypotheses tracker , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[20]  Pietro Perona,et al.  Indexing in large scale image collections: Scaling properties and benchmark , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[21]  Wolfram Burgard,et al.  Semantic labeling of places using information extracted from laser and vision sensor data , 2006 .

[22]  Andreas Nüchter,et al.  Automatic Appearance-Based Loop Detection from 3 D Laser Data Using the Normal Distributions Transform , 2009 .

[23]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[24]  Ben J. A. Kröse,et al.  From images to rooms , 2007, Robotics Auton. Syst..

[25]  Horst-Michael Groß,et al.  Omniview-based concurrent map building and localization using adaptive appearance maps , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[26]  El Mustapha Mouaddib,et al.  Range information in omnidirectional images through laser range finder , 2010 .

[27]  Hugh Durrant-Whyte,et al.  Simultaneous localization and mapping (SLAM): part II , 2006 .

[28]  Joachim Hertzberg,et al.  Towards semantic maps for mobile robots , 2008, Robotics Auton. Syst..

[29]  Tom Duckett,et al.  An adaptive appearance-based map for long-term topological localization of mobile robots , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[31]  Richard C. Atkinson,et al.  Human Memory: A Proposed System and its Control Processes , 1968, Psychology of Learning and Motivation.

[32]  H. Durrant-Whyte,et al.  Simultaneous Localisation and Mapping ( SLAM ) : Part II State of the Art , 2006 .

[33]  Luc Van Gool,et al.  From omnidirectional images to hierarchical localization , 2007, Robotics Auton. Syst..

[34]  Masumi Ishikawa,et al.  Real-time appearance-based Monte Carlo localization , 2006, Robotics Auton. Syst..

[35]  Eduardo Mario Nebot,et al.  Recursive scan-matching SLAM , 2007, Robotics Auton. Syst..

[36]  Paul Newman,et al.  Outdoor SLAM using visual appearance and laser ranging , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[37]  Joaquim Salvi,et al.  Appearance-Based SLAM for Mobile Robots , 2009, CCIA.

[38]  B. Castaño,et al.  Desarrollo de una arquitectura para robots móviles autónomos : aplicación a un sistema de navegación topológica , 2011 .

[39]  Ben J. A. Kröse,et al.  Active Appearance-Based Robot Localization Using Stereo Vision , 2005, Auton. Robots.

[40]  Helder Araújo,et al.  Geometric properties of central catadioptric line images and their application in calibration , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  R. Llinás.  I of the Vortex: From Neurons to Self , 2000 .

[42]  Masayuki Inaba,et al.  Visual navigation using omnidirectional view sequence , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[43]  David Filliat,et al.  Combining Odometry and Visual Loop-Closure Detection for Consistent Topo-Metrical Mapping , 2010, RAIRO Oper. Res..