A Neurologically Inspired Sequence Processing Model for Mobile Robot Place Recognition

Visual place recognition is the problem of camera-based localization of a robot given a database of images of known places, potentially under severe appearance changes (e.g., different weather or illumination). Typically, the current query images and the database images are sequences of consecutive images. In previous work, we proposed to adapt hierarchical temporal memory, a biologically plausible model of sequence processing in the human neocortex to address this task. The previous work described the algorithmic steps and showed synthetic experimental results from simulation. This letter extends the approach to application on real-world data based on a novel encoder for state-of-the-art image processing front ends. The neurologically inspired approach is compared with several state-of-the-art algorithms on a variety of datasets and shows preferable performance. Beyond the place recognition performance, the neurological roots of the algorithm result in appealing properties like potentially very energy efficient implementation due to the usage of sparse distributed representations and natural extendability like the integration of motion estimates similar to entorhinal grid cells. Finally, we underline its practical applicability by online, soft real-time application on a mobile robot.

[1]  Luis Miguel Bergasa,et al.  Towards life-long visual localization using an efficient matching of binary sequences from images , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Niko Sünderhauf,et al.  Are We There Yet? Challenging SeqSLAM on a 3000 km Journey Across All Four Seasons , 2013 .

[3]  Gordon Wyeth,et al.  SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights , 2012, 2012 IEEE International Conference on Robotics and Automation.

[4]  Takeo Kanade,et al.  Visual topometric localization , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[5]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[6]  Daniel Cremers,et al.  Image-Based Localization Using LSTMs for Structured Feature Correlation , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Peter Protzel,et al.  Learning Vector Symbolic Architectures for Reactive Robot Behaviours , 2017 .

[8]  Paul Newman,et al.  1 year, 1000 km: The Oxford RobotCar dataset , 2017, Int. J. Robotics Res..

[9]  Guang-Zhong Yang,et al.  Dynamic scene models for incremental, long-term, appearance-based localisation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[10]  Cyrill Stachniss,et al.  Lazy Data Association For Image Sequences Matching Under Substantial Appearance Changes , 2016, IEEE Robotics and Automation Letters.

[11]  Roddy M. Grieves,et al.  The representation of space in the brain , 2017, Behavioural Processes.

[12]  Peter I. Corke,et al.  Visual Place Recognition: A Survey , 2016, IEEE Transactions on Robotics.

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Wolfram Burgard,et al.  Efficient and effective matching of image sequences under substantial appearance changes exploiting GPS priors , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Gordon Wyeth,et al.  RatSLAM: a hippocampal model for simultaneous localization and mapping , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[16]  Paul Newman,et al.  Detecting Loop Closure with Scene Sequences , 2007, International Journal of Computer Vision.

[17]  Brett Browning,et al.  Visual place recognition using HMM sequence matching , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Yuwei Cui,et al.  A Theory of How Columns in the Neocortex Enable Learning the Structure of the World , 2017, Front. Neural Circuits.

[19]  Scott Purdy Encoding Data for HTM Systems , 2016, ArXiv.

[20]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[21]  Gordon Wyeth,et al.  FAB-MAP + RatSLAM: Appearance-based SLAM for multiple times of day , 2010, 2010 IEEE International Conference on Robotics and Automation.

[22]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[23]  Trevor Cohen,et al.  Reasoning with vectors: A continuous model for fast robust inference , 2015, Log. J. IGPL.

[24]  Niko Sünderhauf,et al.  BRIEF-Gist - closing the loop by simple means , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Sen Wang,et al.  VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Niko Sünderhauf,et al.  On the performance of ConvNet features for place recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[27]  Patrick Pantel,et al.  Randomized Algorithms and NLP: Using Locality Sensitive Hash Functions for High Speed Noun Clustering , 2005, ACL.

[28]  Cyrill Stachniss,et al.  Relocalization under Substantial Appearance Changes using Hashing , 2017 .

[29]  Subutai Ahmad,et al.  A Sequence-Based Neuronal Model for Mobile Robot Localization , 2018, KI.

[30]  Winston Churchill,et al.  Experience-based navigation for long-term localisation , 2013, Int. J. Robotics Res..

[31]  Josef Sivic,et al.  NetVLAD: CNN Architecture for Weakly Supervised Place Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Subutai Ahmad,et al.  Properties of Sparse Distributed Representations and their Application to Hierarchical Temporal Memory , 2015, ArXiv.

[33]  Wolfram Burgard,et al.  Robust Visual Robot Localization Across Seasons Using Network Flows , 2014, AAAI.

[34]  Jan M. Rabaey,et al.  High-Dimensional Computing as a Nanoscalable Paradigm , 2017, IEEE Transactions on Circuits and Systems I: Regular Papers.

[35]  Tomás Pajdla,et al.  NetVLAD: CNN Architecture for Weakly Supervised Place Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[37]  Marcus Lewis,et al.  A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex , 2018, bioRxiv.

[38]  Subutai Ahmad,et al.  Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex , 2015, Front. Neural Circuits.

[39]  Alex Graves,et al.  Associative Long Short-Term Memory , 2016, ICML.

[40]  Paul Newman,et al.  FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance , 2008, Int. J. Robotics Res..

[41]  Peer Neubert,et al.  Beyond Holistic Descriptors, Keypoints, and Fixed Patches: Multiscale Superpixel Grids for Place Recognition in Changing Environments , 2016, IEEE Robotics and Automation Letters.

[42]  Roberto Cipolla,et al.  PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).