Real-Time Visual Place Recognition for Personal Localization on a Mobile Device

The paper presents an approach to indoor personal localization on a mobile device based on visual place recognition. We implemented on a smartphone two state-of-the-art algorithms that are representative to two different approaches to visual place recognition: FAB-MAP that recognizes places using individual images and ABLE-M that utilizes sequences of images. These algorithms are evaluated in environments of different structure, focusing on problems commonly encountered when a mobile device camera is used. The conclusions drawn from this evaluation are guidelines to design the FastABLE system, which is based on the ABLE-M algorithm but introduces major modifications to the concept of image matching. The improvements radically cut down the processing time and improve scalability, making it possible to localize the user in long image sequences with the limited computing power of a mobile device. The resulting place recognition system compares favorably to both the ABLE-M and the FAB-MAP solutions in the context of real-time personal localization.

[1]  Andrzej Kasiński,et al.  Perception network for the team of indoor mobile robots: concept, architecture, implementation , 2001 .

[2]  Michal R. Nowicki,et al.  Experimental evaluation of visual place recognition algorithms for personal indoor localization , 2016, 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[3]  F. Fraundorfer,et al.  Visual Odometry : Part II: Matching, Robustness, Optimization, and Applications , 2012, IEEE Robotics & Automation Magazine.

[4]  Sergio Ilarri,et al.  A Review of the Role of Sensors in Mobile Context-Aware Recommendation Systems , 2015, Int. J. Distributed Sens. Networks.

[5]  Luis Miguel Bergasa,et al.  OpenABLE: An open-source toolbox for application in life-long visual localization of autonomous vehicles , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[6]  Hao Wang,et al.  Crowdsourcing Based Mobile Location Recognition with Richer Fingerprints from Smartphone Sensors , 2015, 2015 IEEE 21st International Conference on Parallel and Distributed Systems (ICPADS).

[7]  Peter I. Corke,et al.  Visual Place Recognition: A Survey , 2016, IEEE Transactions on Robotics.

[8]  David W. Murray,et al.  Parallel Tracking and Mapping on a camera phone , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[9]  Niko Sünderhauf,et al.  Appearance change prediction for long-term navigation across seasons , 2013, 2013 European Conference on Mobile Robots.

[10]  Michal R. Nowicki,et al.  Performance comparison of point feature detectors and descriptors for visual navigation on Android platform , 2014, 2014 International Wireless Communications and Mobile Computing Conference (IWCMC).

[11]  David G. Lowe,et al.  Scalable Nearest Neighbor Algorithms for High Dimensional Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Xin Yang,et al.  LDB: An ultra-fast feature for scalable Augmented Reality on mobile devices , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[13]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Sebastian Thrun,et al.  Sub-meter indoor localization in unmodified environments with inexpensive sensors , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Marina Meila,et al.  An Accelerated Chow and Liu Algorithm: Fitting Tree Distributions to High-Dimensional Sparse Data , 1999, ICML.

[16]  Michal R. Nowicki,et al.  Adopting the FAB-MAP Algorithm for Indoor Localization with WiFi Fingerprints , 2017, AUTOMATION.

[17]  Paul Newman,et al.  FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance , 2008, Int. J. Robotics Res..

[18]  Michał Nowicki,et al.  WiFi - guided visual loop closure for indoor navigation using mobile devices , 2014 .

[19]  Xudong Jiang,et al.  LBP Encoding Schemes Jointly Utilizing the Information of Current Bit and Other LBP Bits , 2015, IEEE Signal Processing Letters.

[20]  Marc Pollefeys,et al.  Semi-direct EKF-based monocular visual-inertial odometry , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[22]  Gordon Wyeth,et al.  CAT-SLAM: probabilistic localisation and mapping using a continuous appearance-based trajectory , 2012, Int. J. Robotics Res..

[23]  Luigi di Stefano,et al.  Interactive RGB-D SLAM on Mobile Devices , 2014, ACCV Workshops.

[24]  Yang Liu,et al.  Visual loop closure detection with a compact image descriptor , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[26]  Zhe Wang,et al.  Multi-Probe LSH: Efficient Indexing for High-Dimensional Similarity Search , 2007, VLDB.

[27]  Paul Newman,et al.  Accelerating FAB-MAP With Concentration Inequalities , 2010, IEEE Transactions on Robotics.

[28]  Luis Miguel Bergasa,et al.  Towards life-long visual localization using an efficient matching of binary sequences from images , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[29]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[30]  A. Redish.  Beyond the Cognitive Map: From Place Cells to Episodic Memory , 1999 .

[31]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[32]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[33]  Michal Fularz,et al.  Adopting Feature-Based Visual Odometry for Resource-Constrained Mobile Devices , 2014, ICIAR.

[34]  Michal R. Nowicki,et al.  Indoor Navigation with a Smartphone Fusing Inertial and WiFi Data via Factor Graph Optimization , 2015, MobiCASE.

[35]  Paul Newman,et al.  Appearance-only SLAM at large scale with FAB-MAP 2.0 , 2011, Int. J. Robotics Res..

[36]  Paul Newman,et al.  Shady dealings: Robust, long-term visual localisation using illumination invariance , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[37]  Roland Siegwart,et al.  Robust visual inertial odometry using a direct EKF-based approach , 2015, IROS 2015.

[38]  Simon Lacroix,et al.  Vision-Based SLAM: Stereo and Monocular Approaches , 2007, International Journal of Computer Vision.

[39]  Ian D. Reid,et al.  Article in Press Robotics and Autonomous Systems ( ) – Robotics and Autonomous Systems a Comparison of Loop Closing Techniques in Monocular Slam , 2022 .

[40]  Claudio Piciarelli,et al.  Visual Indoor Localization in Known Environments , 2016, IEEE Signal Processing Letters.

[41]  Lin Ma,et al.  Visual location recognition using smartphone sensors for indoor environment , 2015, 2015 10th International Conference for Internet Technology and Secured Transactions (ICITST).

[42]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.