3D Image-based Indoor Localization Joint With WiFi Positioning

We realize a system that utilizes WiFi to facilitate the image-based localization system, which avoids the confusion caused by the similar decoration inside the buildings. While WiFi-based localization thread obtains the rough location information, the image-based localization thread retrieves the best matching images and clusters the camera poses associated with the images into different location candidates. The image cluster closest to the WiFi localization outcome is selected for the exact camera pose estimation. The usage of WiFi significantly reduces the search scope, avoiding the extensive search of millions of descriptors in a 3D model. In the image-based localization stage, we also propose a novel 2D-to-2D-to-3D localization framework which follows a coarse-to-fine strategy to quickly locate the query image in several location candidates and performs the local feature matching and camera pose estimation after choosing the correct image location by WiFi positioning. The entire system demonstrates significant benefits in combining both images and WiFi signals in localization tasks and great potential to be deployed in real applications.

[1]  Hugh F. Durrant-Whyte,et al.  Simultaneous map building and localization for an autonomous mobile robot , 1991, Proceedings IROS '91:IEEE/RSJ International Workshop on Intelligent Robots and Systems '91.

[2]  Xun Li,et al.  Image matching techniques for vision-based indoor navigation systems: performance analysis for 3D map based approach , 2012, 2012 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[3]  Paul Newman,et al.  FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance , 2008, Int. J. Robotics Res..

[4]  Jie Yang,et al.  Accurate WiFi Based Localization for Smartphones Using Peer Assistance , 2014, IEEE Transactions on Mobile Computing.

[5]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[6]  Luc Van Gool,et al.  HPAT Indexing for Fast Object/Scene Recognition Based on Local Appearance , 2003, CIVR.

[7]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[8]  Hans-Peter Kriegel,et al.  OPTICS: ordering points to identify the clustering structure , 1999, SIGMOD '99.

[9]  Andrew Owens,et al.  Discrete-continuous optimization for large-scale structure from motion , 2011, CVPR 2011.

[10]  Xun Li,et al.  Image matching techniques for vision-based indoor navigation systems: a 3D map-based approach1 , 2014, J. Locat. Based Serv..

[11]  Torsten Sattler,et al.  Fast image-based localization using direct 2D-to-3D matching , 2011, 2011 International Conference on Computer Vision.

[12]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[13]  Nicu Sebe,et al.  Knowing Where I Am: Exploiting Multi-Task Learning for Multi-view Indoor Image-based Localization , 2014, BMVC.

[14]  Laura Díaz,et al.  A virtual globe tool for searching and visualizing geo-referenced media resources in social networks , 2013, Multimedia Tools and Applications.

[15]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[16]  Dieter Schmalstieg,et al.  Discriminative Feature-to-Point Matching in Image-Based Localization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Torsten Sattler,et al.  Scalable 6-DOF Localization on Mobile Devices , 2014, ECCV.

[18]  Jan-Michael Frahm,et al.  From structure-from-motion point clouds to fast location recognition , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Liviu Iftode,et al.  Indoor Localization Using Camera Phones , 2006, Seventh IEEE Workshop on Mobile Computing Systems & Applications (WMCSA'06 Supplement).

[20]  Luca Carlone,et al.  A convergence analysis for pose graph optimization via Gauss-Newton methods , 2013, 2013 IEEE International Conference on Robotics and Automation.

[21]  Manolis I. A. Lourakis,et al.  SBA: A software package for generic sparse bundle adjustment , 2009, TOMS.

[22]  Wei Zhang,et al.  Image Based Localization in Urban Environments , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[23]  Guoyu Lu,et al.  Improving Image-Based Localization through Increasing Correct Feature Correspondences , 2013, ISVC.

[24]  Changsheng Xu,et al.  GIANT: geo-informative attributes for location recognition and exploration , 2013, ACM Multimedia.

[25]  Robert Pless,et al.  Webcam geo-localization using aggregate light levels , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[26]  Tao Mei,et al.  Finding perfect rendezvous on the go: accurate mobile visual localization and its applications to routing , 2012, ACM Multimedia.

[27]  Daniel P. Huttenlocher,et al.  Location Recognition Using Prioritized Feature Matching , 2010, ECCV.

[28]  Andrew W. Fitzgibbon,et al.  Exploiting uncertainty in regression forests for accurate camera relocalization , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Changsheng Xu,et al.  Interaction Design for Mobile Visual Search , 2013, IEEE Transactions on Multimedia.

[30]  Peter Cheeseman,et al.  On the Representation and Estimation of Spatial Uncertainty , 1986 .

[31]  Roberto Cipolla,et al.  An Image-Based System for Urban Navigation , 2004, BMVC.

[32]  Jana Kosecka,et al.  Qualitative image based localization in indoors environments , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[33]  Bernt Schiele,et al.  How Computer Vision Can Help in Outdoor Positioning , 2007, AmI.

[34]  Jie Yang,et al.  Push the limit of WiFi based localization for smartphones , 2012, Mobicom '12.

[35]  Yan Yan,et al.  A Fast 3D Indoor-Localization Approach Based on Video Queries , 2016, MMM.

[36]  Qian Dong,et al.  Evaluation of the reliability of RSSI for indoor localization , 2012, 2012 International Conference on Wireless Communications in Underground and Confined Areas.

[37]  Kiyoharu Aizawa,et al.  Image-based indoor positioning system: fast image matching using omnidirectional panoramic images , 2010, MPVA '10.

[38]  Andrew W. Fitzgibbon,et al.  Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Nicu Sebe,et al.  Localize Me Anywhere, Anytime: A Multi-task Point-Retrieval Approach , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[40]  David W. Murray,et al.  Video-rate localization in multiple maps for wearable augmented reality , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[41]  M. Fukushima,et al.  Erratum to Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints , 2005 .

[42]  Wen Gao,et al.  Location Discriminative Vocabulary Coding for Mobile Landmark Search , 2011, International Journal of Computer Vision.

[43]  Nicu Sebe,et al.  Indoor localization via multi-view images and videos , 2017, Comput. Vis. Image Underst..

[44]  Xing Xie,et al.  Mining city landmarks from blogs by graph modeling , 2009, ACM Multimedia.