InFo: Indoor localization using Fusion of Visual Information from Static and Dynamic Cameras

Localization in an indoor or Global Positioning System (GPS)-denied environment is paramount. It drives various applications that require locating humans or robots in an unknown environment. Various localization systems using different ubiquitous sensors such as camera, radio frequency, inertial measurement unit have been developed. Most of these systems cannot accommodate for scenarios which have substantial changes in the environment such as a large number of people (unpredictable) and sudden change in the environment floor plan (unstructured). In this paper, we propose a system, InFo that can leverage real-time visual information captured by surveillance cameras and augment that with images captured by the smart device user to deliver accurate discretized location information. Through our experiments, we demonstrate that our deep learning based InFo system provides an improvement of 10% as compared to a system that does not utilize this real-time information.

[1]  Torsten Sattler,et al.  InLoc: Indoor Visual Localization with Dense Matching and View Synthesis , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Stefan Poslad,et al.  A BLE RSSI ranking based indoor positioning system for generic smartphones , 2017, 2017 Wireless Telecommunications Symposium (WTS).

[3]  H.T. Friis,et al.  A Note on a Simple Transmission Formula , 1946, Proceedings of the IRE.

[4]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[6]  Tomasz Malisiewicz,et al.  SuperPoint: Self-Supervised Interest Point Detection and Description , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Aaron C. Courville,et al.  FiLM: Visual Reasoning with a General Conditioning Layer , 2017, AAAI.

[8]  Robert Harle,et al.  Location Fingerprinting With Bluetooth Low Energy Beacons , 2015, IEEE Journal on Selected Areas in Communications.

[9]  Jaime Valls Miró,et al.  Gaussian processes online observation classification for RSSI-based low-cost indoor positioning systems , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Torsten Sattler,et al.  Understanding the Limitations of CNN-Based Absolute Camera Pose Regression , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Antonio F. Gómez-Skarmeta,et al.  MagicFinger: 3D Magnetic Fingerprints for Indoor Location , 2015, Sensors.

[12]  Gamini Dissanayake,et al.  Localization system for carers to track elderly people in visits to a crowded shopping mall , 2017, 2017 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[13]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[14]  Yunhao Liu,et al.  Widar2.0: Passive Human Tracking with a Single Wi-Fi Link , 2018, MobiSys.

[15]  Mitesh Patel,et al.  ContextualNet: Exploiting Contextual Information Using LSTMs to Improve Image-Based Localization , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[16]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[17]  Eric Brachmann,et al.  Random forests versus Neural Networks — What's best for camera localization? , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Masatoshi Okutomi,et al.  24/7 Place Recognition by View Synthesis , 2015, CVPR.

[19]  Chao Dong,et al.  Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Craig M. Hancock,et al.  Low-Cost Vision-Based Positioning System , 2018 .

[21]  Aggelos K. Katsaggelos,et al.  Efficient Video Object Segmentation via Network Modulation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Torsten Sattler,et al.  Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Josef Sivic,et al.  NetVLAD: CNN Architecture for Weakly Supervised Place Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Fadel Adib,et al.  Multi-Person Localization via RF Body Reflections , 2015, NSDI.

[27]  Veerachai Malyavej,et al.  Indoor robot localization by RSSI/IMU sensor fusion , 2013, 2013 10th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology.

[28]  Eric Brachmann,et al.  DSAC — Differentiable RANSAC for Camera Localization , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Matthew Cooper,et al.  LoCo: a ready-to-deploy framework for efficient room localization using Wi-Fi , 2014, UbiComp.

[30]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[31]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[32]  Robert Piché,et al.  A Survey of Selected Indoor Positioning Methods for Smartphones , 2017, IEEE Communications Surveys & Tutorials.

[33]  Wolfram Burgard,et al.  Metric Localization with Scale-Invariant Visual Features Using a Single Perspective Camera , 2006, EUROS.

[34]  Ali Taylan Cemgil,et al.  Model-Based Localization and Tracking Using Bluetooth Low-Energy Beacons , 2017, Sensors.

[35]  Nak Yong Ko,et al.  Fusing Range Measurements from Ultrasonic Beacons and a Laser Range Finder for Localization of a Mobile Robot , 2015, Sensors.

[36]  Roberto Cipolla,et al.  PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[37]  Mitesh Patel,et al.  Fusing Map Information with a Probabilistic Sensor Model for Indoor Localization Using RF Beacons , 2018, 2018 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[38]  Honglak Lee,et al.  Exploring the structure of a real-time, arbitrary neural artistic stylization network , 2017, BMVC.

[39]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Eric Brachmann,et al.  Learning Less is More - 6D Camera Localization via 3D Surface Regression , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.