RGB-W: When Vision Meets Wireless

Inspired by the recent success of RGB-D cameras, we propose the enrichment of RGB data with an additional "quasi-free" modality, namely, the wireless signal (e.g., wifi or Bluetooth) emitted by individuals' cell phones, referred to as RGB-W. The received signal strength acts as a rough proxy for depth and a reliable cue on their identity. Although the measured signals are highly noisy (more than 2m average localization error), we demonstrate that the combination of visual and wireless data significantly improves the localization accuracy. We introduce a novel image-driven representation of wireless data which embeds all received signals onto a single image. We then indicate the ability of this additional data to (i) locate persons within a sparsity-driven framework and to (ii) track individuals with a new confidence measure on the data association problem. Our solution outperforms existing localization methods by a significant margin. It can be applied to the millions of currently installed RGB cameras to better analyze human behavior and offer the next generation of high-accuracy location-based services.

[1]  Vincent W. S. Wong,et al.  Concentric Anchor Beacon Localization Algorithm for Wireless Sensor Networks , 2007, IEEE Transactions on Vehicular Technology.

[2]  Luc Van Gool,et al.  Pedestrian detection at 100 frames per second , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Eric Horvitz,et al.  LOCADIO: inferring motion and location from Wi-Fi signal strengths , 2004, The First Annual International Conference on Mobile and Ubiquitous Systems: Networking and Services, 2004. MOBIQUITOUS 2004..

[4]  David A. McAllester,et al.  Cascade object detection with deformable part models , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Yannick Boursier,et al.  Sparsity Driven People Localization with a Heterogeneous Network of Cameras , 2011, Journal of Mathematical Imaging and Vision.

[6]  Günther Retscher,et al.  Location determination using WiFi fingerprinting versus WiFi trilateration , 2007, J. Locat. Based Serv..

[7]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[8]  Andrew G. Dempster,et al.  WiFi fingerprinting signal strength error modeling for short distances , 2012, 2012 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[9]  Paul A. Zandbergen,et al.  Accuracy of iPhone Locations: A Comparison of Assisted GPS, WiFi and Cellular Positioning , 2009 .

[10]  John Krumm,et al.  Accuracy characterization for metropolitan-scale Wi-Fi localization , 2005, MobiSys '05.

[11]  Richard M. Karp,et al.  A n^5/2 Algorithm for Maximum Matchings in Bipartite Graphs , 1971, SWAT.

[12]  Soura Dasgupta,et al.  Distance estimation from received signal strength under log-normal shadowing: Bias and variance , 2008 .

[13]  Gaetano Borriello,et al.  Location Systems for Ubiquitous Computing , 2001, Computer.

[14]  Óscar Cánovas Reverte,et al.  Using SIFT and WiFi Signals to Provide Location-Based Services for Smartphones , 2011, MobiQuitous.

[15]  Pascal Fua,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Multiple Object Tracking Using K-shortest Paths Optimization , 2022 .

[16]  Manuela M. Veloso,et al.  WiFi localization and navigation for autonomous indoor mobile robots , 2010, 2010 IEEE International Conference on Robotics and Automation.

[17]  Paramvir Bahl,et al.  RADAR: an in-building RF-based user location and tracking system , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[18]  Pierre Vandergheynst,et al.  SCOOP: A Real-Time Sparsity Driven People Localization Algorithm , 2012, Journal of Mathematical Imaging and Vision.

[19]  Yael Moses,et al.  Homography based multiple camera detection and tracking of people in a dense crowd , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Frank Dellaert,et al.  MCMC-based particle filtering for tracking a variable number of interacting targets , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Jie Yang,et al.  Push the limit of WiFi based localization for smartphones , 2012, Mobicom '12.

[22]  Mubarak Shah,et al.  Tracking Multiple Occluding People by Localizing on Multiple Scene Planes , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Richard M. Karp,et al.  A n^5/2 Algorithm for Maximum Matchings in Bipartite Graphs , 1971, SWAT.

[24]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Fei-Fei Li,et al.  Socially-Aware Large-Scale Crowd Forecasting , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Stefan Carlsson,et al.  Multi-Target Tracking - Linking Identities using Bayesian Network Inference , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[27]  Neil D. Lawrence,et al.  WiFi-SLAM Using Gaussian Process Latent Variable Models , 2007, IJCAI.

[28]  Fatih Murat Porikli,et al.  Pedestrian Detection via Classification on Riemannian Manifolds , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[30]  Manuela M. Veloso,et al.  Depth camera based indoor mobile robot localization and navigation , 2012, 2012 IEEE International Conference on Robotics and Automation.

[31]  Zhi Ding,et al.  Distance Estimation From Received Signal Strength Under Log-Normal Shadowing: Bias and Variance , 2008, IEEE Signal Processing Letters.

[32]  Noel E. O'Connor,et al.  Dual-sensor fusion for indoor user localisation , 2011, MM '11.

[33]  Bodo Rosenhahn,et al.  Everybody needs somebody: Modeling social and grouping behavior on a linear programming multiple people tracker , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[34]  Dieter Fox,et al.  Large-Scale Localization from Wireless Signal Strength , 2005, AAAI.

[35]  Daniel Hoffman,et al.  Range-free sensor localisation with ring overlapping based on comparison of received signal strength indicator , 2007, Int. J. Sens. Networks.

[36]  Christophe De Vleeschouwer,et al.  Detection and recognition of sports(wo)men from multiple views , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[37]  Dariu Gavrila,et al.  Monocular Pedestrian Detection: Survey and Experiments , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Kiyoharu Aizawa,et al.  Tracking Persons using Particle Filter Fusing Visual and Wi-Fi Localizations for Widely Distributed Camera , 2007, 2007 IEEE International Conference on Image Processing.