Location-Based Augmented Reality With Pervasive Smartphone Sensors: Inside and Beyond Pokemon Go!

Combining powerful sensors and near ubiquitous distribution, the smartphone has become an irreplaceable part of modern day life. Using its pervasive sensing capabilities, the smart-phone guides us to our destination with precise step-by-step directions, advises us on what to have for lunch, and improves our photography skills through stabilizing our camera. Using the popular augmented reality (AR) smartphone app Pokemon Go as a case study, we explore the world of pervasive sensing. In this paper, we show both the current state of the art that enable applications such as Pokemon Go to thrive, as well as the limitations and opportunities inherent in current pervasive sensing applications.

[1]  Jie Yang,et al.  Accurate WiFi Based Localization for Smartphones Using Peer Assistance , 2014, IEEE Transactions on Mobile Computing.

[2]  Dina Katabi,et al.  RF-IDraw: virtual touch screen in the air using RF signals , 2014, S3@MobiCom.

[3]  Moustafa Youssef,et al.  CoSDEO 2016 Keynote: A decade later — Challenges: Device-free passive localization for wireless environments , 2007, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[4]  Paramvir Bahl,et al.  RADAR: an in-building RF-based user location and tracking system , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[5]  Zhenyu He,et al.  Activity recognition from accelerometer signals based on Wavelet-AR model , 2010, 2010 IEEE International Conference on Progress in Informatics and Computing.

[6]  Yunhao Liu,et al.  ANDMARC: Indoor Location Sensing Using Active RFID , 2003, PerCom.

[7]  Jue Wang,et al.  Dude, where's my card?: RFID positioning that works with multipath and non-line of sight , 2013, SIGCOMM.

[8]  Mamun Bin Ibne Reaz,et al.  A Review of Smart Homes—Past, Present, and Future , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Yunhao Liu,et al.  Wireless Indoor Localization , 2018, Springer Singapore.

[10]  Yunhao Liu,et al.  LANDMARC: Indoor Location Sensing Using Active RFID , 2004, Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. (PerCom 2003)..

[11]  Jihoon Ryoo,et al.  Phase-based Ranging of RFID Tags with Applications to Shopping Cart Localization , 2015, MSWiM.

[12]  Prathima Agrawal,et al.  ARIADNE: a dynamic indoor signal map construction and localization system , 2006, MobiSys '06.

[13]  Lionel M. Ni,et al.  A Survey on Wireless Indoor Localization from the Device Perspective , 2016, ACM Comput. Surv..

[14]  Hal Hodson,et al.  Google's Ingress game is a gold mine for augmented reality , 2012 .

[15]  Ignas Niemegeers,et al.  A survey of indoor positioning systems for wireless personal networks , 2009, IEEE Communications Surveys & Tutorials.

[16]  Jiangchuan Liu,et al.  i2tag: RFID Mobility and Activity Identification Through Intelligent Profiling , 2017, ACM Trans. Intell. Syst. Technol..

[17]  Venkata N. Padmanabhan,et al.  Centaur: locating devices in an office environment , 2012, Mobicom '12.

[18]  Lei Yang,et al.  Anchor-free backscatter positioning for RFID tags with high accuracy , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[19]  Yunhao Liu,et al.  From RSSI to CSI , 2013, ACM Comput. Surv..

[20]  Gérard Dreyfus,et al.  Single-layer learning revisited: a stepwise procedure for building and training a neural network , 1989, NATO Neurocomputing.

[21]  Lei Yang,et al.  Tagoram: real-time tracking of mobile RFID tags to high precision using COTS devices , 2014, MobiCom.

[22]  Moustafa Youssef,et al.  The Horus WLAN location determination system , 2005, MobiSys '05.

[23]  Evangelos Kalogerakis,et al.  RisQ: recognizing smoking gestures with inertial sensors on a wristband , 2014, MobiSys.

[24]  Khaled A. Harras,et al.  Wigest: A Ubiquitous Wifi-based Gesture Recognition System , 2014 .

[25]  Ryan Shea,et al.  Cloud Gaming: Understanding the Support From Advanced Virtualization and Hardware , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Anshul Rai,et al.  Zee: zero-effort crowdsourcing for indoor localization , 2012, Mobicom '12.

[27]  Alanson P. Sample,et al.  RapID: A Framework for Fabricating Low-Latency Interactive Objects with RFID Tags , 2016, CHI.

[28]  Yunhao Liu,et al.  Swadloon: Direction Finding and Indoor Localization Using Acoustic Signal by Shaking Smartphones , 2015, IEEE Transactions on Mobile Computing.

[29]  Wei Xi,et al.  FEMO: A Platform for Free-weight Exercise Monitoring with RFIDs , 2015, SenSys.

[30]  Deepak Ganesan,et al.  BLINK: a high throughput link layer for backscatter communication , 2012, MobiSys '12.

[31]  Antonio Fernández-Caballero,et al.  A survey of video datasets for human action and activity recognition , 2013, Comput. Vis. Image Underst..

[32]  Venkata N. Padmanabhan,et al.  Indoor localization without the pain , 2010, MobiCom.

[33]  Philip Chan,et al.  Toward accurate dynamic time warping in linear time and space , 2007, Intell. Data Anal..

[34]  Wei Cai,et al.  The Future of Cloud Gaming [Point of View] , 2016, Proc. IEEE.

[35]  Jie Yang,et al.  E-eyes: device-free location-oriented activity identification using fine-grained WiFi signatures , 2014, MobiCom.

[36]  Liang Zhou,et al.  Mobile Device-to-Device Video Distribution , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[37]  Yunhao Liu,et al.  VIRE: Active RFID-based Localization Using Virtual Reference Elimination , 2007, 2007 International Conference on Parallel Processing (ICPP 2007).

[38]  Fadel Adib,et al.  See through walls with WiFi! , 2013, SIGCOMM.

[39]  Ryan Shea,et al.  Rhizome: utilizing the public cloud to provide 3D gaming infrastructure , 2015, MMSys.

[40]  P H Veltink,et al.  Detection of static and dynamic activities using uniaxial accelerometers. , 1996, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[41]  David Howard,et al.  A Comparison of Feature Extraction Methods for the Classification of Dynamic Activities From Accelerometer Data , 2009, IEEE Transactions on Biomedical Engineering.

[42]  Ross A. Knepper,et al.  RF-compass: robot object manipulation using RFIDs , 2013, MobiCom.

[43]  Yunhao Liu,et al.  Relative Localization of RFID Tags using Spatial-Temporal Phase Profiling , 2015, NSDI.

[44]  Moeness G. Amin,et al.  Multifrequency-based range estimation of RFID Tags , 2009, 2009 IEEE International Conference on RFID.

[45]  Moustafa Youssef,et al.  Nuzzer: A Large-Scale Device-Free Passive Localization System for Wireless Environments , 2009, IEEE Transactions on Mobile Computing.

[46]  Guobin Shen,et al.  Experiencing and handling the diversity in data density and environmental locality in an indoor positioning service , 2014, MobiCom.

[47]  Bernt Schiele,et al.  A tutorial on human activity recognition using body-worn inertial sensors , 2014, CSUR.

[48]  Shueng-Han Gary Chan,et al.  Contour-based Trilateration for Indoor Fingerprinting Localization , 2015, SenSys.

[49]  F. Gustafsson,et al.  Mobile positioning using wireless networks: possibilities and fundamental limitations based on available wireless network measurements , 2005, IEEE Signal Processing Magazine.

[50]  Paul Lukowicz,et al.  Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Wei Wang,et al.  Moving tag detection via physical layer analysis for large-scale RFID systems , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[52]  Guobin Shen,et al.  BeepBeep: a high accuracy acoustic ranging system using COTS mobile devices , 2007, SenSys '07.

[53]  Gerd Kortuem,et al.  A relative positioning system for co-located mobile devices , 2005, MobiSys '05.