MultiTrack: Multi-User Tracking and Activity Recognition Using Commodity WiFi

This paper presents MultiTrack, a commodity WiFi based human sensing system that can track multiple users and recognize activities of multiple users performing them simultaneously. Such a system can enable easy and large-scale deployment for multi-user tracking and sensing without the need for additional sensors through the use of existing WiFi devices (e.g., desktops, laptops and smart appliances). The basic idea is to identify and extract the signal reflection corresponding to each individual user with the help of multiple WiFi links and all the available WiFi channels at 5GHz. Given the extracted signal reflection of each user, MultiTrack examines the path of the reflected signals at multiple links to simultaneously track multiple users. It further reconstructs the signal profile of each user as if only a single user has performed activity in the environment to facilitate multi-user activity recognition. We evaluate MultiTrack in different multipath environments with up to 4 users for multi-user tracking and up to 3 users for activity recognition. Experimental results show that our system can achieve decimeter localization accuracy and over 92% activity recognition accuracy under multi-user scenarios.

[1]  Dan Morris,et al.  RecoFit: using a wearable sensor to find, recognize, and count repetitive exercises , 2014, CHI.

[2]  Chen Wang,et al.  Low Human-Effort, Device-Free Localization with Fine-Grained Subcarrier Information , 2018, IEEE Transactions on Mobile Computing.

[3]  Chenglin Miao,et al.  Towards Environment Independent Device Free Human Activity Recognition , 2018, MobiCom.

[4]  Dan Wu,et al.  Human respiration detection with commodity wifi devices: do user location and body orientation matter? , 2016, UbiComp.

[5]  Dina Katabi,et al.  RF-IDraw: virtual touch screen in the air using RF signals , 2014, S3@MobiCom.

[6]  Fadel Adib,et al.  Multi-Person Localization via RF Body Reflections , 2015, NSDI.

[7]  Zheng Wang,et al.  CrossSense: Towards Cross-Site and Large-Scale WiFi Sensing , 2018, MobiCom.

[8]  Muhammad Shahzad,et al.  Multi-User Gesture Recognition Using WiFi , 2018, MobiSys.

[9]  Andrea Goldsmith,et al.  Wireless Communications , 2005, 2021 15th International Conference on Advanced Technologies, Systems and Services in Telecommunications (TELSIKS).

[10]  Muhammad Shahzad,et al.  Position and Orientation Agnostic Gesture Recognition Using WiFi , 2017, MobiSys.

[11]  Nicolai Marquardt,et al.  EagleSense: Tracking People and Devices in Interactive Spaces using Real-Time Top-View Depth-Sensing , 2017, CHI.

[12]  Xu Chen,et al.  Tracking Vital Signs During Sleep Leveraging Off-the-shelf WiFi , 2015, MobiHoc.

[13]  Xia Zhou,et al.  Human Sensing Using Visible Light Communication , 2015, MobiCom.

[14]  Mohamed Ibrahim,et al.  Visible light based activity sensing using ceiling photosensors , 2016, VLCS '16.

[15]  Yunhao Liu,et al.  Widar2.0: Passive Human Tracking with a Single Wi-Fi Link , 2018, MobiSys.

[16]  Hui-Shyong Yeo,et al.  WatchMI: pressure touch, twist and pan gesture input on unmodified smartwatches , 2016, MobileHCI.

[17]  Xinyu Zhang,et al.  Enabling High-Precision Visible Light Localization in Today's Buildings , 2017, MobiSys.

[18]  Thad Starner,et al.  Using GPS to learn significant locations and predict movement across multiple users , 2003, Personal and Ubiquitous Computing.

[19]  Mikkel Baun Kjærgaard,et al.  Spatio-temporal facility utilization analysis from exhaustive WiFi monitoring , 2015, PERCOM 2015.

[20]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[21]  Yang Zhang,et al.  Wall++: Room-Scale Interactive and Context-Aware Sensing , 2018, CHI.

[22]  Xiang Li,et al.  IndoTrack , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[23]  David Wetherall,et al.  Tool release: gathering 802.11n traces with channel state information , 2011, CCRV.

[24]  Jue Wang,et al.  RF-IDraw: virtual touch screen in the air using RF signals , 2015, SIGCOMM 2015.

[25]  Kaishun Wu,et al.  WiFall: Device-free fall detection by wireless networks , 2017, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[26]  Patrick Baudisch,et al.  Imaginary reality gaming: ball games without a ball , 2013, UIST.

[27]  Rob Miller,et al.  Smart Homes that Monitor Breathing and Heart Rate , 2015, CHI.

[28]  Sheng Tan,et al.  WiFinger: leveraging commodity WiFi for fine-grained finger gesture recognition , 2016, MobiHoc.

[29]  Fadel Adib,et al.  See through walls with WiFi! , 2013, SIGCOMM.

[30]  Jie Yang,et al.  E-eyes: device-free location-oriented activity identification using fine-grained WiFi signatures , 2014, MobiCom.

[31]  Matteo Munaro,et al.  OpenPTrack: Open source multi-camera calibration and people tracking for RGB-D camera networks , 2016, Robotics Auton. Syst..

[32]  Sachin Katti,et al.  WiDeo: Fine-grained Device-free Motion Tracing using RF Backscatter , 2015, NSDI.

[33]  Mo Li,et al.  Precise Power Delay Profiling with Commodity Wi-Fi , 2015, IEEE Transactions on Mobile Computing.

[34]  Kanad K. Biswas,et al.  Gesture recognition using Microsoft Kinect® , 2011, The 5th International Conference on Automation, Robotics and Applications.

[35]  Chieko Asakawa,et al.  Environmental Factors in Indoor Navigation Based on Real-World Trajectories of Blind Users , 2018, CHI.

[36]  Hsi-Pin Ma,et al.  A fall detection system based on infrared array sensors with tracking capability for the elderly at home , 2015, 2015 17th International Conference on E-health Networking, Application & Services (HealthCom).

[37]  Yunhao Liu,et al.  Inferring Motion Direction using Commodity Wi-Fi for Interactive Exergames , 2017, CHI.

[38]  Bir Bhanu,et al.  Human Activity Recognition in Thermal Infrared Imagery , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[39]  Yunhao Liu,et al.  Widar: Decimeter-Level Passive Tracking via Velocity Monitoring with Commodity Wi-Fi , 2017, MobiHoc.

[40]  Mike Y. Chen,et al.  TouchRing: subtle and always-available input using a multi-touch ring , 2016, MobileHCI Adjunct.

[41]  Desney S. Tan,et al.  FingerIO: Using Active Sonar for Fine-Grained Finger Tracking , 2016, CHI.

[42]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[43]  Wei Wang,et al.  Understanding and Modeling of WiFi Signal Based Human Activity Recognition , 2015, MobiCom.

[44]  Swarun Kumar,et al.  Decimeter-Level Localization with a Single WiFi Access Point , 2016, NSDI.

[45]  Mounira Taileb,et al.  Children Tracking System in Indoor and Outdoor Places , 2015, HCI.

[46]  Kaishun Wu,et al.  We Can Hear You with Wi-Fi! , 2014, IEEE Transactions on Mobile Computing.

[47]  Zhi-Quan Luo,et al.  On the Linear Convergence of the Proximal Gradient Method for Trace Norm Regularization , 2013, NIPS.

[48]  Xiang Li,et al.  Dynamic-MUSIC: accurate device-free indoor localization , 2016, UbiComp.

[49]  Mikkel Baun Kjærgaard,et al.  DCount - A Probabilistic Algorithm for Accurately Disaggregating Building Occupant Counts into Room Counts , 2018, 2018 19th IEEE International Conference on Mobile Data Management (MDM).

[50]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[51]  Yang Zhang,et al.  Advancing Hand Gesture Recognition with High Resolution Electrical Impedance Tomography , 2016, UIST.

[52]  Robert Kovacs,et al.  Level-Ups: Motorized Stilts that Simulate Stair Steps in Virtual Reality , 2015, CHI Extended Abstracts.