Multi-User Activity Recognition and Tracking Using Commodity WiFi

This paper presents MultiTrack, a commodity WiFi based human sensing system that can track multiple users and recognize activities of multiple users performing them simultaneously. Such a system can enable easy and large-scale deployment for multi-user tracking and sensing without the need for additional sensors through the use of existing WiFi devices (e.g., desktops, laptops and smart appliances). The basic idea is to identify and extract the signal reflection corresponding to each individual user with the help of multiple WiFi links and all the available WiFi channels at 5GHz. Given the extracted signal reflection of each user, MultiTrack examines the path of the reflected signals at multiple links to simultaneously track multiple users. It further reconstructs the signal profile of each user as if only a single user has performed activity in the environment to facilitate multi-user activity recognition. We evaluate MultiTrack in different multipath environments with up to 4 users for multi-user tracking and up to 3 users for activity recognition. Experimental results show that our system can achieve decimeter localization accuracy and over 92% activity recognition accuracy under multi-user scenarios.

[1]  David Wetherall,et al.  Tool release: gathering 802.11n traces with channel state information , 2011, CCRV.

[2]  Muhammad Shahzad,et al.  Position and Orientation Agnostic Gesture Recognition Using WiFi , 2017, MobiSys.

[3]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[4]  Xu Chen,et al.  Tracking Vital Signs During Sleep Leveraging Off-the-shelf WiFi , 2015, MobiHoc.

[5]  Muhammad Shahzad,et al.  Multi-User Gesture Recognition Using WiFi , 2018, MobiSys.

[6]  Swarun Kumar,et al.  Decimeter-Level Localization with a Single WiFi Access Point , 2016, NSDI.

[7]  Dan Wu,et al.  Human respiration detection with commodity wifi devices: do user location and body orientation matter? , 2016, UbiComp.

[8]  Chieko Asakawa,et al.  Environmental Factors in Indoor Navigation Based on Real-World Trajectories of Blind Users , 2018, CHI.

[9]  Wei Wang,et al.  Understanding and Modeling of WiFi Signal Based Human Activity Recognition , 2015, MobiCom.

[10]  Matteo Munaro,et al.  OpenPTrack: Open source multi-camera calibration and people tracking for RGB-D camera networks , 2016, Robotics Auton. Syst..

[11]  Yunhao Liu,et al.  Widar: Decimeter-Level Passive Tracking via Velocity Monitoring with Commodity Wi-Fi , 2017, MobiHoc.

[12]  Xiang Li,et al.  IndoTrack , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[13]  Mikkel Baun Kjærgaard,et al.  DCount - A Probabilistic Algorithm for Accurately Disaggregating Building Occupant Counts into Room Counts , 2018, 2018 19th IEEE International Conference on Mobile Data Management (MDM).

[14]  Jie Yang,et al.  E-eyes: device-free location-oriented activity identification using fine-grained WiFi signatures , 2014, MobiCom.

[15]  Sachin Katti,et al.  WiDeo: Fine-grained Device-free Motion Tracing using RF Backscatter , 2015, NSDI.

[16]  Mike Y. Chen,et al.  TouchRing: subtle and always-available input using a multi-touch ring , 2016, MobileHCI Adjunct.

[17]  J. LaFountain Inc. , 2013, American Art.

[18]  Hui-Shyong Yeo,et al.  WatchMI: pressure touch, twist and pan gesture input on unmodified smartwatches , 2016, MobileHCI.

[19]  Chen Wang,et al.  Low Human-Effort, Device-Free Localization with Fine-Grained Subcarrier Information , 2018, IEEE Transactions on Mobile Computing.

[20]  Yang Zhang,et al.  Wall++: Room-Scale Interactive and Context-Aware Sensing , 2018, CHI.

[21]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[22]  Kaishun Wu,et al.  WiFall: Device-free fall detection by wireless networks , 2017, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[23]  Mohamed Ibrahim,et al.  Visible light based activity sensing using ceiling photosensors , 2016, VLCS '16.

[24]  Zheng Wang,et al.  CrossSense: Towards Cross-Site and Large-Scale WiFi Sensing , 2018, MobiCom.

[25]  Thad Starner,et al.  Using GPS to learn significant locations and predict movement across multiple users , 2003, Personal and Ubiquitous Computing.

[26]  Xiang Li,et al.  Dynamic-MUSIC: accurate device-free indoor localization , 2016, UbiComp.

[27]  Xia Zhou,et al.  Human Sensing Using Visible Light Communication , 2015, MobiCom.

[28]  Yunhao Liu,et al.  Inferring Motion Direction using Commodity Wi-Fi for Interactive Exergames , 2017, CHI.

[29]  Yang Zhang,et al.  Advancing Hand Gesture Recognition with High Resolution Electrical Impedance Tomography , 2016, UIST.

[30]  Mo Li,et al.  Precise Power Delay Profiling with Commodity Wi-Fi , 2015, IEEE Transactions on Mobile Computing.

[31]  Abbas Jamalipour,et al.  Wireless communications , 2005, GLOBECOM '05. IEEE Global Telecommunications Conference, 2005..

[32]  Nicolai Marquardt,et al.  EagleSense: Tracking People and Devices in Interactive Spaces using Real-Time Top-View Depth-Sensing , 2017, CHI.

[33]  Desney S. Tan,et al.  FingerIO: Using Active Sonar for Fine-Grained Finger Tracking , 2016, CHI.

[34]  Kaishun Wu,et al.  We Can Hear You with Wi-Fi! , 2014, IEEE Transactions on Mobile Computing.

[35]  Sheng Tan,et al.  WiFinger: leveraging commodity WiFi for fine-grained finger gesture recognition , 2016, MobiHoc.

[36]  Dina Katabi,et al.  RF-IDraw: virtual touch screen in the air using RF signals , 2014, S3 '14.

[37]  Robert Kovacs,et al.  Level-Ups: Motorized Stilts that Simulate Stair Steps in Virtual Reality , 2015, CHI Extended Abstracts.

[38]  Chenglin Miao,et al.  Towards Environment Independent Device Free Human Activity Recognition , 2018, MobiCom.

[39]  Zhi-Quan Luo,et al.  On the Linear Convergence of the Proximal Gradient Method for Trace Norm Regularization , 2013, NIPS.

[40]  Fadel Adib,et al.  Multi-Person Localization via RF Body Reflections , 2015, NSDI.

[41]  ZhouZimu,et al.  We Can Hear You with Wi-Fi! , 2016 .

[42]  Patrick Baudisch,et al.  Imaginary reality gaming: ball games without a ball , 2013, UIST.

[43]  Bir Bhanu,et al.  Human Activity Recognition in Thermal Infrared Imagery , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[44]  Fadel Adib,et al.  See through walls with WiFi! , 2013, SIGCOMM.

[45]  Dan Morris,et al.  RecoFit: using a wearable sensor to find, recognize, and count repetitive exercises , 2014, CHI.

[46]  Rob Miller,et al.  Smart Homes that Monitor Breathing and Heart Rate , 2015, CHI.

[47]  Mounira Taileb,et al.  Children Tracking System in Indoor and Outdoor Places , 2015, HCI.

[48]  Xinyu Zhang,et al.  Enabling High-Precision Visible Light Localization in Today's Buildings , 2017, MobiSys.

[49]  Hsi-Pin Ma,et al.  A fall detection system based on infrared array sensors with tracking capability for the elderly at home , 2015, 2015 17th International Conference on E-health Networking, Application & Services (HealthCom).

[50]  Kanad K. Biswas,et al.  Gesture recognition using Microsoft Kinect® , 2011, The 5th International Conference on Automation, Robotics and Applications.

[51]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[52]  Mikkel Baun Kjærgaard,et al.  Spatio-temporal facility utilization analysis from exhaustive WiFi monitoring , 2015, PERCOM 2015.