Towards Position-Independent Sensing for Gesture Recognition with Wi-Fi

Past decades have witnessed the extension of the Wi-Fi signals as a useful tool sensing human activities. One common assumption behind it is that there is a one-to-one mapping between human activities and Wi-Fi received signal patterns. However, this assumption does not hold when the user conducts activities in different locations and orientations. Actually, the received signal patterns of the same activity would become inconsistent when the relative location and orientation of the user with respect to transceivers change, leading to unstable sensing performance. This problem is known as the position-dependent problem, hindering the actual deployment of Wi-Fi-based sensing applications. In this paper, to tackle this fundamental problem, we develop a new position-independent sensing strategy and use gesture recognition as an application example to demonstrate its effectiveness. The key idea is to shift our observation from the traditional transceiver view to the hand-oriented view, and extract features that are irrespective of position-specific factors. Following the strategy, we design a position-independent feature, denoted as Motion Navigation Primitive(MNP). MNP captures the pattern of moving direction changes of the hand, which shares consistent patterns when the user performs the same gesture with different position-specific factors. By analyzing the pattern of MNP, we convert gestures into sequences of strokes (e.g, line, arc and corner) which makes them easy to be recognized. We build a prototype WiFi gesture recognition system, i.e., WiGesture to validate the effectiveness of the proposed strategy. Experiments show that our system can outperform the start-of-arts significantly in different settings. Given its novelty and superiority, we believe the proposed method symbolizes a major step towards gesture recognition and would inspire other solutions to position-independent activity recognition in the future.

[1]  Nan Yu,et al.  QGesture , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[2]  Khaled A. Harras,et al.  WiGest: A ubiquitous WiFi-based gesture recognition system , 2014, 2015 IEEE Conference on Computer Communications (INFOCOM).

[3]  Sangki Yun,et al.  Strata: Fine-Grained Acoustic-based Device-Free Tracking , 2017, MobiSys.

[4]  David Wetherall,et al.  Tool release: gathering 802.11n traces with channel state information , 2011, CCRV.

[5]  Bing Chen,et al.  Wi-Wri: Fine-Grained Writing Recognition Using Wi-Fi Signals , 2016, 2016 IEEE Trustcom/BigDataSE/ISPA.

[6]  Kannan Srinivasan,et al.  Mudra: User-friendly Fine-grained Gesture Recognition using WiFi Signals , 2016, CoNEXT.

[7]  Antonio Torralba,et al.  Through-Wall Human Pose Estimation Using Radio Signals , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Antonio Torralba,et al.  RF-based 3D skeletons , 2018, SIGCOMM.

[9]  Muhammad Shahzad,et al.  Multi-User Gesture Recognition Using WiFi , 2018, MobiSys.

[10]  Daqing Zhang,et al.  Understanding WiFi Signal Frequency Features for Position-Independent Gesture Sensing , 2021, IEEE Transactions on Mobile Computing.

[11]  Xiang Li,et al.  WiFit: Ubiquitous Bodyweight Exercise Monitoring with Commodity Wi-Fi Devices , 2018, 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[12]  Yunhao Liu,et al.  Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi , 2019, MobiSys.

[13]  Desney S. Tan,et al.  FingerIO: Using Active Sonar for Fine-Grained Finger Tracking , 2016, CHI.

[14]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[15]  Wei Wang,et al.  Keystroke Recognition Using WiFi Signals , 2015, MobiCom.

[16]  Changsheng Xu,et al.  Latent Support Vector Machine Modeling for Sign Language Recognition with Kinect , 2015, ACM Trans. Intell. Syst. Technol..

[17]  Jian Liu,et al.  Multi - Touch in the Air: Device-Free Finger Tracking and Gesture Recognition via COTS RFID , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[18]  Dan Wu,et al.  Toward Centimeter-Scale Human Activity Sensing with Wi-Fi Signals , 2017, Computer.

[19]  Sheng Tan,et al.  WiFinger: leveraging commodity WiFi for fine-grained finger gesture recognition , 2016, MobiHoc.

[20]  Zhengxiong Li,et al.  E-Eye: Hidden Electronics Recognition through mmWave Nonlinear Effects , 2018, SenSys.

[21]  Beihong Jin,et al.  From Fresnel Diffraction Model to Fine-grained Human Respiration Sensing with Commodity Wi-Fi Devices , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[22]  Dina Katabi,et al.  RF-IDraw: virtual touch screen in the air using RF signals , 2014, S3 '14.

[23]  Dan Wu,et al.  Human respiration detection with commodity wifi devices: do user location and body orientation matter? , 2016, UbiComp.

[24]  Xiang Li,et al.  Boosting fine-grained activity sensing by embracing wireless multipath effects , 2018, CoNEXT.

[25]  Daqing Zhang,et al.  RT-Fall: A Real-Time and Contactless Fall Detection System with Commodity WiFi Devices , 2017, IEEE Transactions on Mobile Computing.

[26]  Yunhao Liu,et al.  Widar: Decimeter-Level Passive Tracking via Velocity Monitoring with Commodity Wi-Fi , 2017, MobiHoc.

[27]  Xiang Li,et al.  Device-Free WiFi Human Sensing: From Pattern-Based to Model-Based Approaches , 2017, IEEE Communications Magazine.

[28]  Wei Wang,et al.  Understanding and Modeling of WiFi Signal Based Human Activity Recognition , 2015, MobiCom.

[29]  Muhammad Shahzad,et al.  Augmenting User Identification with WiFi Based Gesture Recognition , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[30]  Wei Wang,et al.  Device-free gesture tracking using acoustic signals , 2016, MobiCom.

[31]  Muhammad Shahzad,et al.  Position and Orientation Agnostic Gesture Recognition Using WiFi , 2017, MobiSys.

[32]  Dan Wu,et al.  MFDL: A Multicarrier Fresnel Penetration Model based Device-Free Localization System leveraging Commodity Wi-Fi Cards , 2017, ArXiv.

[33]  Dan Wu,et al.  FarSense , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[34]  Dan Wu,et al.  WiDir: walking direction estimation using wireless signals , 2016, UbiComp.

[35]  Sign Language Recognition using Leap Motion Controller , 2015 .

[36]  Wei Wang,et al.  Gait recognition using wifi signals , 2016, UbiComp.

[37]  Kang Huang,et al.  WiTrace: Centimeter-Level Passive Gesture Tracking Using OFDM Signals , 2021, IEEE Transactions on Mobile Computing.

[38]  Daqing Zhang,et al.  WiMorse: A Contactless Morse Code Text Input System Using Ambient WiFi Signals , 2019, IEEE Internet of Things Journal.