Wi-SL: Contactless Fine-Grained Gesture Recognition Uses Channel State Information

In recent years, with the development of wireless sensing technology and the widespread popularity of WiFi devices, human perception based on WiFi has become possible, and gesture recognition has become an active topic in the field of human-computer interaction. As a kind of gesture, sign language is widely used in life. The establishment of an effective sign language recognition system can help people with aphasia and hearing impairment to better interact with the computer and facilitate their daily life. For this reason, this paper proposes a contactless fine-grained gesture recognition method using Channel State Information (CSI), namely Wi-SL. This method uses a commercial WiFi device to establish the correlation mapping between the amplitude and phase difference information of the subcarrier level in the wireless signal and the sign language action, without requiring the user to wear any device. We combine an efficient denoising method to filter environmental interference with an effective selection of optimal subcarriers to reduce the computational cost of the system. We also use K-means combined with a Bagging algorithm to optimize the Support Vector Machine (SVM) classification (KSB) model to enhance the classification of sign language action data. We implemented the algorithms and evaluated them for three different scenarios. The experimental results show that the average accuracy of Wi-SL gesture recognition can reach 95.8%, which realizes device-free, non-invasive, high-precision sign language gesture recognition.

[1]  Xiaochao Dang,et al.  Endpoints-Clipping CSI Amplitude for SVM-Based Indoor Localization , 2019, Sensors.

[2]  Fangmin Li,et al.  WiGeR: WiFi-Based Gesture Recognition System , 2016, ISPRS Int. J. Geo Inf..

[3]  Tanima Dutta,et al.  A Continuous Hand Gestures Recognition Technique for Human-Machine Interaction Using Accelerometer and Gyroscope Sensors , 2016, IEEE Sensors Journal.

[4]  Panlong Yang,et al.  TW-See: Human Activity Recognition Through the Wall With Commodity Wi-Fi Devices , 2019, IEEE Transactions on Vehicular Technology.

[5]  Jiacheng Shang,et al.  A Robust Sign Language Recognition System with Multiple Wi-Fi Devices , 2017, MobiArch@SIGCOMM.

[6]  Soon Ae Chun,et al.  Smart SecureGov: mobile government security framework , 2014, DG.O.

[7]  Shan Liu,et al.  Research on 3D Positioning Indoor Simulation Based on RSSI , 2014 .

[8]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[9]  Kongqiao Wang,et al.  A Framework for Hand Gesture Recognition Based on Accelerometer and EMG Sensors , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[10]  Pyeong-Gook Jung,et al.  A Wearable Gesture Recognition Device for Detecting Muscular Activities Based on Air-Pressure Sensors , 2015, IEEE Transactions on Industrial Informatics.

[11]  Yongsen Ma,et al.  WiFi Sensing with Channel State Information , 2019, ACM Comput. Surv..

[12]  Muhammad Shahzad,et al.  Position and Orientation Agnostic Gesture Recognition Using WiFi , 2017, MobiSys.

[13]  Philipp Bolliger,et al.  Redpin - adaptive, zero-configuration indoor localization through user collaboration , 2008, MELT '08.

[14]  Tong Zhao,et al.  A Robust Passive Intrusion Detection System with Commodity WiFi Devices , 2018, J. Sensors.

[15]  Jilin Zhou,et al.  A New Hand-Measurement Method to Simplify Calibration in CyberGlove-Based Virtual Rehabilitation , 2010, IEEE Transactions on Instrumentation and Measurement.

[16]  Yubo Yan,et al.  Motion-Fi$^+$+: Recognizing and Counting Repetitive Motions With Wireless Backscattering , 2021, IEEE Trans. Mob. Comput..

[17]  Dietrich von Rosen,et al.  On the efficiency of bootstrap method into the analysis contingency table , 2011, Comput. Methods Programs Biomed..

[18]  Zhu Wang,et al.  FreeSense: human-behavior understanding using Wi-Fi signals , 2018, J. Ambient Intell. Humaniz. Comput..

[19]  Tom Minka,et al.  You are facing the Mona Lisa: spot localization using PHY layer information , 2012, MobiSys '12.

[20]  Jie Xiong,et al.  Phaser: enabling phased array signal processing on commodity WiFi access points , 2014, MobiCom.

[21]  Yang Xu,et al.  WiFinger: talk to your smart devices with finger-grained gesture , 2016, UbiComp.

[22]  Kaishun Wu,et al.  WiFall: Device-free fall detection by wireless networks , 2017, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[23]  Khaled A. Harras,et al.  WiGest: A ubiquitous WiFi-based gesture recognition system , 2014, 2015 IEEE Conference on Computer Communications (INFOCOM).

[24]  Lei Yang,et al.  Analysis and Design of an Effective Light Interference Methane Sensor Based on Three-Dimensional Optical Path Model , 2018, J. Sensors.

[25]  Wei Wang,et al.  Understanding and Modeling of WiFi Signal Based Human Activity Recognition , 2015, MobiCom.

[26]  Xiaolong Yang,et al.  WiCatch: A Wi-Fi Based Hand Gesture Recognition System , 2018, IEEE Access.

[27]  Fadel Adib,et al.  Multi-Person Motion Tracking via RF Body Reflections , 2014 .

[28]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[29]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[30]  Mo Li,et al.  Precise Power Delay Profiling with Commodity Wi-Fi , 2015, IEEE Transactions on Mobile Computing.

[31]  Wang Lin,et al.  From Lens to Prism: Device-Free Modeling and Recognition of Multi-Part Activities , 2018, IEEE Access.

[32]  Kaishun Wu,et al.  WiG: WiFi-Based Gesture Recognition System , 2015, 2015 24th International Conference on Computer Communication and Networks (ICCCN).

[33]  Hasmath Farhana Thariq Ahmed,et al.  Device free human gesture recognition using Wi-Fi CSI: A survey , 2020, Eng. Appl. Artif. Intell..

[34]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[35]  Marcin Gabryel,et al.  Evolutionary Methods for Designing Neuro-fuzzy Modular Systems Combined by Bagging Algorithm , 2008, ICAISC.

[36]  Daqing Zhang,et al.  WiMorse: A Contactless Morse Code Text Input System Using Ambient WiFi Signals , 2019, IEEE Internet of Things Journal.

[37]  Neena Damodaran,et al.  Device free human activity and fall recognition using WiFi channel state information (CSI) , 2020, CCF Transactions on Pervasive Computing and Interaction.

[38]  Xiaochao Dang,et al.  A Novel Passive Indoor Localization Method by Fusion CSI Amplitude and Phase Information , 2019, Sensors.

[39]  Mohammadreza Balouchestani,et al.  Effective Low-Power Wearable Wireless Surface EMG Sensor Design Based on Analog-Compressed Sensing , 2014, Sensors.

[40]  Jianwei Zhang,et al.  Multi-LeapMotion sensor based demonstration for robotic refine tabletop object manipulation task , 2016, CAAI Trans. Intell. Technol..

[41]  Zhu Wang,et al.  FreeSense , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[42]  Xiaochao Dang,et al.  A Device-Free Indoor Localization Method Using CSI with Wi-Fi Signals , 2019, Sensors.

[43]  Zhu Wang,et al.  Wi-Fi CSI-Based Behavior Recognition: From Signals and Actions to Activities , 2017, IEEE Communications Magazine.