Contactless Continuous Activity Recognition based on Meta-Action Temporal Correlation in Mobile Environments

Continuous activity recognition (CAR) plays an important role in human daily indoor activity monitoring and can be widely used in smart home, human-computer interaction and user authentication. Due to the privacy issue and limited coverage of video signals, RF-based CAR has attracted more and more attention in recent years. This paper focuses on three key problems in RF-based CAR: denoising, segmentation and recognition. We present the design and implementation of a contactless and sensorless continuous activity recognition system, namely WiCheck. Our basic idea is to utilize the temporal correlation between two adjacent actions in continuous activity to eliminate the cumulative error in continuous activity segmentation. Firstly, the multi-layer optimized noise elimination method is used to decrease the environment interference. Secondly, a method based on dual-swing window is proposed to reduce the cumulative error of continuous activity segmentation. Finally, WiCheck is implemented in different indoor environments, and 6 continuous activity sequences are designed to evaluate and analyze the influencing factors. The continuous activity recognition accuracy of WiCheck to two actions and three actions can approach 90% and 75%, respectively.

[1]  Qingming Huang,et al.  Spatial Pyramid-Enhanced NetVLAD With Weighted Triplet Loss for Place Recognition , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Lynne E. Parker,et al.  Fuzzy segmentation and recognition of continuous human activities , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Wei Wang,et al.  Device-Free Human Activity Recognition Using Commercial WiFi Devices , 2017, IEEE Journal on Selected Areas in Communications.

[4]  Jun Yu,et al.  Hierarchical Deep Click Feature Prediction for Fine-Grained Image Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Yunhao Liu,et al.  Design and Implementation of a CSI-Based Ubiquitous Smoking Detection System , 2017, IEEE/ACM Transactions on Networking.

[6]  Dina Katabi,et al.  Extracting Gait Velocity and Stride Length from Surrounding Radio Signals , 2017, CHI.

[7]  Honghao Gao,et al.  Offloading decision methods for multiple users with structured tasks in edge computing for smart cities , 2020, Future Gener. Comput. Syst..

[8]  Larry S. Davis,et al.  Temporal Recurrent Networks for Online Action Detection , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[9]  Nan Yu,et al.  QGesture , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[10]  Yunhao Liu,et al.  Decimeter level passive tracking with wifi , 2016, HotWireless@MobiCom.

[11]  Xuemei Guo,et al.  Online activity recognition and daily habit modeling for solitary elderly through indoor position-based stigmergy , 2018 .

[12]  Jie Xiong,et al.  Phaser: enabling phased array signal processing on commodity WiFi access points , 2014, MobiCom.

[13]  Jake K. Aggarwal,et al.  Segmentation and recognition of continuous human activity , 2001, Proceedings IEEE Workshop on Detection and Recognition of Events in Video.

[14]  Yucong Duan,et al.  Transformation-based processing of typed resources for multimedia sources in the IoT environment , 2019, Wireless Networks.

[15]  Albert Y. Zomaya,et al.  Dynamical Resource Allocation in Edge for Trustable Internet-of-Things Systems: A Reinforcement Learning Method , 2020, IEEE Transactions on Industrial Informatics.

[16]  Rui Li,et al.  Context-Aware QoS Prediction With Neural Collaborative Filtering for Internet-of-Things Services , 2020, IEEE Internet of Things Journal.

[17]  Bingbing Ni,et al.  Progressively Parsing Interactional Objects for Fine Grained Action Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Sheng Tan,et al.  WiFinger: leveraging commodity WiFi for fine-grained finger gesture recognition , 2016, MobiHoc.

[19]  Wang Lin,et al.  Concurrent Recognition of Cross-Scale Activities via Sensorless Sensing , 2019, IEEE Sensors Journal.

[20]  Panlong Yang,et al.  TW-See: Human Activity Recognition Through the Wall With Commodity Wi-Fi Devices , 2019, IEEE Transactions on Vehicular Technology.

[21]  Wei Xi,et al.  FEMO: A Platform for Free-weight Exercise Monitoring with RFIDs , 2015, SenSys.

[22]  Zhou Yu,et al.  Multimodal Transformer With Multi-View Visual Representation for Image Captioning , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Honghao Gao,et al.  V2VR: Reliable Hybrid-Network-Oriented V2V Data Transmission and Routing Considering RSUs and Connectivity Probability , 2020, IEEE Transactions on Intelligent Transportation Systems.

[24]  Bin Guo,et al.  Mining consuming Behaviors with Temporal Evolution for Personalized Recommendation in Mobile Marketing Apps , 2020, Mobile Networks and Applications.

[25]  Sinisa Todorovic,et al.  Temporal Deformable Residual Networks for Action Segmentation in Videos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Wei Wang,et al.  Gait recognition using wifi signals , 2016, UbiComp.

[27]  Xu Chen,et al.  Tracking Vital Signs During Sleep Leveraging Off-the-shelf WiFi , 2015, MobiHoc.

[28]  Yunhao Liu,et al.  PADS: Passive detection of moving targets with dynamic speed using PHY layer information , 2014, 2014 20th IEEE International Conference on Parallel and Distributed Systems (ICPADS).

[29]  Yunhao Liu,et al.  Inferring Motion Direction using Commodity Wi-Fi for Interactive Exergames , 2017, CHI.

[30]  Dina Katabi,et al.  Zero-Effort In-Home Sleep and Insomnia Monitoring using Radio Signals , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[31]  Shan Chang,et al.  $\pi$ -Splicer: Perceiving Accurate CSI Phases with Commodity WiFi Devices , 2018, IEEE Transactions on Mobile Computing.

[32]  Gregory D. Hager,et al.  Temporal Convolutional Networks for Action Segmentation and Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Gerhard Tröster,et al.  Fusion of String-Matched Templates for Continuous Activity Recognition , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[34]  Jonathan Loo,et al.  Continuous authentication of smartphone users based on activity pattern recognition using passive mobile sensing , 2018, J. Netw. Comput. Appl..

[35]  Daqing Zhang,et al.  RT-Fall: A Real-Time and Contactless Fall Detection System with Commodity WiFi Devices , 2017, IEEE Transactions on Mobile Computing.

[36]  Juan Song,et al.  An Online Continuous Human Action Recognition Algorithm Based on the Kinect Sensor , 2016, Sensors.

[37]  Wang Lin,et al.  From Lens to Prism: Device-Free Modeling and Recognition of Multi-Part Activities , 2018, IEEE Access.

[38]  Laurie Davies,et al.  The identification of multiple outliers , 1993 .