Detecting State Changes of Indoor Everyday Objects using Wi-Fi Channel State Information

Detecting the events of indoor everyday objects such as door or window open/close events has been actively studied to implement such applications as intrusion detection, adaptive HVAC control, and monitoring an independently living elderly person. This study proposes a method for detecting the events and states of indoor everyday objects such as doors and windows without using distributed sensors attached to the objects. In this study, we achieve practical and unobtrusive event detection using a commodity Wi-Fi access point and a computer equipped with a commodity Wi-Fi module. Specifically, we detect the events using Wi-Fi channel state information (CSI), which describes how a signal propagates from a transmitter to a receiver, and is affected by such events. To handle CSI data that consists of the mixed effects of multiple indoor objects in an environment of interest, we employ independent component analysis to separate the events caused by the objects. The decomposed data are then fed into our event classifier based on convolutional and recurrent neural networks to automatically extract features from CSI data, as it is difficult to intuitively design features to be extracted from the CSI data. Moreover, we correct the neural network estimates by incorporating knowledge about the state transitions of an object using hidden Markov models. For example, because the “open” event of a door occurs only when the door is in a “closed” state. We correct impossible state transitions estimated by the neural network based on this knowledge.

[1]  Zhiyuan Liu,et al.  A C-LSTM Neural Network for Text Classification , 2015, ArXiv.

[2]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[3]  Gregory D. Abowd,et al.  Detecting Human Movement by Differential Air Pressure Sensing in HVAC System Ductwork: An Exploration in Infrastructure Mediated Sensing , 2009, Pervasive.

[4]  Sunny Consolvo,et al.  Some Assembly Required: Supporting End-User Sensor Installation in Domestic Ubiquitous Computing Environments , 2004, UbiComp.

[5]  Wei Wang,et al.  Understanding and Modeling of WiFi Signal Based Human Activity Recognition , 2015, MobiCom.

[6]  Takuya Maekawa,et al.  Transferring positioning model for device-free passive indoor localization , 2015, UbiComp.

[7]  Yi Wang,et al.  Robust Indoor Human Activity Recognition Using Wireless Signals , 2015, Sensors.

[8]  Kaishun Wu,et al.  WiFall: Device-free fall detection by wireless networks , 2017, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[9]  David Wetherall,et al.  Tool release: gathering 802.11n traces with channel state information , 2011, CCRV.

[10]  Parth H. Pathak,et al.  Monitoring building door events using barometer sensor in smartphones , 2015, UbiComp.

[11]  Jie Yang,et al.  E-eyes: device-free location-oriented activity identification using fine-grained WiFi signatures , 2014, MobiCom.

[12]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[13]  Sanjay Jha,et al.  CSI-MIMO: An efficient Wi-Fi fingerprinting using Channel State Information with MIMO , 2015, Pervasive Mob. Comput..

[14]  Bernhard Schölkopf,et al.  Introduction to Semi-Supervised Learning , 2006, Semi-Supervised Learning.

[15]  Tom Minka,et al.  You are facing the Mona Lisa: spot localization using PHY layer information , 2012, MobiSys '12.

[16]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[17]  Jing Liu,et al.  Survey of Wireless Indoor Positioning Techniques and Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[18]  Gwenn Englebienne,et al.  Accurate activity recognition in a home setting , 2008, UbiComp.

[19]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[20]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[21]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[22]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[25]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[26]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[27]  Moustafa Youssef,et al.  MonoPHY: Mono-stream-based device-free WLAN localization via physical layer information , 2013, 2013 IEEE Wireless Communications and Networking Conference (WCNC).

[28]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[29]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[30]  Kaishun Wu,et al.  CSI-Based Indoor Localization , 2013, IEEE Transactions on Parallel and Distributed Systems.

[31]  Henry A. Kautz,et al.  Inferring activities from interactions with objects , 2004, IEEE Pervasive Computing.

[32]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[33]  Ignas Niemegeers,et al.  A survey of indoor positioning systems for wireless personal networks , 2009, IEEE Communications Surveys & Tutorials.

[34]  Richard Howard,et al.  Improving RF-based device-free passive localization in cluttered indoor environments through probabilistic classification methods , 2012, 2012 ACM/IEEE 11th International Conference on Information Processing in Sensor Networks (IPSN).

[35]  Matthew J. Johnson,et al.  Bayesian nonparametric hidden semi-Markov models , 2012, J. Mach. Learn. Res..

[36]  Rob Miller,et al.  3D Tracking via Body Radio Reflections , 2014, NSDI.

[37]  Khaled A. Harras,et al.  UbiBreathe: A Ubiquitous non-Invasive WiFi-based Breathing Estimator , 2015, MobiHoc.

[38]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[39]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[40]  Moustafa Youssef,et al.  Nuzzer: A Large-Scale Device-Free Passive Localization System for Wireless Environments , 2009, IEEE Transactions on Mobile Computing.

[41]  Wei Wang,et al.  Keystroke Recognition Using WiFi Signals , 2015, MobiCom.

[42]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Shiwen Mao,et al.  PhaseFi: Phase Fingerprinting for Indoor Localization with a Deep Learning Approach , 2014, 2015 IEEE Global Communications Conference (GLOBECOM).

[44]  Moustafa Youssef,et al.  CoSDEO 2016 Keynote: A decade later — Challenges: Device-free passive localization for wireless environments , 2016, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[45]  Eric A. Wan,et al.  MobileRF: a robust device-free tracking system based on a hybrid neural network HMM classifier , 2014, UbiComp.

[46]  Shaojie Tang,et al.  Wi-Sleep: Contactless Sleep Monitoring via WiFi Signals , 2014, 2014 IEEE Real-Time Systems Symposium.

[47]  Yusheng Ji,et al.  Passive detection of situations from ambient FM-radio signals , 2012, UbiComp.

[48]  Prabal Dutta,et al.  An energy-harvesting sensor architecture and toolkit for building monitoring and event detection , 2014, BuildSys@SenSys.

[49]  Ricardo Chavarriaga,et al.  The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition , 2013, Pattern Recognit. Lett..

[50]  G. Miller Learning to Forget , 2004, Science.

[51]  Shiwen Mao,et al.  DeepFi: Deep learning for indoor fingerprinting using channel state information , 2015, 2015 IEEE Wireless Communications and Networking Conference (WCNC).

[52]  Xiaoli Li,et al.  Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition , 2015, IJCAI.

[53]  Daqing Zhang,et al.  Anti-fall: A Non-intrusive and Real-Time Fall Detector Leveraging CSI from Commodity WiFi Devices , 2015, ICOST.

[54]  Gerhard Tröster,et al.  The telepathic phone: Frictionless activity recognition from WiFi-RSSI , 2014, 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[55]  Bill N. Schilit,et al.  Place Lab: Device Positioning Using Radio Beacons in the Wild , 2005, Pervasive.

[56]  Mudhakar Srivatsa,et al.  Idea: A System for Efficient Failure Management in Smart IoT Environments , 2016, MobiSys.