Wireless Sensing for Human Activity: A Survey

With the advancement of wireless technologies and sensing methodologies, many studies have shown the success of re-using wireless signals (e.g., WiFi) to sense human activities and thereby realize a set of emerging applications, ranging from intrusion detection, daily activity recognition, gesture recognition to vital signs monitoring and user identification involving even finer-grained motion sensing. These applications arguably can brace various domains for smart home and office environments, including safety protection, well-being monitoring/management, smart healthcare and smart-appliance interaction. The movements of the human body impact the wireless signal propagation (e.g., reflection, diffraction and scattering), which provide great opportunities to capture human motions by analyzing the received wireless signals. Researchers take the advantage of the existing wireless links among mobile/smart devices (e.g., laptops, smartphones, smart thermostats, smart refrigerators and virtual assistance systems) by either extracting the ready-to-use signal measurements or adopting frequency modulated signals to detect the frequency shift. Due to the low-cost and non-intrusive sensing nature, wireless-based human activity sensing has drawn considerable attention and become a prominent research field over the past decade. In this paper, we survey the existing wireless sensing systems in terms of their basic principles, techniques and system structures. Particularly, we describe how the wireless signals could be utilized to facilitate an array of applications including intrusion detection, room occupancy monitoring, daily activity recognition, gesture recognition, vital signs monitoring, user identification and indoor localization. The future research directions and limitations of using wireless signals for human activity sensing are also discussed.

[1]  Suresh Venkatasubramanian,et al.  Multiple Target Tracking with RF Sensor Networks , 2013, IEEE Transactions on Mobile Computing.

[2]  Dirk Pesch,et al.  FallDeFi , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[3]  Laura Anitori,et al.  FMCW radar for life-sign detection , 2009, 2009 IEEE Radar Conference.

[4]  Tomoaki Ohtsuki,et al.  Falling detection using multiple doppler sensors , 2012, 2012 IEEE 14th International Conference on e-Health Networking, Applications and Services (Healthcom).

[5]  Fei Li,et al.  An Indoor Continuous Positioning Algorithm on the Move by Fusing Sensors and Wi-Fi on Smartphones , 2015, Sensors.

[6]  Tomoaki Ohtsuki,et al.  Human activity classification and localization using bistatic three frequency CW radar , 2013, 2013 IEEE International Conference on Communications (ICC).

[7]  Khaled A. Harras,et al.  WiGest demo: A ubiquitous WiFi-based gesture recognition system , 2015, 2015 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[8]  Moustafa Youssef,et al.  RASID: A robust WLAN device-free passive motion detection system , 2011, 2012 IEEE International Conference on Pervasive Computing and Communications.

[9]  Jin-xiang Wang Research and implementation of intrusion detection algorithm in video surveillance , 2016, 2016 International Conference on Audio, Language and Image Processing (ICALIP).

[10]  Daqing Zhang,et al.  RT-Fall: A Real-Time and Contactless Fall Detection System with Commodity WiFi Devices , 2017, IEEE Transactions on Mobile Computing.

[11]  Dan Wu,et al.  Human respiration detection with commodity wifi devices: do user location and body orientation matter? , 2016, UbiComp.

[12]  Fadel Adib,et al.  Multi-Person Localization via RF Body Reflections , 2015, NSDI.

[13]  Tomoaki Ohtsuki,et al.  Signal Eigenvector-Based Device-Free Passive Localization Using Array Sensor , 2015, IEEE Transactions on Vehicular Technology.

[14]  Wei Wang,et al.  Understanding and Modeling of WiFi Signal Based Human Activity Recognition , 2015, MobiCom.

[15]  Yunhao Liu,et al.  Non-Invasive Detection of Moving and Stationary Human With WiFi , 2015, IEEE Journal on Selected Areas in Communications.

[16]  Moustafa Youssef,et al.  Smart cevices for smart environments: Device-free passive detection in real environments , 2009, 2009 IEEE International Conference on Pervasive Computing and Communications.

[17]  A. Stelzer,et al.  A 77-GHz FMCW MIMO Radar Based on an SiGe Single-Chip Transceiver , 2009, IEEE Transactions on Microwave Theory and Techniques.

[18]  Richard Howard,et al.  SCPL: Indoor device-free multi-subject counting and localization using radio signal strength , 2013, 2013 ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[19]  Ismail Guvenc,et al.  Enhancements to RSS Based Indoor Tracking Systems Using Kalman Filters , 2003 .

[20]  Myoung An,et al.  Cross-ambiguity function , 1998 .

[21]  Venkata N. Padmanabhan,et al.  Indoor localization without the pain , 2010, MobiCom.

[22]  Dan Zhang,et al.  FMCW Radar for Small Displacement Detection of Vital Signal Using Projection Matrix Method , 2013 .

[23]  Xu Chen,et al.  Monitoring Vital Signs and Postures During Sleep Using WiFi Signals , 2018, IEEE Internet of Things Journal.

[24]  Min Gao,et al.  FILA: Fine-grained indoor localization , 2012, 2012 Proceedings IEEE INFOCOM.

[25]  Graeme E. Smith,et al.  Through-the-Wall Sensing of Personnel Using Passive Bistatic WiFi Radar at Standoff Distances , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Xiaonan Guo,et al.  WiFi-Enabled Smart Human Dynamics Monitoring , 2017, SenSys.

[27]  Yan Yu,et al.  Robust Device-Free Wireless Localization Based on Differential RSS Measurements , 2013, IEEE Transactions on Industrial Electronics.

[28]  Shambhu Upadhyaya,et al.  Is RSSI a Reliable Parameter in Sensor Localization Algorithms – An Experimental Study , 2009 .

[29]  Yusheng Ji,et al.  RF-Sensing of Activities from Non-Cooperative Subjects in Device-Free Recognition Systems Using Ambient and Local Signals , 2014, IEEE Transactions on Mobile Computing.

[30]  Umesh Chandra Pati,et al.  IoT based intrusion detection system using PIR sensor , 2017, 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT).

[31]  Jiang Long,et al.  Instrument-Based Noncontact Doppler Radar Vital Sign Detection System Using Heterodyne Digital Quadrature Demodulation Architecture , 2010, IEEE Transactions on Instrumentation and Measurement.

[32]  R. Weigel,et al.  A Reconfigurable MIMO System for High-Precision FMCW Local Positioning , 2011, IEEE Transactions on Microwave Theory and Techniques.

[33]  Paramvir Bahl,et al.  RADAR: an in-building RF-based user location and tracking system , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[34]  Pavlo Molchanov,et al.  Short-range FMCW monopulse radar for hand-gesture sensing , 2015, 2015 IEEE Radar Conference (RadarCon).

[35]  Karl Woodbridge,et al.  Awireless passive radar system for real-time through-wall movement detection , 2016, IEEE Transactions on Aerospace and Electronic Systems.

[36]  Karl Woodbridge,et al.  A real-time high resolution passive WiFi Doppler-radar and its applications , 2014, 2014 International Radar Conference.

[37]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[38]  Hongbo Liu,et al.  Smart User Authentication through Actuation of Daily Activities Leveraging WiFi-enabled IoT , 2017, MobiHoc.

[39]  Jie Xiong,et al.  ArrayTrack: A Fine-Grained Indoor Location System , 2011, NSDI.

[40]  Dina Katabi,et al.  Extracting Gait Velocity and Stride Length from Surrounding Radio Signals , 2017, CHI.

[41]  Pei Zhang,et al.  Monitoring a Person's Heart Rate and Respiratory Rate on a Shared Bed Using Geophones , 2017, SenSys.

[42]  Jie Yang,et al.  E-eyes: device-free location-oriented activity identification using fine-grained WiFi signatures , 2014, MobiCom.

[43]  Rob Miller,et al.  3D Tracking via Body Radio Reflections , 2014, NSDI.

[44]  Khaled A. Harras,et al.  UbiBreathe: A Ubiquitous non-Invasive WiFi-based Breathing Estimator , 2015, MobiHoc.

[45]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.

[46]  Lionel M. Ni,et al.  RASS: A real-time, accurate and scalable system for tracking transceiver-free objects , 2011, 2011 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[47]  Tomoaki Ohtsuki,et al.  Doppler Analysis Based Fall Detection Using Array Antenna , 2018, 2018 IEEE International Conference on Communications (ICC).

[48]  Shiwen Mao,et al.  CSI-Based Fingerprinting for Indoor Localization: A Deep Learning Approach , 2016, IEEE Transactions on Vehicular Technology.

[49]  Kentaro Nishimori,et al.  MIMO Sensor -Evaluation on Antenna Arrangement- , 2011, Proceedings of the 5th European Conference on Antennas and Propagation (EUCAP).

[50]  Ivan Poupyrev,et al.  Soli , 2016, ACM Trans. Graph..

[51]  Yang Xu,et al.  WiFinger: talk to your smart devices with finger-grained gesture , 2016, UbiComp.

[52]  L. Li,et al.  On pixel count based crowd density estimation for visual surveillance , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[53]  Kaishun Wu,et al.  WiFall: Device-free fall detection by wireless networks , 2017, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[54]  Kyu-Han Kim,et al.  SAIL: single access point-based indoor localization , 2014, MobiSys.

[55]  Subhas Chandra Mukhopadhyay,et al.  Wearable Sensors for Human Activity Monitoring: A Review , 2015, IEEE Sensors Journal.

[56]  Kaishun Wu,et al.  We Can Hear You with Wi-Fi! , 2014, IEEE Transactions on Mobile Computing.

[57]  Mo Li,et al.  Precise Power Delay Profiling with Commodity Wi-Fi , 2015, IEEE Transactions on Mobile Computing.

[58]  Changzhi Li,et al.  A Review on Recent Advances in Doppler Radar Sensors for Noncontact Healthcare Monitoring , 2013, IEEE Transactions on Microwave Theory and Techniques.

[59]  Eric A. Wan,et al.  RSSI-Based Indoor Localization and Tracking Using Sigma-Point Kalman Smoothers , 2009, IEEE Journal of Selected Topics in Signal Processing.

[60]  Mandy Eberhart,et al.  Spotlight Synthetic Aperture Radar Signal Processing Algorithms , 2016 .

[61]  Dina Katabi,et al.  Zero-Effort In-Home Sleep and Insomnia Monitoring using Radio Signals , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[62]  Lu Wang,et al.  FIMD: Fine-grained Device-free Motion Detection , 2012, 2012 IEEE 18th International Conference on Parallel and Distributed Systems.

[63]  Shiwen Mao,et al.  DeepFi: Deep learning for indoor fingerprinting using channel state information , 2015, 2015 IEEE Wireless Communications and Networking Conference (WCNC).

[64]  Paul Congdon,et al.  Avoiding multipath to revive inbuilding WiFi localization , 2013, MobiSys '13.

[65]  Jin Zhang,et al.  WiFi-ID: Human Identification Using WiFi Signal , 2016, 2016 International Conference on Distributed Computing in Sensor Systems (DCOSS).

[66]  Tom Minka,et al.  You are facing the Mona Lisa: spot localization using PHY layer information , 2012, MobiSys '12.

[67]  Kaishun Wu,et al.  WiG: WiFi-Based Gesture Recognition System , 2015, 2015 24th International Conference on Computer Communication and Networks (ICCCN).

[68]  Yan Wang,et al.  An Improved K-Nearest-Neighbor Indoor Localization Method Based on Spearman Distance , 2016, IEEE Signal Processing Letters.

[69]  Wei Wang,et al.  Gait recognition using wifi signals , 2016, UbiComp.

[70]  W. Carrara,et al.  Spotlight synthetic aperture radar : signal processing algorithms , 1995 .

[71]  Xiaolong Yang,et al.  WiCatch: A Wi-Fi Based Hand Gesture Recognition System , 2018, IEEE Access.

[72]  Fadel Adib,et al.  Multi-Person Motion Tracking via RF Body Reflections , 2014 .

[73]  Maurizio Bocca,et al.  Breathfinding: A Wireless Network That Monitors and Locates Breathing in a Home , 2013, IEEE Journal of Selected Topics in Signal Processing.

[74]  Li Sun,et al.  WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices , 2015, MobiCom.

[75]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[76]  S. Seidel,et al.  914 MHz path loss prediction models for indoor wireless communications in multifloored buildings , 1992 .

[77]  Anding Zhu,et al.  Breathing rate estimation from a non-contact biosensor using an adaptive IIR notch filter , 2012, 2012 IEEE Topical Conference on Biomedical Wireless Technologies, Networks, and Sensing Systems (BioWireleSS).

[78]  Stefano Chessa,et al.  A Novel Approach to Indoor RSSI Localization by Automatic Calibration of the Wireless Propagation Model , 2009, VTC Spring 2009 - IEEE 69th Vehicular Technology Conference.

[79]  Robert J. Piechocki,et al.  Non-contact breathing detection using passive radar , 2016, 2016 IEEE International Conference on Communications (ICC).

[80]  Wei Wang,et al.  Device-Free Human Activity Recognition Using Commercial WiFi Devices , 2017, IEEE Journal on Selected Areas in Communications.

[81]  Chao Yang,et al.  PhaseBeat: Exploiting CSI Phase Data for Vital Sign Monitoring with Commodity WiFi Devices , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[82]  Neal Patwari,et al.  Radio Tomographic Imaging with Wireless Networks , 2010, IEEE Transactions on Mobile Computing.

[83]  Mahmut Tenruh,et al.  Increasing RSSI Localization Accuracy with Distance Reference Anchor in Wireless Sensor Networks , 2014 .

[84]  Jake K. Aggarwal,et al.  Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[85]  Xinyu Zhang,et al.  Continuous and fine-grained breathing volume monitoring from afar using wireless signals , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[86]  Anand Sivasubramaniam,et al.  HB-Phone: A Bed-Mounted Geophone-Based Heartbeat Monitoring System , 2017, 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[87]  P. Lukowicz,et al.  Collaborative Crowd Density Estimation with Mobile Phones , 2011 .

[88]  Mun Choon Chan,et al.  Low cost crowd counting using audio tones , 2012, SenSys '12.

[89]  Xiang Wang,et al.  WFID: Passive Device-free Human Identification Using WiFi Signal , 2016, MobiQuitous.

[90]  Sneha Kumar Kasera,et al.  Monitoring Breathing via Signal Strength in Wireless Networks , 2011, IEEE Transactions on Mobile Computing.

[91]  Yusheng Ji,et al.  Leveraging RF-channel fluctuation for activity recognition: Active and passive systems, continuous and RSSI-based signal features , 2013, MoMM '13.

[92]  Tong Zhao,et al.  A Robust Passive Intrusion Detection System with Commodity WiFi Devices , 2018, J. Sensors.

[93]  Shaojie Tang,et al.  Electronic frog eye: Counting crowd using WiFi , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[94]  Tomoaki Ohtsuki,et al.  State Classification with Array Sensor Using Support Vector Machine for Wireless Monitoring Systems , 2012, IEICE transactions on communications.

[95]  Bo Yang,et al.  Radio-Frequency Tomography for Passive Indoor Multitarget Tracking , 2013, IEEE Transactions on Mobile Computing.

[96]  Mi Zhang,et al.  BodyScan: Enabling Radio-based Sensing on Wearable Devices for Contactless Activity and Vital Sign Monitoring , 2016, MobiSys.

[97]  Xue Liu,et al.  Taming the inconsistency of Wi-Fi fingerprints for device-free passive indoor localization , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[98]  David Wetherall,et al.  Tool release: gathering 802.11n traces with channel state information , 2011, CCRV.

[99]  Khaled A. Harras,et al.  WiGest: A ubiquitous WiFi-based gesture recognition system , 2014, 2015 IEEE Conference on Computer Communications (INFOCOM).

[100]  Mi Zhang,et al.  HeadScan: A Wearable System for Radio-Based Sensing of Head and Mouth-Related Activities , 2016, 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[101]  Parth H. Pathak,et al.  WiWho: WiFi-Based Person Identification in Smart Spaces , 2016, 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[102]  Alessio Brutti,et al.  Acoustic Based Surveillance System for Intrusion Detection , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[103]  Changzhi Li,et al.  Random Body Movement Cancellation in Doppler Radar Vital Sign Detection , 2008, IEEE Transactions on Microwave Theory and Techniques.

[104]  Hiroyuki Tsuji,et al.  Indoor Event Detection with Eigenvector Spanning Signal Subspace for Home or Office Security , 2008, 2008 IEEE 68th Vehicular Technology Conference.

[105]  Naoki Honma,et al.  Compact antenna arrangement for MIMO sensor in indoor environment , 2012, 2012 International Symposium on Antennas and Propagation (ISAP).

[106]  Rob Miller,et al.  Smart Homes that Monitor Breathing and Heart Rate , 2015, CHI.

[107]  Parameswaran Ramanathan,et al.  Leveraging directional antenna capabilities for fine-grained gesture recognition , 2014, UbiComp.

[108]  Shuangquan Wang,et al.  SignFi , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[109]  Ossi Kaltiokallio,et al.  Non-invasive respiration rate monitoring using a single COTS TX-RX pair , 2014, IPSN-14 Proceedings of the 13th International Symposium on Information Processing in Sensor Networks.

[110]  Wei Xi,et al.  Estimating Crowd Density in an RF-Based Dynamic Environment , 2013, IEEE Sensors Journal.

[111]  Daqing Zhang,et al.  Anti-fall: A Non-intrusive and Real-Time Fall Detector Leveraging CSI from Commodity WiFi Devices , 2015, ICOST.

[112]  Gerhard Tröster,et al.  The telepathic phone: Frictionless activity recognition from WiFi-RSSI , 2014, 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[113]  Fadel Adib,et al.  Emotion recognition using wireless signals , 2016, MobiCom.

[114]  Fadel Adib,et al.  See through walls with WiFi! , 2013, SIGCOMM.

[115]  Wei Wang,et al.  Keystroke Recognition Using WiFi Signals , 2015, MobiCom.

[116]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[117]  Tomoaki Ohtsuki,et al.  Human activity classification and localization algorithm based on temporal-spatial virtual array , 2013, 2013 IEEE International Conference on Communications (ICC).

[118]  Kaishun Wu,et al.  CSI-Based Indoor Localization , 2013, IEEE Transactions on Parallel and Distributed Systems.

[119]  Xiaohui Liang,et al.  When CSI Meets Public WiFi: Inferring Your Mobile Phone Password via WiFi Signals , 2016, CCS.

[120]  V. Koivunen,et al.  Continuous wave radar based vital sign estimation: Modeling and experiments , 2012, 2012 IEEE Radar Conference.

[121]  Shyamnath Gollakota,et al.  Wi-Fi Gesture Recognition on Existing Devices , 2014, ArXiv.

[122]  Jiro Katto,et al.  A Study on Passive Crowd Density Estimation using Wireless Sensors , 2008 .

[123]  Frédo Durand,et al.  Capturing the human figure through a wall , 2015, ACM Trans. Graph..

[124]  Xiaohua Zhu,et al.  Noncontact Physiological Dynamics Detection Using Low-power Digital-IF Doppler Radar , 2017, IEEE Transactions on Instrumentation and Measurement.

[125]  Tzyy-Sheng Horng,et al.  Single-Antenna Doppler Radars Using Self and Mutual Injection Locking for Vital Sign Detection With Random Body Movement Cancellation , 2011, IEEE Transactions on Microwave Theory and Techniques.

[126]  Nitin H. Vaidya,et al.  Proceedings of the sixteenth annual international conference on Mobile computing and networking , 2010, MobiCom 2010.

[127]  Xu Chen,et al.  Tracking Vital Signs During Sleep Leveraging Off-the-shelf WiFi , 2015, MobiHoc.

[128]  Yunhao Liu,et al.  PADS: Passive detection of moving targets with dynamic speed using PHY layer information , 2014, 2014 20th IEEE International Conference on Parallel and Distributed Systems (ICPADS).

[129]  Ashok Veeraraghavan,et al.  DistancePPG: Robust non-contact vital signs monitoring using a camera , 2015, Biomedical optics express.

[130]  Moustafa Youssef,et al.  CoSDEO 2016 Keynote: A decade later — Challenges: Device-free passive localization for wireless environments , 2016, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[131]  Muhammad Shahzad,et al.  Position and Orientation Agnostic Gesture Recognition Using WiFi , 2017, MobiSys.

[132]  Neal Patwari,et al.  See-Through Walls: Motion Tracking Using Variance-Based Radio Tomography Networks , 2011, IEEE Transactions on Mobile Computing.

[133]  Jenshan Lin,et al.  Complex signal demodulation and random body movement cancellation techniques for non-contact vital sign detection , 2008, 2008 IEEE MTT-S International Microwave Symposium Digest.

[134]  M. Vossiek,et al.  Precise 3-D Object Position Tracking using FMCW Radar , 1999, 1999 29th European Microwave Conference.

[135]  Yunhao Liu,et al.  Towards omnidirectional passive human detection , 2013, 2013 Proceedings IEEE INFOCOM.

[136]  Tommy W. S. Chow,et al.  A neural-based crowd estimation by hybrid global learning algorithm , 1999, IEEE Trans. Syst. Man Cybern. Part B.