MoSen: Activity Modelling in Multiple-Occupancy Smart Homes

Smart home solutions increasingly rely on a variety of sensors for behavioral analytics and activity recognition to provide contextaware applications and personalized care. Optimizing the sensor network is one of the most important approaches to ensure the classification accuracy and system’s efficiency. However, the tradeoff between the cost and performance is often a challenge in real deployments, particularly for multiple-occupancy smart homes or care homes. In this paper, using real indoor activity and mobility traces, floor plans, and synthetic multi-occupancy behavior models, we evaluate several multi-occupancy household scenarios with 2-5 residents. We explore and quantify the trade-offs between the cost of sensor deployments and expected labeling accuracy in different scenarios. Our evaluation across different scenarios show that the performance of the desired context-aware task is affected by different localization resolutions, the number of residents, the number of sensors, and varying sensor deployments. To aid in accelerating the adoption of practical sensor-based activity recognition technology, we design MoSen, a framework to simulate the interaction dynamics between sensor-based environments and multiple residents. By evaluating the factors that affect the performance of the desired sensor network, we provide a sensor selection strategy and design metrics for sensor layout in real environments. Using our selection strategy in a 5-person scenario case study, we demonstrate that MoSen can significantly improve overall system performance without increasing the deployment costs.

[1]  Li-Chen Fu,et al.  Robust Location-Aware Activity Recognition Using Wireless Sensor Network in an Attentive Home , 2009, IEEE Transactions on Automation Science and Engineering.

[2]  Steven M. Bellovin,et al.  Privacy and Synthetic Datasets , 2018 .

[3]  Joaquín Torres-Sospedra,et al.  A Meta-Review of Indoor Positioning Systems , 2019, Sensors.

[4]  Suresh Venkatasubramanian,et al.  Multiple Target Tracking with RF Sensor Networks , 2013, IEEE Transactions on Mobile Computing.

[5]  Yasamin Mostofi,et al.  Tracking from One Side - Multi-Person Passive Tracking with WiFi Magnitude Measurements , 2019, 2019 18th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[6]  Fadel Adib,et al.  Multi-Person Localization via RF Body Reflections , 2015, NSDI.

[7]  Yutaka Arakawa,et al.  Beacon-based multi-person activity monitoring system for day care center , 2016, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[8]  Florica Moldoveanu,et al.  A Comprehensive Survey of Indoor Localization Methods Based on Computer Vision , 2020, Sensors.

[9]  Diane J. Cook,et al.  Transfer learning for activity recognition: a survey , 2013, Knowledge and Information Systems.

[10]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[11]  Mikhail Belkin,et al.  Automatic Annotation of Daily Activity from Smartphone-Based Multisensory Streams , 2012, MobiCASE.

[12]  Amjad Anvari-Moghaddam,et al.  Multi-Residential Activity Labelling in Smart Homes with Wearable Tags Using BLE Technology , 2018, Sensors.

[13]  Fernando Seco Granja,et al.  Comparing Ubisense, BeSpoon, and DecaWave UWB Location Systems: Indoor Performance Analysis , 2017, IEEE Transactions on Instrumentation and Measurement.

[14]  Hans W. Guesgen Human Behavior Recognition Technologies - Intelligent Applications for Monitoring and Security , 2013, Human Behavior Recognition Technologies.

[15]  Yun Cheng,et al.  UWB Indoor Positioning Algorithm Based on TDOA Technology , 2019, 2019 10th International Conference on Information Technology in Medicine and Education (ITME).

[16]  Mihaela van der Schaar,et al.  PATE-GAN: Generating Synthetic Data with Differential Privacy Guarantees , 2018, ICLR.

[17]  Vicente Matellán Olivera,et al.  A context‐awareness model for activity recognition in robot‐assisted scenarios , 2020, Expert Syst. J. Knowl. Eng..

[18]  Christopher G. Atkeson,et al.  Simultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors , 2005, Pervasive.

[19]  Daijin Kim,et al.  Robust human activity recognition from depth video using spatiotemporal multi-fused features , 2017, Pattern Recognit..

[20]  Luca Benini,et al.  Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection , 2008, EWSN.

[21]  Mikkel Baun Kjærgaard,et al.  Smart Devices are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition , 2015, SenSys.

[22]  Young-Im Cho,et al.  A Mobile Robot Localization using External Surveillance Cameras at Indoor , 2015, FNC/MobiSPC.

[23]  Zhi Chen,et al.  Adversarial Feature Matching for Text Generation , 2017, ICML.

[24]  Hamed Haddadi,et al.  Activity prediction for improving well-being of both the elderly and caregivers , 2019, UbiComp/ISWC Adjunct.

[25]  Alessandro Montanari,et al.  A closer look at quality-aware runtime assessment of sensing models in multi-device environments , 2019, SenSys.

[26]  Inseok Hwang,et al.  CoMon+: A Cooperative Context Monitoring System for Multi-Device Personal Sensing Environments , 2016, IEEE Transactions on Mobile Computing.

[27]  Paolo Fornacciari,et al.  IoT Wearable Sensor and Deep Learning: An Integrated Approach for Personalized Human Activity Recognition in a Smart Home Environment , 2019, IEEE Internet of Things Journal.

[28]  Zhe Gan,et al.  AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[30]  Kin K. Leung,et al.  A Survey of Indoor Localization Systems and Technologies , 2017, IEEE Communications Surveys & Tutorials.

[31]  Diamantino Freitas,et al.  Indoor localization with audible sound - Towards practical implementation , 2016, Pervasive Mob. Comput..

[32]  Peng Guo,et al.  A Deep-learning-based Method for PIR-based Multi-person Localization , 2020, 2004.04329.

[33]  Peng Li,et al.  An RFID Indoor Positioning Algorithm Based on Bayesian Probability and K-Nearest Neighbor , 2017, Sensors.

[34]  Ramón F. Brena,et al.  Evolution of Indoor Positioning Technologies: A Survey , 2017, J. Sensors.

[35]  Mani B. Srivastava,et al.  SenseGen: A deep learning architecture for synthetic sensor data generation , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[36]  Rob Miller,et al.  Smart Homes that Monitor Breathing and Heart Rate , 2015, CHI.

[37]  Hongnian Yu,et al.  Elderly activities recognition and classification for applications in assisted living , 2013, Expert Syst. Appl..

[38]  Maged N Kamel Boulos,et al.  Real-time locating systems (RTLS) in healthcare: a condensed primer. , 2012, International journal of health geographics.

[39]  Yongtao Ma,et al.  Basmag: An Optimized HMM-Based Localization System Using Backward Sequences Matching Algorithm Exploiting Geomagnetic Information , 2016, IEEE Sensors Journal.

[40]  Latifur Khan,et al.  Facing the reality of data stream classification: coping with scarcity of labeled data , 2012, Knowledge and Information Systems.

[41]  Diane J. Cook,et al.  SynSys: A Synthetic Data Generation System for Healthcare Applications , 2019, Sensors.

[42]  Moustafa Youssef,et al.  WiDeep: WiFi-based Accurate and Robust Indoor Localization System using Deep Learning , 2019, 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom.

[43]  Hayit Greenspan,et al.  GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification , 2018, Neurocomputing.

[44]  Diane J. Cook,et al.  CASAS: A Smart Home in a Box , 2013, Computer.

[45]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[46]  Guoliang Xing,et al.  PBN: towards practical activity recognition using smartphone-based body sensor networks , 2011, SenSys.

[47]  R. Venkatesha Prasad,et al.  LocED: Location-aware Energy Disaggregation Framework , 2015, BuildSys@SenSys.

[48]  Maria João Nicolau,et al.  Wi-Fi fingerprinting in the real world - RTLS@UM at the EvAAL competition , 2015, 2015 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[49]  Cynthia Dwork,et al.  Differential Privacy: A Survey of Results , 2008, TAMC.

[50]  Mohammed Feham,et al.  Multioccupant Activity Recognition in Pervasive Smart Home Environments , 2015, ACM Comput. Surv..

[51]  Abdelhamid Bouchachia,et al.  Multi-resident Activity Recognition Using Incremental Decision Trees , 2014, ICAIS.

[52]  Jie Xiong,et al.  ArrayTrack: A Fine-Grained Indoor Location System , 2011, NSDI.

[53]  Ahmad Almogren,et al.  A robust human activity recognition system using smartphone sensors and deep learning , 2018, Future Gener. Comput. Syst..

[54]  H. Erfani,et al.  Unravelling Effects of the Pore‐Size Correlation Length on the Two‐Phase Flow and Solute Transport Properties: GPU‐based Pore‐Network Modeling , 2020, Water Resources Research.

[55]  Jerry Zhao,et al.  Habitat monitoring: application driver for wireless communications technology , 2001, CCRV.

[56]  Özlem Durmaz Incel,et al.  ARAS human activity datasets in multiple homes with multiple residents , 2013, 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops.

[57]  G ShinKang,et al.  Geomagnetism for Smartphone-Based Indoor Localization , 2017 .

[58]  Hae Young Noh,et al.  Indoor Person Identification through Footstep Induced Structural Vibration , 2015, HotMobile.

[59]  Svetha Venkatesh,et al.  Learning and detecting activities from movement trajectories using the hierarchical hidden Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[60]  Fernando J. Álvarez,et al.  Broadband acoustic local positioning system for mobile devices with multiple access interference cancellation , 2018 .

[61]  Yunhao Liu,et al.  LANDMARC: Indoor Location Sensing Using Active RFID , 2004, Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. (PerCom 2003)..

[62]  Timo Sztyler,et al.  Online personalization of cross-subjects based activity recognition models on wearable devices , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[63]  Youngki Lee,et al.  An Active Resource Orchestration Framework for PAN-Scale, Sensor-Rich Environments , 2014, IEEE Transactions on Mobile Computing.

[64]  John Adcock,et al.  Indoor localization using controlled ambient sounds , 2012, 2012 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[65]  JazizadehFarrokh,et al.  Indoor Positioning Based on Visible Light Communication , 2019 .

[66]  Dieter Schmalstieg,et al.  Indoor Positioning and Navigation with Camera Phones , 2009, IEEE Pervasive Computing.

[67]  Miguel Á. Carreira-Perpiñán,et al.  Occupancy Modeling and Prediction for Building Energy Management , 2014, ACM Trans. Sens. Networks.

[68]  Muhammad Adeel Pasha,et al.  Highly accurate 3D wireless indoor positioning system using white LED lights , 2014 .

[69]  Emiliano Miluzzo,et al.  A survey of mobile phone sensing , 2010, IEEE Communications Magazine.

[70]  Gwenn Englebienne,et al.  Accurate activity recognition in a home setting , 2008, UbiComp.

[71]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[72]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[73]  Athanasios V. Vasilakos,et al.  GCHAR: An efficient Group-based Context - aware human activity recognition on smartphone , 2017, J. Parallel Distributed Comput..

[74]  Ling Bao,et al.  A context-aware experience sampling tool , 2003, CHI Extended Abstracts.

[75]  Konrad Schindler,et al.  Learning by Tracking: Siamese CNN for Robust Target Association , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[76]  Margaret Martonosi,et al.  Location-based trust for mobile user-generated content: applications, challenges and implementations , 2008, HotMobile '08.

[77]  Ygal Bendavid,et al.  RFID-enabled Real-Time Location System (RTLS) to improve hospital's operations management: An up-to-date typology , 2013, Int. J. RF Technol. Res. Appl..

[78]  Rabea Kurdi,et al.  IoT based mobile healthcare system for human activity recognition , 2018, 2018 15th Learning and Technology Conference (L&T).

[79]  Mohammad Syafrudin,et al.  LOSNUS: An ultrasonic system enabling high accuracy and secure TDoA locating of numerous devices , 2010, 2010 International Conference on Indoor Positioning and Indoor Navigation.

[80]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[81]  Ming Liu,et al.  Plugo: a VLC Systematic Perspective of Large-scale Indoor Localization , 2017, ArXiv.

[82]  Yi-Ting Chiang,et al.  Interaction models for multiple-resident activity recognition in a smart home , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[83]  Tayeb Lemlouma,et al.  A survey on health monitoring systems for health smart homes , 2018, International Journal of Industrial Ergonomics.

[84]  Ashish Patel,et al.  Sensor-based activity recognition in the context of ambient assisted living systems: A review , 2019, J. Ambient Intell. Smart Environ..

[85]  Youngnam Han,et al.  SmartPDR: Smartphone-Based Pedestrian Dead Reckoning for Indoor Localization , 2015, IEEE Sensors Journal.