Smart secure homes: a survey of smart home technologies that sense, assess, and respond to security threats

Smart home design has undergone a metamorphosis in recent years. The field has evolved from designing theoretical smart home frameworks and performing scripted tasks in laboratories. Instead, we now find robust smart home technologies that are commonly used by large segments of the population in a variety of settings. Recent smart home applications are focused on activity recognition, health monitoring, and automation. In this paper, we take a look at another important role for smart homes: security. We first explore the numerous ways smart homes can and do provide protection for their residents. Next, we provide a comparative analysis of the alternative tools and research that has been developed for this purpose. We investigate not only existing commercial products that have been introduced but also discuss the numerous research that has been focused on detecting and identifying potential threats. Finally, we close with open challenges and ideas for future research that will keep individuals secure and healthy while in their own homes.

[1]  鈴木 喜久.  Home Security , 1943 .

[2]  P J Nichols,et al.  Functional Assessment , 2019, The SAGE Encyclopedia of Human Communication Sciences and Disorders.

[3]  M. Weiser.  The Computer for the Twenty-First Century , 1991 .

[4]  Mark Weiser.  The computer for the 21st century , 1991 .

[5]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[6]  P. Barberger‐Gateau,et al.  Four Instrumental Activities of Daily Living Score as a predictor of one-year incident dementia. , 1993, Age and ageing.

[7]  S. Zimmerman,et al.  Methodological issues in measuring the functional status of cognitively impaired nursing home residents: the use of proxies and performance-based measures. , 1994, Alzheimer Disease and Associated Disorders.

[8]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[9]  U. Sonn,et al.  Activities of daily living studied longitudinally between 70 and 76 years of age. , 1996, Disability and rehabilitation.

[10]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[11]  Jun Rekimoto,et al.  Wearable key: device for personalizing nearby environment , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[12]  N. Noury,et al.  Monitoring behavior in home using a smart fall sensor and position sensors , 2000, 1st Annual International IEEE-EMBS Special Topic Conference on Microtechnologies in Medicine and Biology. Proceedings (Cat. No.00EX451).

[13]  B. Vellas,et al.  Instrumental activities of daily living as a potential marker of frailty: a study of 7364 community-dwelling elderly women (the EPIDOS study). , 2001, The journals of gerontology. Series A, Biological sciences and medical sciences.

[14]  L. Levy.  Falls in Older People: Risk Factors and Strategies for Prevention , 2002 .

[15]  Lesley M. Day.  Falls in Older People: Risk Factors and Strategies for Prevention. , 2003 .

[16]  Diane J. Cook,et al.  MavHome: an agent-based smart home , 2003, Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. (PerCom 2003)..

[17]  Diane J. Cook,et al.  Graph-based anomaly detection , 2003, KDD '03.

[18]  Daniel K. Blandford,et al.  NetMine: New Mining Tools for Large Graphs , 2004 .

[19]  A. Desai,et al.  Activities of Daily Living in Patients with Dementia , 2004, CNS drugs.

[20]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[21]  Andrew W. Moore,et al.  Active Learning for Anomaly and Rare-Category Detection , 2004, NIPS.

[22]  Jay Stewart,et al.  The American Time Use Survey , 2005 .

[23]  William C. Mann,et al.  The Gator Tech Smart House: a programmable pervasive space , 2005, Computer.

[24]  Gregory D. Abowd,et al.  Designing for the Human Experience in Smart Environments , 2005 .

[25]  Michael C. Mozer,et al.  Lessons from an Adaptive Home , 2005 .

[26]  Donald E. Brown,et al.  Health-status monitoring through analysis of behavioral patterns , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[27]  Henry A. Kautz,et al.  Location-Based Activity Recognition using Relational Markov Networks , 2005, IJCAI.

[28]  Kent Larson,et al.  Using a Live-In Laboratory for Ubiquitous Computing Research , 2006, Pervasive.

[29]  M. Alwan,et al.  A Smart and Passive Floor-Vibration Based Fall Detector for Elderly , 2006, 2006 2nd International Conference on Information & Communication Technologies.

[30]  C. Fabrigoule,et al.  Restriction in complex activities of daily living in MCI , 2006, Neurology.

[31]  Masamichi Shimosaka,et al.  Typical Behavior Patterns Extraction and Anomaly Detection Algorithm Based on Accumulated Home Sensor Data , 2007, Future Generation Communication and Networking (FGCN 2007).

[32]  Darren Leigh,et al.  The MERL Motion Detector Dataset: 2007 Workshop on Massive Datasets , 2007 .

[33]  Svetha Venkatesh,et al.  Multi-modal emotive computing in a smart house environment , 2007, Pervasive Mob. Comput..

[34]  Paul Cuddihy,et al.  Algorithm to automatically detect abnormally long periods of inactivity in a home , 2007, HealthNet '07.

[35]  Hani Hagras,et al.  An Incremental Adaptive Life Long Learning Approach for Type-2 Fuzzy Embedded Agents in Ambient Intelligent Environments , 2007, IEEE Transactions on Fuzzy Systems.

[36]  Darren Leigh,et al.  The MERL motion detector dataset , 2007, MD '07.

[37]  Carla Schlatter Ellis,et al.  Using Ground Reaction Forces from Gait Analysis: Body Mass as a Weak Biometric , 2007, Pervasive.

[38]  Diane J. Cook,et al.  Data Mining for Hierarchical Model Creation , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[39]  Jessica K. Hodgins,et al.  Guide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database , 2008 .

[40]  Fabien Cardinaux,et al.  Modelling of Behavioural Patterns for Abnormality Detection in the Context of Lifestyle Reassurance , 2008, CIARP.

[41]  Koby Crammer,et al.  Active Learning with Confidence , 2008, ACL.

[42]  Alladi Venkatesh.  Digital home technologies and transformation of households , 2008, Inf. Syst. Frontiers.

[43]  Rama Chellappa,et al.  Machine Recognition of Human Activities: A Survey , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[44]  G. Demiris,et al.  Technologies for an Aging Society: A Systematic Review of “Smart Home” Applications , 2008, Yearbook of Medical Informatics.

[45]  Nikolaos Papanikolopoulos,et al.  Multi-class active learning for image classification , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Lawrence B. Holder,et al.  Graph-Based Analysis of Nuclear Smuggling Data , 2009 .

[47]  Gregory D. Abowd,et al.  Developing shared home behavior datasets to advance HCI and ubiquitous computing research , 2009, CHI Extended Abstracts.

[48]  Nikolaos Papanikolopoulos,et al.  Multi-class active learning for image classification , 2009, CVPR.

[49]  Gilles Virone.  Assessing everyday life behavioral rhythms for the older generation , 2009, Pervasive Mob. Comput..

[50]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[51]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[52]  N. Noury,et al.  Supervised classification of activities of daily living in health smart homes using SVM , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[53]  Mark Hasegawa-Johnson,et al.  Acoustic fall detection using Gaussian mixture models and GMM supervectors , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[54]  Shwetak N. Patel,et al.  ElectriSense: single-point sensing using EMI for electrical event detection and classification in the home , 2010, UbiComp.

[55]  Diane J. Cook,et al.  Predicting air quality in smart environments , 2010, J. Ambient Intell. Smart Environ..

[56]  Abdelsalam Helal,et al.  The Making of a Dataset for Smart Spaces , 2010, UIC.

[57]  Jonathan Dodge,et al.  Studying always-on electricity feedback in the home , 2010, CHI.

[58]  Mark W. Newman,et al.  Automatic Assessment of Cognitive Impairment through Electronic Observation of Object Usage , 2010, Pervasive.

[59]  渡邊 嘉二郎.  Ubiquitous health monitoring at home : sensing of human biosignals on flooring, on tatami mat, in the bathtub, and in the lavatory , 2010 .

[60]  Chong Eng Tan,et al.  A neural network approach towards reinforcing smart home security , 2010, 8th Asia-Pacific Symposium on Information and Telecommunication Technologies.

[61]  Mario Cannataro,et al.  Protein-to-protein interactions: Technologies, databases, and algorithms , 2010, CSUR.

[62]  Mihail Popescu,et al.  Acoustic fall detection using a circular microphone array , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[63]  Tai-hoon Kim,et al.  A Review on Security in Smart Home Development , 2010 .

[64]  Liming Chen,et al.  Activity Recognition: Approaches, Practices and Trends , 2011 .

[65]  John Krumm,et al.  PreHeat: controlling home heating using occupancy prediction , 2011, UbiComp '11.

[66]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[67]  Jean Meunier,et al.  Robust Video Surveillance for Fall Detection Based on Human Shape Deformation , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[68]  Biming Tian,et al.  Anomaly detection in wireless sensor networks: A survey , 2011, J. Netw. Comput. Appl..

[69]  Abdenour Bouzouane,et al.  A Possibilistic Approach for Activity Recognition in Smart Homes for Cognitive Assistance to Alzheimer’s Patients , 2011 .

[70]  Michael Eisenbarth,et al.  An approach for detecting deviations in daily routine for long-term behavior analysis , 2011, 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[71]  Diane J. Cook,et al.  Behaviometrics for multiple residents in a smart environment , 2011 .

[72]  Misha Pavel,et al.  SVM to detect the presence of visitors in a smart home environment , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[73]  Enamul Hoque,et al.  Semantic anomaly detection in daily activities , 2012, UbiComp '12.

[74]  Mamun Bin Ibne Reaz,et al.  A Review of Smart Homes—Past, Present, and Future , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[75]  Anil K. Jain,et al.  Biometric Authentication: System Security and User Privacy , 2012, Computer.

[76]  Anna Fensel,et al.  SESAME-S: Semantic Smart Home System for Energy Efficiency , 2013, Informatik-Spektrum.

[77]  Young-Koo Lee,et al.  A Framework for Supervising Lifestyle Diseases Using Long-Term Activity Monitoring , 2012, Sensors.

[78]  Jesse Hoey,et al.  Sensor-Based Activity Recognition , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[79]  Ahmad Lotfi,et al.  Smart homes for the elderly dementia sufferers: identification and prediction of abnormal behaviour , 2012, J. Ambient Intell. Humaniz. Comput..

[80]  Lawrence B. Holder,et al.  Graph-Based Anomaly Detection Applied to Homeland Security Cargo Screening , 2012, FLAIRS Conference.

[81]  Daniel Austin,et al.  In-home walking speeds and variability trajectories associated with mild cognitive impairment , 2012, Neurology.

[82]  Diane J. Cook,et al.  CASAS: A Smart Home in a Box , 2013, Computer.

[83]  Ling Shao,et al.  A survey on fall detection: Principles and approaches , 2013, Neurocomputing.

[84]  Patrick Olivier,et al.  People, sensors, decisions: Customizable and adaptive technologies for assistance in healthcare , 2012, TIIS.

[85]  Diane J. Cook,et al.  Behaviometrics for Identifying Smart Home Residents , 2013 .

[86]  Jenq-Neng Hwang,et al.  A Review on Video-Based Human Activity Recognition , 2013, Comput..

[87]  Abdenour Bouzouane,et al.  Correct by Construction Security Approach to Design Fault Tolerant Smart Homes for Disabled People , 2013, EUSPN/ICTH.

[88]  K. J. Miller,et al.  Smart-Home Technologies to Assist Older People to Live Well at Home , 2013 .

[89]  Diane J. Cook,et al.  Detecting Abnormality in Activities Performed by People with Dementia in a Smart Environment , 2014 .

[90]  Abdenour Hadid,et al.  Face Biometrics Under Spoofing Attacks: Vulnerabilities, Countermeasures, Open Issues, and Research Directions , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[91]  Ahmed Nait Aicha,et al.  Modeling visit behaviour in smart homes using unsupervised learning , 2014, UbiComp Adjunct.

[92]  Araceli Sanchis,et al.  Sensor-based Bayesian detection of anomalous living patterns in a home setting , 2014, Personal and Ubiquitous Computing.

[93]  Andreas Pitsillides,et al.  Survey in Smart Grid and Smart Home Security: Issues, Challenges and Countermeasures , 2014, IEEE Communications Surveys & Tutorials.

[94]  Bob Hardian,et al.  Predicting smart home lighting behavior from sensors and user input using very fast decision tree with Kernel Density Estimation and improved Laplace correction , 2014, 2014 International Conference on Advanced Computer Science and Information System.

[95]  Julie Doyle,et al.  Inferring health metrics from ambient smart home data , 2014, 2014 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[96]  Danai Koutra,et al.  Graph based anomaly detection and description: a survey , 2014, Data Mining and Knowledge Discovery.

[97]  Liming Chen,et al.  Dynamic sensor data segmentation for real-time knowledge-driven activity recognition , 2014, Pervasive Mob. Comput..

[98]  Charlie Wilson,et al.  Smart homes and their users: a systematic analysis and key challenges , 2014, Personal and Ubiquitous Computing.

[99]  Bernt Schiele,et al.  A tutorial on human activity recognition using body-worn inertial sensors , 2014, CSUR.

[100]  Chen Hu,et al.  Recognizing human activity in smart home using deep learning algorithm , 2014, Proceedings of the 33rd Chinese Control Conference.

[101]  Ricardo Matsumura de Araújo,et al.  Anthropometric and human gait identification using skeleton data from Kinect sensor , 2014, SAC.

[102]  Diane J. Cook,et al.  Activity recognition on streaming sensor data , 2014, Pervasive Mob. Comput..

[103]  Misha Pavel,et al.  Unobtrusive In-Home Detection of Time Spent Out-of-Home With Applications to Loneliness and Physical Activity , 2014, IEEE Journal of Biomedical and Health Informatics.

[104]  Vincent Lemaire,et al.  Optimised probabilistic active learning (OPAL) , 2015, Machine Learning.

[105]  Diane J. Cook,et al.  Activity Learning: Cook/Activity Learning , 2015 .

[106]  Marjorie Skubic,et al.  Fall Detection in Homes of Older Adults Using the Microsoft Kinect , 2015, IEEE Journal of Biomedical and Health Informatics.

[107]  Syed Mahfuzul Aziz,et al.  Sensor Anomaly Detection in Wireless Sensor Networks for Healthcare , 2015, Sensors.

[108]  Enamul Hoque,et al.  Holmes: A Comprehensive Anomaly Detection System for Daily In-home Activities , 2015, 2015 International Conference on Distributed Computing in Sensor Systems.

[109]  Diane J. Cook,et al.  Analyzing Activity Behavior and Movement in a Naturalistic Environment Using Smart Home Techniques , 2015, IEEE Journal of Biomedical and Health Informatics.

[110]  Prafulla N. Dawadi,et al.  Automated Clinical Assessment from Smart home-based Behavior Data , 2015 .

[111]  Rong Chen,et al.  Hidden State Conditional Random Field for Abnormal Activity Recognition in Smart Homes , 2015, Entropy.

[112]  Diane J. Cook,et al.  Activity Learning: Discovering, Recognizing, and Predicting Human Behavior from Sensor Data , 2015 .

[113]  Lawrence B. Holder,et al.  Scalable anomaly detection in graphs , 2015, Intell. Data Anal..

[114]  V. Lemaire,et al.  Optimised probabilistic active learning ( OPAL ) For fast , non-myopic , cost-sensitive active classification , 2015 .

[115]  Yun Sing Koh,et al.  Unsupervised Rare Pattern Mining , 2016, ACM Trans. Knowl. Discov. Data.

[116]  David Espes,et al.  A Framework for Anomaly Diagnosis in Smart Homes Based on Ontology , 2016, ANT/SEIT.

[117]  Daniel Gatica-Perez,et al.  Anomaly Detection in Elderly Daily Behavior in Ambient Sensing Environments , 2016, HBU.

[118]  Diane J. Cook,et al.  Using Smart Homes to Detect and Analyze Health Events , 2016, Computer.

[119]  Diane J. Cook,et al.  One-Class Classification-Based Real-Time Activity Error Detection in Smart Homes , 2016, IEEE Journal of Selected Topics in Signal Processing.

[120]  Diane J. Cook,et al.  Unsupervised detection and analysis of changes in everyday physical activity data , 2016, J. Biomed. Informatics.

[121]  Diane J. Cook,et al.  Modeling patterns of activities using activity curves , 2016, Pervasive Mob. Comput..

[122]  Leman Akoglu,et al.  Less is More , 2016, ACM Trans. Knowl. Discov. Data.

[123]  Jan-Michael Frahm,et al.  Virtual U: Defeating Face Liveness Detection by Building Virtual Models from Your Public Photos , 2016, USENIX Security Symposium.

[124]  Maureen Schmitter-Edgecombe,et al.  Automated Cognitive Health Assessment From Smart Home-Based Behavior Data , 2016, IEEE Journal of Biomedical and Health Informatics.

[125]  Lorenzo Chiari,et al.  Fall detection algorithms for real-world falls harvested from lumbar sensors in the elderly population: A machine learning approach , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[126]  Diane J. Cook,et al.  A survey of methods for time series change point detection , 2017, Knowledge and Information Systems.

[127]  Chi-Chun Lo,et al.  Using malware for software-defined networking–based smart home security management through a taint checking approach , 2016, Int. J. Distributed Sens. Networks.

[128]  Arvind D. Shaligram,et al.  An Investigative Study for Smart Home Security: Issues, Challenges and Countermeasures , 2016 .

[129]  Earlence Fernandes,et al.  Security Analysis of Emerging Smart Home Applications , 2016, 2016 IEEE Symposium on Security and Privacy (SP).

[130]  K. C. Ho,et al.  Testing non-wearable fall detection methods in the homes of older adults , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[131]  Winston Khoon Guan Seah,et al.  Rare Event Detection and Propagation in Wireless Sensor Networks , 2016, ACM Comput. Surv..

[132]  ChinLun Lai,et al.  A Smart Spoofing Face Detector by Display Features Analysis , 2016, Sensors.

[133]  Diane J Cook,et al.  Forecasting behavior in smart homes based on sleep and wake patterns. , 2017, Technology and health care : official journal of the European Society for Engineering and Medicine.

[134]  M. Novák,et al.  Anomaly Detection in User Daily Patterns in Smart-Home Environment , 2022 .