newNECTAR: Collaborative active learning for knowledge-based probabilistic activity recognition

Abstract The increasing popularity of ambient assisted living solutions is claiming adaptive and scalable tools to monitor activities of daily living. Currently, most sensor-based activity recognition techniques rely on supervised learning algorithms. However, the acquisition of comprehensive training sets of activities in smart homes is expensive and violates the individual’s privacy. In this work, we address this problem by proposing a novel hybrid approach that couples collaborative active learning with probabilistic and knowledge-based reasoning. The rationale of our approach is that a generic, and possibly incomplete, knowledge-based model of activities can be refined to target specific individuals and environments by collaboratively acquiring feedback from inhabitants. Specifically, we propose a collaborative active learning method exploiting users’ feedback to (i) refine correlations among sensor events and activity types that are initially extracted from a high-level ontology, and (ii) mine temporal patterns of sensor events that are frequently generated by the execution of specific activities. A Markov Logic Network is used to recognize activities with probabilistic rules that capture both the ontological knowledge and the information obtained by active learning. We experimented our solution with a real-world dataset of activities carried out by several individuals in an interleaved fashion. Experimental results show that our collaborative and personalized active learning solution significantly improves recognition rates, while triggering a small number of feedback requests. Moreover, the overall recognition rates compare favorably with existing supervised and unsupervised activity recognition methods.

[1]  Maureen Schmitter-Edgecombe,et al.  Automated Cognitive Health Assessment Using Smart Home Monitoring of Complex Tasks , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[2]  Diane J. Cook,et al.  Activity recognition on streaming sensor data , 2014, Pervasive Mob. Comput..

[3]  Diane J. Cook,et al.  Transfer learning for activity recognition: a survey , 2013, Knowledge and Information Systems.

[4]  Majid Sarrafzadeh,et al.  Toward Unsupervised Activity Discovery Using Multi-Dimensional Motif Detection in Time Series , 2009, IJCAI.

[5]  Timo Sztyler,et al.  Online personalization of cross-subjects based activity recognition models on wearable devices , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[6]  Heiner Stuckenschmidt,et al.  Markov Logic Networks with Numerical Constraints , 2016, ECAI.

[7]  Matthai Philipose,et al.  Building Reliable Activity Models Using Hierarchical Shrinkage and Mined Ontology , 2006, Pervasive.

[8]  Jian Lu,et al.  A Pattern Mining Approach to Sensor-Based Human Activity Recognition , 2011, IEEE Transactions on Knowledge and Data Engineering.

[9]  Lawrence B. Holder,et al.  Discovering Activities to Recognize and Track in a Smart Environment , 2011, IEEE Transactions on Knowledge and Data Engineering.

[10]  Timo Sztyler,et al.  NECTAR: Knowledge-based Collaborative Active Learning for Activity Recognition , 2018, 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[11]  Nirmalya Roy,et al.  Active learning enabled activity recognition , 2016, 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[12]  Matthai Philipose,et al.  Mining models of human activities from the web , 2004, WWW '04.

[13]  John F. Roddick,et al.  A Survey of Temporal Knowledge Discovery Paradigms and Methods , 2002, IEEE Trans. Knowl. Data Eng..

[14]  Diane J. Cook,et al.  CASAS: A Smart Home in a Box , 2013, Computer.

[15]  Timo Sztyler,et al.  Unsupervised recognition of interleaved activities of daily living through ontological and probabilistic reasoning , 2016, UbiComp.

[16]  Diane J. Cook,et al.  Mining Sensor Streams for Discovering Human Activity Patterns over Time , 2010, 2010 IEEE International Conference on Data Mining.

[17]  Henry A. Kautz,et al.  Real-time crowd labeling for deployable activity recognition , 2013, CSCW.

[18]  Claudio Bettini,et al.  SmartFABER: Recognizing fine-grained abnormal behaviors for early detection of mild cognitive impairment , 2016, Artif. Intell. Medicine.

[19]  Lina Yao,et al.  Learning from less for better: semi-supervised activity recognition via shared structure discovery , 2016, UbiComp.

[20]  Enamul Hoque,et al.  AALO: Activity recognition in smart homes using Active Learning in the presence of Overlapped activities , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[21]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[22]  Seng Wai Loke Representing and reasoning with situations for context-aware pervasive computing: a logic programming perspective , 2004, Knowl. Eng. Rev..

[23]  Gita Reese Sukthankar,et al.  Robust Active Learning Using Crowdsourced Annotations for Activity Recognition , 2011, Human Computation.

[24]  Qiang Yang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Transfer Learning for Activity Recognition via Sensor Mapping , 2022 .

[25]  Aderemi A. Atayero,et al.  Security Issues in Cloud Computing: The Potentials of Homomorphic Encryption , 2011 .

[26]  Ting Chen,et al.  Research on human activity recognition based on active learning , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[27]  Bernt Schiele,et al.  Exploring semi-supervised and active learning for activity recognition , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[28]  Yehuda Lindell,et al.  Secure Multiparty Computation for Privacy-Preserving Data Mining , 2009, IACR Cryptol. ePrint Arch..

[29]  Diane J Cook,et al.  Tracking Activities in Complex Settings Using Smart Environment Technologies. , 2009, International journal of biosciences, psychiatry, and technology.

[30]  Daniele Riboni,et al.  Web Mining & Computer Vision: New Partners for Object-Based Activity Recognition , 2017, 2017 IEEE 26th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE).

[31]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[32]  Chris D. Nugent,et al.  Ontology-based activity recognition in intelligent pervasive environments , 2009, Int. J. Web Inf. Syst..

[33]  Simon A. Dobson,et al.  USMART , 2014, ACM Trans. Interact. Intell. Syst..

[34]  Li-Chen Fu,et al.  Active-learning assisted self-reconfigurable activity recognition in a dynamic environment , 2009, 2009 IEEE International Conference on Robotics and Automation.

[35]  Jadwiga Indulska,et al.  Adaptive activity learning with dynamically available context , 2016, 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[36]  Juan Ye,et al.  SLearn: Shared learning human activity labels across multiple datasets , 2018, 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[37]  Andreas Savvides,et al.  Extracting spatiotemporal human activity patterns in assisted living using a home sensor network , 2008, PETRA '08.

[38]  Tao Gu,et al.  Ontology based context modeling and reasoning using OWL , 2004, IEEE Annual Conference on Pervasive Computing and Communications Workshops, 2004. Proceedings of the Second.

[39]  Kay Römer,et al.  Distributed Mining of Spatio-Temporal Event Patterns in Sensor Networks , 2007 .

[40]  Heiner Stuckenschmidt,et al.  A probabilistic ontological framework for the recognition of multilevel human activities , 2013, UbiComp.

[41]  Claudio Bettini,et al.  OWL 2 modeling and reasoning with complex human activities , 2011, Pervasive Mob. Comput..

[42]  Timo Sztyler,et al.  On-body localization of wearable devices: An investigation of position-aware activity recognition , 2016, 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[43]  Matthai Philipose,et al.  Unsupervised Activity Recognition Using Automatically Mined Common Sense , 2005, AAAI.