NECTAR: Knowledge-based Collaborative Active Learning for Activity Recognition

Due to the emerging popularity of pervasive healthcare applications, tools for monitoring activities in smart homes are gaining momentum. Existing methods mainly rely on supervised learning algorithms for recognizing activities based on sensor data. A key issue with those approaches is the acquisition of comprehensive training sets of activities. Indeed, that task incurs significant costs in terms of manual labeling effort; moreover, labeling by external observers violates the individual's privacy. For these reasons, there is an increasing interest in unsupervised activity recognition methods. A popular approach relies on knowledge-based models expressed by ontologies of activities, environment and sensors. Unfortunately, those models require significant knowledge engineering efforts, and are often limited to a specific application. In this paper, we address the issues of existing methods by proposing a novel hybrid approach. Our intuition is that a generic knowledge-based model of activities can be refined to target specific individuals and environments by collaboratively acquiring feedback from inhabitants. Specifically, we propose a collaborative active learning method to refine correlations among sensor events and activity types that are initially extracted from a high-level ontology. Generic correlations are personalized to each target smart-home considering the similarity between the feedback target and the feedback provider in terms of environment and inhabitant's profiles. Moreover, thanks to this method, new sensors installed in the home are seamlessly integrated in the recognition framework. In order to reduce the burden of providing feedback, we also propose a technique to carefully select the conditions that trigger a feedback request. We conducted experiments with a real-world dataset and a generic ontology of activities. Results show that our hybrid method outperforms state-of-the-art supervised and unsupervised activity recognition techniques while triggering an acceptable number of feedback queries.

[1]  Diane J. Cook,et al.  Transfer learning for activity recognition: a survey , 2013, Knowledge and Information Systems.

[2]  Timo Sztyler,et al.  Unsupervised recognition of interleaved activities of daily living through ontological and probabilistic reasoning , 2016, UbiComp.

[3]  Majid Sarrafzadeh,et al.  Toward Unsupervised Activity Discovery Using Multi-Dimensional Motif Detection in Time Series , 2009, IJCAI.

[4]  Matthai Philipose,et al.  Unsupervised Activity Recognition Using Automatically Mined Common Sense , 2005, AAAI.

[5]  Nirmalya Roy,et al.  Pervasive and Mobile Computing , 2022 .

[6]  Matthai Philipose,et al.  Mining models of human activities from the web , 2004, WWW '04.

[7]  Claudio Bettini,et al.  SmartFABER: Recognizing fine-grained abnormal behaviors for early detection of mild cognitive impairment , 2016, Artif. Intell. Medicine.

[8]  Enamul Hoque,et al.  AALO: Activity recognition in smart homes using Active Learning in the presence of Overlapped activities , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[9]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[10]  Chris D. Nugent,et al.  Ontology-based activity recognition in intelligent pervasive environments , 2009, Int. J. Web Inf. Syst..

[11]  Simon A. Dobson,et al.  USMART , 2014, ACM Trans. Interact. Intell. Syst..

[12]  Seng Wai Loke Representing and reasoning with situations for context-aware pervasive computing: a logic programming perspective , 2004, Knowl. Eng. Rev..

[13]  Li-Chen Fu,et al.  Active-learning assisted self-reconfigurable activity recognition in a dynamic environment , 2009, 2009 IEEE International Conference on Robotics and Automation.

[14]  Yehuda Lindell,et al.  Secure Multiparty Computation for Privacy-Preserving Data Mining , 2009, IACR Cryptol. ePrint Arch..

[15]  Jadwiga Indulska,et al.  Adaptive activity learning with dynamically available context , 2016, 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[16]  Aderemi A. Atayero,et al.  Security Issues in Cloud Computing: The Potentials of Homomorphic Encryption , 2011 .

[17]  Timo Sztyler,et al.  Online personalization of cross-subjects based activity recognition models on wearable devices , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[18]  Heiner Stuckenschmidt,et al.  Markov Logic Networks with Numerical Constraints , 2016, ECAI.

[19]  Matthai Philipose,et al.  Building Reliable Activity Models Using Hierarchical Shrinkage and Mined Ontology , 2006, Pervasive.

[20]  Diane J Cook,et al.  Tracking Activities in Complex Settings Using Smart Environment Technologies. , 2009, International journal of biosciences, psychiatry, and technology.

[21]  Daniele Riboni,et al.  Web Mining & Computer Vision: New Partners for Object-Based Activity Recognition , 2017, 2017 IEEE 26th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE).

[22]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[23]  Diane J. Cook,et al.  CASAS: A Smart Home in a Box , 2013, Computer.

[24]  Henry A. Kautz,et al.  Real-time crowd labeling for deployable activity recognition , 2013, CSCW.

[25]  Lina Yao,et al.  Learning from less for better: semi-supervised activity recognition via shared structure discovery , 2016, UbiComp.

[26]  Diane J. Cook,et al.  Activity recognition on streaming sensor data , 2014, Pervasive Mob. Comput..

[27]  Lawrence B. Holder,et al.  Discovering Activities to Recognize and Track in a Smart Environment , 2011, IEEE Transactions on Knowledge and Data Engineering.

[28]  Gita Reese Sukthankar,et al.  Robust Active Learning Using Crowdsourced Annotations for Activity Recognition , 2011, Human Computation.

[29]  Qiang Yang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Transfer Learning for Activity Recognition via Sensor Mapping , 2022 .

[30]  Heiner Stuckenschmidt,et al.  A probabilistic ontological framework for the recognition of multilevel human activities , 2013, UbiComp.

[31]  Claudio Bettini,et al.  OWL 2 modeling and reasoning with complex human activities , 2011, Pervasive Mob. Comput..

[32]  Ting Chen,et al.  Research on human activity recognition based on active learning , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[33]  Bernt Schiele,et al.  Exploring semi-supervised and active learning for activity recognition , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[34]  Maureen Schmitter-Edgecombe,et al.  Automated Cognitive Health Assessment Using Smart Home Monitoring of Complex Tasks , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[35]  Tao Gu,et al.  Ontology based context modeling and reasoning using OWL , 2004, IEEE Annual Conference on Pervasive Computing and Communications Workshops, 2004. Proceedings of the Second.