Activity recognition through interactive machine learning in a dynamic sensor setting

The advances in Internet of things lead to an increased number of devices generating and streaming data. These devices can be useful data sources for activity recognition by using machine learning. However, the set of available sensors may vary over time, e.g. due to mobility of the sensors and technical failures. Since the machine learning model uses the data streams from the sensors as input, it must be able to handle a varying number of input variables, i.e. that the feature space might change over time. Moreover, the labelled data necessary for the training is often costly to acquire. In active learning, the model is given a budget for requesting labels from an oracle, and aims to maximize accuracy by careful selection of what data instances to label. It is generally assumed that the role of the oracle only is to respond to queries and that it will always do so. In many real-world scenarios however, the oracle is a human user and the assumptions are simplifications that might not give a proper depiction of the setting. In this work we investigate different interactive machine learning strategies, out of which active learning is one, which explore the effects of an oracle that can be more proactive and factors that might influence a user to provide or withhold labels. We implement five interactive machine learning strategies as well as hybrid versions of them and evaluate them on two datasets. The results show that a more proactive user can improve the performance, especially when the user is influenced by the accuracy of earlier predictions. The experiments also highlight challenges related to evaluating performance when the set of classes is changing over time.

[1]  Kristof Van Laerhoven,et al.  Assessing activity recognition feedback in long-term psychology trials , 2015, MUM.

[2]  Changseok Bae,et al.  Unsupervised learning for human activity recognition using smartphone sensors , 2014, Expert Syst. Appl..

[3]  Bernt Schiele,et al.  Activity Recognition from Sparsely Labeled Data Using Multi-Instance Learning , 2009, LoCA.

[4]  Xiaojin Zhu,et al.  Machine Teaching: An Inverse Problem to Machine Learning and an Approach Toward Optimal Education , 2015, AAAI.

[5]  Agnes Tegen,et al.  Towards a taxonomy of interactive continual and multimodal learning for the internet of things , 2019, UbiComp/ISWC Adjunct.

[6]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[7]  Amit K. Roy-Chowdhury,et al.  A Continuous Learning Framework for Activity Recognition Using Deep Hybrid Feature Models , 2015, IEEE Transactions on Multimedia.

[8]  Paul Davidsson,et al.  Collaborative Sensing with Interactive Learning using Dynamic Intelligent Virtual Sensors , 2019, Sensors.

[9]  Rashid Mehmood,et al.  Data Fusion and IoT for Smart Ubiquitous Environments: A Survey , 2017, IEEE Access.

[10]  Sandra Zilles,et al.  An Overview of Machine Teaching , 2018, ArXiv.

[11]  Jaime G. Carbonell,et al.  Proactive learning: cost-sensitive active learning with multiple imperfect oracles , 2008, CIKM '08.

[12]  Bartosz Krawczyk,et al.  Active and adaptive ensemble learning for online activity recognition from data streams , 2017, Knowl. Based Syst..

[13]  Aitor Almeida,et al.  A Scalable Hybrid Activity Recognition Approach for Intelligent Environments , 2018, IEEE Access.

[14]  Sung-Bae Cho,et al.  Human activity recognition with smartphone sensors using deep learning neural networks , 2016, Expert Syst. Appl..

[15]  Erdogan Dogdu,et al.  Context-Aware Computing, Learning, and Big Data in Internet of Things: A Survey , 2018, IEEE Internet of Things Journal.

[16]  Amit P. Sheth,et al.  Machine learning for Internet of Things data analysis: A survey , 2017, Digit. Commun. Networks.

[17]  Edwin Lughofer,et al.  On-line active learning: A new paradigm to improve practical useability of data stream modeling methods , 2017, Inf. Sci..

[18]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[19]  Arkady B. Zaslavsky,et al.  Context Aware Computing for The Internet of Things: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[20]  A. Thomaz,et al.  Mixed-Initiative Active Learning , 2012 .

[21]  Bernt Schiele,et al.  Exploring semi-supervised and active learning for activity recognition , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[22]  João Gama,et al.  A survey on concept drift adaptation , 2014, ACM Comput. Surv..

[23]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[24]  Luis M. Candanedo,et al.  Accurate occupancy detection of an office room from light, temperature, humidity and CO2 measurements using statistical learning models , 2016 .

[25]  Timo Sztyler,et al.  Unsupervised recognition of interleaved activities of daily living through ontological and probabilistic reasoning , 2016, UbiComp.

[26]  Jiang Wang,et al.  Feedback-driven multiclass active learning for data streams , 2013, CIKM.

[27]  Paul Davidsson,et al.  Interactive Machine Learning for the Internet of Things: A Case Study on Activity Detection , 2019, IOT.

[28]  Geoff Holmes,et al.  Active Learning With Drifting Streaming Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Ricardo Chavarriaga,et al.  The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition , 2013, Pattern Recognit. Lett..

[30]  Abdus Samad,et al.  A Study of Machine Learning in Wireless Sensor Network , 2017 .

[31]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[32]  Gabriel J. Brostow,et al.  Becoming the expert - interactive multi-class machine teaching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Edwin Naroska,et al.  Unsupervised Recognition of ADLs , 2010, SETN.

[34]  Hermann Hellwagner,et al.  Batch-based active learning: Application to social media data for crisis management , 2018, Expert Syst. Appl..

[35]  Salima Benbernou,et al.  A survey on service quality description , 2013, CSUR.

[36]  Simon A. Dobson,et al.  USMART , 2014, ACM Trans. Interact. Intell. Syst..