AROMA

Human activity recognition (HAR) is a promising research issue in ubiquitous and wearable computing. However, there are some problems existing in traditional methods: 1) They treat HAR as a single label classification task, and ignore the information from other related tasks, which is helpful for the original task. 2) They need to predesign features artificially, which are heuristic and not tightly related to HAR task. To address these problems, we propose AROMA (human activity recognition using deep multi-task learning). Human activities can be divided into simple and complex activities. They are closely linked. Simple and complex activity recognitions are two related tasks in AROMA. For simple activity recognition task, AROMA utilizes a convolutional neural network (CNN) to extract deep features, which are task dependent and nonhandcrafted. For complex activity recognition task, AROMA applies a long short-term memory (LSTM) network to learn the temporal context of activity data. In addition, there is a shared structure between the two tasks, and the object functions of these two tasks are optimized jointly. We evaluate AROMA on two public datasets, and the experimental results show that AROMA is able to yield a competitive performance in both simple and complex activity recognitions. 1

[1]  Thomas Plötz,et al.  Ensembles of Deep LSTM Learners for Activity Recognition using Wearables , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[2]  Diane J. Cook,et al.  Simple and Complex Activity Recognition through Smart Phones , 2012, 2012 Eighth International Conference on Intelligent Environments.

[3]  Takuya Maekawa,et al.  Toward practical factory activity recognition: unsupervised understanding of repetitive assembly work in a factory , 2016, UbiComp.

[4]  Samuel Kaski,et al.  Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) , 2016, IJCAI 2016.

[5]  Xu Sun,et al.  A New Multi-task Learning Method for Personalized Activity Recognition , 2011, 2011 IEEE 11th International Conference on Data Mining.

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Tim Dallas,et al.  Feature Selection and Activity Recognition System Using a Single Triaxial Accelerometer , 2014, IEEE Transactions on Biomedical Engineering.

[8]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[9]  Ling Chen,et al.  Bi-View Semi-Supervised Learning Based Semantic Human Activity Recognition Using Accelerometers , 2018, IEEE Transactions on Mobile Computing.

[10]  Xiaoou Tang,et al.  Facial Landmark Detection by Deep Multi-task Learning , 2014, ECCV.

[11]  Ming Liu,et al.  Sensor-based human activity recognition system with a multilayered model using time series shapelets , 2015, Knowl. Based Syst..

[12]  LiuLi,et al.  Sensor-based human activity recognition system with a multilayered model using time series shapelets , 2015 .

[13]  Stefan C. Kremer,et al.  Recurrent Neural Networks , 2013, Handbook on Neural Information Processing.

[14]  Fanglin Chen,et al.  StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones , 2014, UbiComp.

[15]  Daniel Roggen,et al.  Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations , 2016, SEMWEB.

[16]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[18]  Miguel A. Labrador,et al.  Centinela: A human activity recognition system based on acceleration and vital sign data , 2012, Pervasive Mob. Comput..

[19]  Qiang Yang,et al.  An Overview of Multi-task Learning , 2018 .

[20]  Lian-Wen Jin,et al.  Activity recognition from acceleration data using AR model representation and SVM , 2008, 2008 International Conference on Machine Learning and Cybernetics.

[21]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[22]  Bernt Schiele,et al.  Remember and transfer what you have learned - recognizing composite activities based on activity spotting , 2010, International Symposium on Wearable Computers (ISWC) 2010.

[23]  Yuxin Peng,et al.  Complex activity recognition using time series pattern dictionary learned from ubiquitous sensors , 2016, Inf. Sci..

[24]  Bernt Schiele,et al.  Daily Routine Recognition through Activity Spotting , 2009, LoCA.

[25]  Sebastian Ruder,et al.  An Overview of Multi-Task Learning in Deep Neural Networks , 2017, ArXiv.

[26]  Bernt Schiele,et al.  Discovery of activity patterns using topic models , 2008 .

[27]  Hongnian Yu,et al.  Elderly activities recognition and classification for applications in assisted living , 2013, Expert Syst. Appl..

[28]  Thomas Plötz,et al.  Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables , 2016, IJCAI.

[29]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[30]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[31]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[32]  Xiaoli Li,et al.  Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition , 2015, IJCAI.

[33]  Hal Daumé,et al.  Learning Task Grouping and Overlap in Multi-task Learning , 2012, ICML.

[34]  Xiaojie Wu,et al.  Hierarchical Complex Activity Representation and Recognition Using Topic Model and Classifier Level Fusion , 2017, IEEE Transactions on Biomedical Engineering.

[35]  Xu Sun,et al.  Large-Scale Personalized Human Activity Recognition Using Online Multitask Learning , 2013, IEEE Transactions on Knowledge and Data Engineering.