Weakly Supervised Human Activity Recognition From Wearable Sensors by Recurrent Attention Learning

Traditional methods of human activity recognition from wearable sensors rely on good training datasets in which thousands of training sequences should be carefully labeled. However, unlike images or videos which can be easily classified by human beings, strictly labeling such sequences of sensor data needs much more manpower and computing resources. In this paper, we present a new weakly supervised human activity recognition model based on recurrent attention learning, in which an agent is trained to extract information from weakly labeled sensor data by adaptively selecting a sequence of locations. Since, the model is non-differentiable and multiple activities may occur in a sequence of sensor data, it is trained by reinforcement learning with novel reward strategies. We evaluated our model on the traditional UCI HAR dataset and our collected weakly labeled dataset. The experimental results show that our model is superior to the traditional CNN model and the DeepConvLSTM model on both datasets.

[1]  Gary M. Weiss,et al.  Activity recognition using cell phone accelerometers , 2011, SKDD.

[2]  Diane J. Cook,et al.  Keeping the Resident in the Loop: Adapting the Smart Home to the User , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[3]  Saso Dzeroski,et al.  Combining Classifiers with Meta Decision Trees , 2003, Machine Learning.

[4]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[5]  Michael L. Littman,et al.  Activity Recognition from Accelerometer Data , 2005, AAAI.

[6]  Friedrich Foerster,et al.  Detection of posture and motion by accelerometry : a validation study in ambulatory monitoring , 1999 .

[7]  Jung-Yoon Kim,et al.  Unobtrusive Monitoring to Detect Depression for Elderly With Chronic Illnesses , 2017, IEEE Sensors Journal.

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  Tatsuo Nakajima,et al.  Feature Selection and Activity Recognition from Wearable Sensors , 2006, UCS.

[10]  Duc Ngoc Tran,et al.  Human Activities Recognition in Android Smartphone Using Support Vector Machine , 2016, 2016 7th International Conference on Intelligent Systems, Modelling and Simulation (ISMS).

[11]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[12]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[13]  Zhaozheng Yin,et al.  Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks , 2015, ACM Multimedia.

[14]  Angelo M. Sabatini,et al.  Machine Learning Methods for Classifying Human Physical Activity from On-Body Accelerometers , 2010, Sensors.

[15]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[16]  Guo-Jun Qi,et al.  Differential Recurrent Neural Networks for Action Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Zhi-Hua Zhou,et al.  A brief introduction to weakly supervised learning , 2018 .

[18]  Senem Velipasalar,et al.  A Survey on Activity Detection and Classification Using Wearable Sensors , 2017, IEEE Sensors Journal.

[19]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[20]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[21]  Wanmin Wu,et al.  Classification Accuracies of Physical Activities Using Smartphone Motion Sensors , 2012, Journal of medical Internet research.

[22]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[23]  Nabil Zerrouki,et al.  Vision-Based Human Action Classification Using Adaptive Boosting Algorithm , 2018, IEEE Sensors Journal.

[24]  Ian H. Witten,et al.  Weka: Practical machine learning tools and techniques with Java implementations , 1999 .

[25]  William Robson Schwartz,et al.  Human Activity Recognition Based on Wearable Sensor Data: A Standardization of the State-of-the-Art , 2018, ArXiv.

[26]  Enrico Vicario,et al.  Using Temporal Logic and Model Checking in Automated Recognition of Human Activities for Ambient-Assisted Living , 2013, IEEE Transactions on Human-Machine Systems.

[27]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[28]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[29]  Xiaoli Li,et al.  Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition , 2015, IJCAI.

[30]  Sung-Bae Cho,et al.  Deep Convolutional Neural Networks for Human Activity Recognition with Smartphone Sensors , 2015, ICONIP.

[31]  Yuwei Chen,et al.  Human Behavior Cognition Using Smartphone Sensors , 2013, Sensors.

[32]  Paul J. M. Havinga,et al.  A Survey of Online Activity Recognition Using Mobile Phones , 2015, Sensors.

[33]  Koray Kavukcuoglu,et al.  Visual Attention , 2020, Computational Models for Cognitive Vision.

[34]  Seungjin Choi,et al.  Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[35]  Jeen-Shing Wang,et al.  Development of a portable activity detector for daily activity recognition , 2009, 2009 IEEE International Symposium on Industrial Electronics.

[36]  Paul J. M. Havinga,et al.  Activity Recognition Using Inertial Sensing for Healthcare, Wellbeing and Sports Applications: A Survey , 2010, ARCS Workshops.

[37]  Yuqing Chen,et al.  A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[38]  Thomas Plötz,et al.  Ensembles of Deep LSTM Learners for Activity Recognition using Wearables , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[39]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.