Human Activity Recognition Using Deep Models and Its Analysis from Domain Adaptation Perspective

Human activity recognition (HAR) is a broad area of research which solves the problem of determining a user’s activity from a set of observations recorded on video or low-level sensors (accelerometer, gyroscope, etc.) HAR has important applications in medical care and entertainment. In this paper, we address sensor-based HAR, because it could be deployed on a smartphone and eliminates the need to use additional equipment. Using machine learning methods for HAR is common. However, such, methods are vulnerable to changes in the domain of training and test data. More specifically, a model trained on data collected by one user loses accuracy when utilised by another user, because of the domain gap (differences in devices and movement pattern results in differences in sensors’ readings.) Despite significant results achieved in HAR, it is not well-investigated from domain adaptation (DA) perspective. In this paper, we implement a CNN-LSTM based architecture along with several classical machine learning methods for HAR and conduct a series of cross-domain tests. The result of this work is a collection of statistics on the performance of our model under DA task. We believe that our findings will serve as a foundation for future research in solving DA problem for HAR.

[1]  Philip S. Yu,et al.  Stratified Transfer Learning for Cross-domain Activity Recognition , 2017, 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[2]  David Windridge,et al.  A Comprehensive Classification of Deep Learning Libraries , 2018, Advances in Intelligent Systems and Computing.

[3]  Tae-Seong Kim,et al.  A Triaxial Accelerometer-Based Physical-Activity Recognition via Augmented-Signal Features and a Hierarchical Recognizer , 2010, IEEE Transactions on Information Technology in Biomedicine.

[4]  Elnaz Soleimani,et al.  Cross-Subject Transfer Learning in Human Activity Recognition Systems using Generative Adversarial Networks , 2019, Neurocomputing.

[5]  Wala'a N. Jasim,et al.  Human Activity Recognition System to Benefit Healthcare Field by using HOG and Harris Techniques with K-NN Model , 2018 .

[6]  Swati Nigam,et al.  On human activity recognition in video sequences , 2011, 2011 2nd International Conference on Computer and Communication Technology (ICCCT-2011).

[7]  A. M. Khan,et al.  Accelerometer signal-based human activity recognition using augmented autoregressive model coefficients and artificial neural nets , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[8]  Paul J. M. Havinga,et al.  Fusion of Smartphone Motion Sensors for Physical Activity Recognition , 2014, Sensors.

[9]  Sung-Bae Cho,et al.  Deep Convolutional Neural Networks for Human Activity Recognition with Smartphone Sensors , 2015, ICONIP.

[10]  Usha Mary Sharma,et al.  A study on human activity recognition from video , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[11]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[12]  Seok-Won Lee,et al.  Exploratory Data Analysis of Acceleration Signals to Select Light-Weight and Accurate Features for Real-Time Activity Recognition on Smartphones , 2013, Sensors.

[13]  Thomas Plötz,et al.  Ensembles of Deep LSTM Learners for Activity Recognition using Wearables , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[14]  David Howard,et al.  A Comparison of Feature Extraction Methods for the Classification of Dynamic Activities From Accelerometer Data , 2009, IEEE Transactions on Biomedical Engineering.

[15]  Zhongmin Wang,et al.  Human Activity Recognition Model Based on Decision Tree , 2013, 2013 International Conference on Advanced Cloud and Big Data.

[16]  Ming Zeng,et al.  Understanding and improving recurrent networks for human activity recognition by continuous attention , 2018, UbiComp.

[17]  Y.-K. Lee,et al.  Human Activity Recognition via an Accelerometer-Enabled-Smartphone Using Kernel Discriminant Analysis , 2010, 2010 5th International Conference on Future Information Technology.

[18]  Thomas George,et al.  An effective approach for human activity recognition on smartphone , 2015, 2015 IEEE International Conference on Engineering and Technology (ICETECH).

[19]  Attila Kertész-Farkas,et al.  RapidHARe: A computationally inexpensive method for real-time human activity recognition from wearable sensors , 2018, J. Ambient Intell. Smart Environ..

[20]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[21]  Ying Wang,et al.  Human Activity Recognition Based on R Transform , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Li Xiaowei.  Application of Decision Tree Classification Method Based on Information Entropy to Web Marketing , 2014, 2014 Sixth International Conference on Measuring Technology and Mechatronics Automation.

[23]  Seok-Won Lee,et al.  User-Independent Activity Recognition via Three-Stage GA-Based Feature Selection , 2014, Int. J. Distributed Sens. Networks.

[24]  Kai Kang,et al.  A New Multi-Layer Classification Method Based on Logistic Regression , 2018, 2018 13th International Conference on Computer Science & Education (ICCSE).

[25]  Patrick Olivier,et al.  Feature Learning for Activity Recognition in Ubiquitous Computing , 2011, IJCAI.

[26]  Adil Mehmood Khan,et al.  Activity Recognition on Smartphones via Sensor-Fusion and KDA-Based SVMs , 2014, Int. J. Distributed Sens. Networks.

[27]  Yoshua Bengio,et al.  Attention-Based Models for Speech Recognition , 2015, NIPS.

[28]  Bernt Schiele,et al.  A tutorial on human activity recognition using body-worn inertial sensors , 2014, CSUR.

[29]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[30]  Theodoros Giannakopoulos,et al.  A ROS framework for audio-based activity recognition , 2016, PETRA.

[31]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Parviz Asghari,et al.  Activity Recognition Using Hierarchical Hidden Markov Models on Streaming Sensor Data , 2018, 2018 9th International Symposium on Telecommunications (IST).

[33]  Ming Zeng,et al.  Semi-supervised convolutional neural networks for human activity recognition , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[34]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[35]  George J. Knafl,et al.  Logistic regression modeling for context-based classification , 1999, Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99.

[36]  R. Rodrigo,et al.  Faster human activity recognition with SVM , 2012, International Conference on Advances in ICT for Emerging Regions (ICTer2012).

[37]  Andrea Vitaletti,et al.  Comparison of Decision Tree Based Classification Strategies to Detect External Chemical Stimuli from Raw and Filtered Plant Electrical Response , 2017, ArXiv.