Leveraging Wearable Sensors for Human Daily Activity Recognition with Stacked Denoising Autoencoders

Activity recognition has received considerable attention in many research fields, such as industrial and healthcare fields. However, many researches about activity recognition have focused on static activities and dynamic activities in current literature, while, the transitional activities, such as stand-to-sit and sit-to-stand, are more difficult to recognize than both of them. Consider that it may be important in real applications. Thus, a novel framework is proposed in this paper to recognize static activities, dynamic activities, and transitional activities by utilizing stacked denoising autoencoders (SDAE), which is able to extract features automatically as a deep learning model rather than utilize manual features extracted by conventional machine learning methods. Moreover, the resampling technique (random oversampling) is used to improve problem of unbalanced samples due to relatively short duration characteristic of transitional activity. The experiment protocol is designed to collect twelve daily activities (three types) by using wearable sensors from 10 adults in smart lab of Ulster University, the experiment results show the significant performance on transitional activity recognition and achieve the overall accuracy of 94.88% on three types of activities. The results obtained by comparing with other methods and performances on other three public datasets verify the feasibility and priority of our framework. This paper also explores the effect of multiple sensors (accelerometer and gyroscope) to determine the optimal combination for activity recognition.

[1]  Sophia Bano,et al.  Deep Human Activity Recognition With Localisation of Wearable Sensors , 2020, IEEE Access.

[2]  Xiao Zhang,et al.  Device-Free Wireless Localization and Activity Recognition: A Deep Learning Approach , 2017, IEEE Transactions on Vehicular Technology.

[3]  Guilin Chen,et al.  Human Activity Recognition in a Smart Home Environment with Stacked Denoising Autoencoders , 2016, WAIM Workshops.

[4]  Shaohan Hu,et al.  DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing , 2016, WWW.

[5]  John D. Kelleher,et al.  Towards a Deep Learning-based Activity Discovery System , 2016, AICS.

[6]  Yufei Chen,et al.  Performance Analysis of Smartphone-Sensor Behavior for Human Activity Recognition , 2017, IEEE Access.

[7]  Davide Anguita,et al.  Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine , 2012, IWAAL.

[8]  Billur Barshan,et al.  Comparative study on classifying human activities with miniature inertial and magnetic sensors , 2010, Pattern Recognit..

[9]  Huaijun Wang,et al.  Segmentation and Recognition of Basic and Transitional Activities for Continuous Physical Human Activity , 2019, IEEE Access.

[10]  Marco Morana,et al.  Human Activity Recognition Process Using 3-D Posture Data , 2015, IEEE Transactions on Human-Machine Systems.

[11]  Carmen C. Y. Poon,et al.  Unobtrusive Sensing and Wearable Devices for Health Informatics , 2014, IEEE Transactions on Biomedical Engineering.

[12]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[13]  Kamiar Aminian,et al.  Classification and characterization of postural transitions using instrumented shoes , 2017, Medical & Biological Engineering & Computing.

[14]  Munoz-Organero Mario,et al.  Human Activity Recognition Based on Single Sensor Square HV Acceleration Images and Convolutional Neural Networks , 2019, IEEE Sensors Journal.

[15]  Yuqing Chen,et al.  A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[16]  Xiaodong Yang,et al.  Super Normal Vector for Human Activity Recognition with Depth Cameras , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  James Bruce Lee,et al.  Decision-tree-based human activity classification algorithm using single-channel foot-mounted gyroscope , 2015 .

[18]  Shahrokh Valaee,et al.  Locomotion Activity Recognition Using Stacked Denoising Autoencoders , 2018, IEEE Internet of Things Journal.

[19]  Jianbo Yang,et al.  Deep Learning for Human Activity Recognition , 2020 .

[20]  Tae-Seong Kim,et al.  A Triaxial Accelerometer-Based Physical-Activity Recognition via Augmented-Signal Features and a Hierarchical Recognizer , 2010, IEEE Transactions on Information Technology in Biomedicine.

[21]  Petia Radeva,et al.  Human Activity Recognition from Accelerometer Data Using a Wearable Device , 2011, IbPRIA.

[22]  Mianxiong Dong,et al.  Robust Activity Recognition for Aging Society , 2018, IEEE Journal of Biomedical and Health Informatics.

[23]  Genming Ding,et al.  Human activity recognition method based on inertial sensor and barometer , 2018, 2018 IEEE International Symposium on Inertial Sensors and Systems (INERTIAL).

[24]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[25]  Tim Dallas,et al.  Feature Selection and Activity Recognition System Using a Single Triaxial Accelerometer , 2014, IEEE Transactions on Biomedical Engineering.

[26]  Seok-Won Lee,et al.  Exploratory Data Analysis of Acceleration Signals to Select Light-Weight and Accurate Features for Real-Time Activity Recognition on Smartphones , 2013, Sensors.

[27]  Ming-Ai Li,et al.  A novel feature extraction method for scene recognition based on Centered Convolutional Restricted Boltzmann Machines , 2015, Neurocomputing.

[28]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[29]  Arno Knobbe,et al.  Activity recognition using wearable sensors for tracking the elderly , 2020, User Modeling and User-Adapted Interaction.

[30]  Paul Lukowicz,et al.  Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  A. Duarte,et al.  The Five Times Sit-to-Stand Test: safety and reliability with older intensive care unit patients at discharge , 2019, Revista Brasileira de terapia intensiva.

[32]  Guang-Zhong Yang,et al.  Transitional Activity Recognition with Manifold Embedding , 2009, 2009 Sixth International Workshop on Wearable and Implantable Body Sensor Networks.

[33]  Junqi Guo,et al.  Motion Recognition by Using a Stacked Autoencoder-Based Deep Learning Algorithm with Smart Phones , 2015, WASA.

[34]  Lei He,et al.  Human activity recognition based on feature selection in smart home using back-propagation algorithm. , 2014, ISA transactions.

[35]  Augustine Ikpehai,et al.  Deep Sensing: Inertial and Ambient Sensing for Activity Context Recognition Using Deep Convolutional Neural Networks , 2020, Sensors.

[36]  Reza Malekian,et al.  Physical Activity Recognition From Smartphone Accelerometer Data for User Context Awareness Sensing , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[37]  Long Qin,et al.  Human Activity Recognition with Smartphone Inertial Sensors Using Bidir-LSTM Networks , 2018, 2018 3rd International Conference on Mechanical, Control and Computer Engineering (ICMCCE).

[38]  Zhenyu He,et al.  Activity recognition from acceleration data based on discrete consine transform and SVM , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[39]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[40]  Sardar Jaf,et al.  Deep Learning for Natural Language Parsing , 2019, IEEE Access.

[41]  Thamer Alhussain,et al.  Speech Emotion Recognition Using Deep Learning Techniques: A Review , 2019, IEEE Access.

[42]  Davide Anguita,et al.  Transition-Aware Human Activity Recognition Using Smartphones , 2016, Neurocomputing.

[43]  Petar M. Djuric,et al.  Resampling algorithms and architectures for distributed particle filters , 2005, IEEE Transactions on Signal Processing.

[44]  Jon Atli Benediktsson,et al.  Deep Learning for Hyperspectral Image Classification: An Overview , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Yeng Chai Soh,et al.  Robust Human Activity Recognition Using Smartphone Sensors via CT-PCA and Online SVM , 2017, IEEE Transactions on Industrial Informatics.

[46]  Lianwen Jin,et al.  Ensemble Manifold Rank Preserving for Acceleration-Based Human Activity Recognition , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Josef Hallberg,et al.  Hardware for Recognition of Human Activities: A Review of Smart Home and AAL Related Technologies , 2020, Sensors.

[48]  Xiaofei Xu,et al.  Activity Recognition Method for Home-Based Elderly Care Service Based on Random Forest and Activity Similarity , 2019, IEEE Access.

[49]  Ying Wah Teh,et al.  Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges , 2018, Expert Syst. Appl..

[50]  Mohamad Khalil,et al.  Recognition of different daily living activities using hidden Markov model regression , 2016, 2016 3rd Middle East Conference on Biomedical Engineering (MECBME).

[51]  Jesse Hoey,et al.  Sensor-Based Activity Recognition , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[52]  Francisco Herrera,et al.  A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based Approaches , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[53]  Sung-Bae Cho,et al.  Human activity recognition with smartphone sensors using deep learning neural networks , 2016, Expert Syst. Appl..

[54]  Yu-Liang Hsu,et al.  Human Daily and Sport Activity Recognition Using a Wearable Inertial Sensor Network , 2018, IEEE Access.

[55]  Sang Min Yoon,et al.  Human activity recognition from accelerometer data using Convolutional Neural Network , 2017, 2017 IEEE International Conference on Big Data and Smart Computing (BigComp).

[56]  Takeshi Nishida,et al.  Deep recurrent neural network for mobile human activity recognition with high throughput , 2017, Artificial Life and Robotics.

[57]  Sattar Hashemi,et al.  To Combat Multi-Class Imbalanced Problems by Means of Over-Sampling Techniques , 2016, IEEE Transactions on Knowledge and Data Engineering.

[58]  Yu Guan,et al.  Deep Learning for Human Activity Recognition in Mobile Computing , 2018, Computer.

[59]  Niall Twomey,et al.  Energy-efficient activity recognition framework using wearable accelerometers , 2020, J. Netw. Comput. Appl..

[60]  Diane J. Cook,et al.  Simple and Complex Activity Recognition through Smart Phones , 2012, 2012 Eighth International Conference on Intelligent Environments.