A Comparative Analysis of Hybrid Deep Learning Models for Human Activity Recognition

Recent advances in artificial intelligence and machine learning (ML) led to effective methods and tools for analyzing the human behavior. Human Activity Recognition (HAR) is one of the fields that has seen an explosive research interest among the ML community due to its wide range of applications. HAR is one of the most helpful technology tools to support the elderly’s daily life and to help people suffering from cognitive disorders, Parkinson’s disease, dementia, etc. It is also very useful in areas such as transportation, robotics and sports. Deep learning (DL) is a branch of ML based on complex Artificial Neural Networks (ANNs) that has demonstrated a high level of accuracy and performance in HAR. Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are two types of DL models widely used in the recent years to address the HAR problem. The purpose of this paper is to investigate the effectiveness of their integration in recognizing daily activities, e.g., walking. We analyze four hybrid models that integrate CNNs with four powerful RNNs, i.e., LSTMs, BiLSTMs, GRUs and BiGRUs. The outcomes of our experiments on the PAMAP2 dataset indicate that our proposed hybrid models achieve an outstanding level of performance with respect to several indicative measures, e.g., F-score, accuracy, sensitivity, and specificity.

[1]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[2]  Paolo Fornacciari,et al.  IoT Wearable Sensor and Deep Learning: An Integrated Approach for Personalized Human Activity Recognition in a Smart Home Environment , 2019, IEEE Internet of Things Journal.

[3]  Fernando Fernández Martínez,et al.  Human activity recognition adapted to the type of movement , 2020, Comput. Electr. Eng..

[4]  Davide Anguita,et al.  Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine , 2012, IWAAL.

[5]  María de Lourdes Martínez-Villaseñor,et al.  A Novel Wearable Sensor-Based Human Activity Recognition Approach Using Artificial Hydrocarbon Networks , 2016, Sensors.

[6]  Deba Prasad Dash,et al.  Hidden Markov Model based human activity recognition using shape and optical flow based features , 2016, 2016 IEEE Region 10 Conference (TENCON).

[7]  Dimitris Vrakas,et al.  Non-intrusive human activity recognition and abnormal behavior detection on elderly people: a review , 2019, Artificial Intelligence Review.

[8]  Zhongmin Wang,et al.  Human Activity Recognition Model Based on Decision Tree , 2013, 2013 International Conference on Advanced Cloud and Big Data.

[9]  Md. Kamrul Hasan,et al.  Activity recognition of a badminton game through accelerometer and gyroscope , 2016, 2016 19th International Conference on Computer and Information Technology (ICCIT).

[10]  Ming Zeng,et al.  Understanding and improving recurrent networks for human activity recognition by continuous attention , 2018, UbiComp.

[11]  Claudio Bettini,et al.  SmartWheels: Detecting urban features for wheelchair users' navigation , 2020, Pervasive Mob. Comput..

[12]  Yunhao Liu,et al.  Deep Learning for Sensor-based Human Activity Recognition , 2020, ACM Comput. Surv..

[13]  Ali Mahmood Khan,et al.  Recognizing Physical Activities Using Wii Remote , 2013 .

[14]  Jim Tørresen,et al.  A Robust Human Activity Recognition Approach Using OpenPose, Motion Features, and Deep Recurrent Neural Network , 2019, SCIA.

[15]  Negar Golestani,et al.  Human activity recognition using magnetic induction-based motion signals and deep recurrent neural networks , 2020, Nature Communications.

[16]  Md. Rashedul Islam,et al.  Enhanced Human Activity Recognition Based on Smartphone Sensor Data Using Hybrid Feature Selection Model , 2020, Sensors.

[17]  Vicente Matellán Olivera,et al.  A context‐awareness model for activity recognition in robot‐assisted scenarios , 2020, Expert Syst. J. Knowl. Eng..

[18]  Didier Stricker,et al.  Introducing a New Benchmarked Dataset for Activity Monitoring , 2012, 2012 16th International Symposium on Wearable Computers.

[19]  Wenjun Zeng,et al.  Spatio-Temporal Attention-Based LSTM Networks for 3D Action Recognition and Detection , 2018, IEEE Transactions on Image Processing.

[20]  Ting Chen,et al.  Research on human activity recognition based on active learning , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[21]  Matthieu Geist,et al.  Human Activity Recognition Using Recurrent Neural Networks , 2017, CD-MAKE.

[22]  Wan-Yu Deng,et al.  Cross-person activity recognition using reduced kernel extreme learning machine , 2014, Neural Networks.

[23]  Vinod Chandran,et al.  Physical Activity Recognition Using Posterior-Adapted Class-Based Fusion of Multiaccelerometer Data , 2017, IEEE Journal of Biomedical and Health Informatics.

[24]  Shaohua Wan,et al.  Deep Learning Models for Real-time Human Activity Recognition with Smartphones , 2019, Mobile Networks and Applications.

[25]  Davide Anguita,et al.  Transition-Aware Human Activity Recognition Using Smartphones , 2016, Neurocomputing.

[26]  Antonio Corradi,et al.  Activity recognition for Smart City scenarios: Google Play Services vs. MoST facilities , 2014, 2014 IEEE Symposium on Computers and Communications (ISCC).

[27]  Fan Yang,et al.  Efficient health-related abnormal behavior detection with visual and inertial sensor integration , 2019, Pattern Analysis and Applications.

[28]  Sung-Bae Cho,et al.  Human activity recognition with smartphone sensors using deep learning neural networks , 2016, Expert Syst. Appl..

[29]  Jae-Young Pyun,et al.  Deep Recurrent Neural Networks for Human Activity Recognition , 2017, Sensors.

[30]  Billur Barshan,et al.  Sensor-Activity Relevance in Human Activity Recognition with Wearable Motion Sensors and Mutual Information Criterion , 2013, ISCIS.

[31]  Faicel Chamroukhi,et al.  Physical Human Activity Recognition Using Wearable Sensors , 2015, Sensors.

[32]  M. Tahar Kechadi,et al.  Human Activity Recognition with Convolutional Neural Networks , 2018, ECML/PKDD.

[33]  Thomas Plötz,et al.  Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables , 2016, IJCAI.

[34]  Matthieu Geist,et al.  Convolutional and Recurrent Neural Networks for Activity Recognition in Smart Environment , 2015, BIRS-IMLKE.

[35]  Wanmin Wu,et al.  Classification Accuracies of Physical Activities Using Smartphone Motion Sensors , 2012, Journal of medical Internet research.

[36]  Shenghui Zhao,et al.  A Comparative Study on Human Activity Recognition Using Inertial Sensors in a Smartphone , 2016, IEEE Sensors Journal.

[37]  Thaier Hayajneh,et al.  Smartphone and Smartwatch-Based Biometrics Using Activities of Daily Living , 2019, IEEE Access.

[38]  Seungjin Choi,et al.  Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[39]  João Gama,et al.  Human Activity Recognition Using Inertial Sensors in a Smartphone: An Overview , 2019, Sensors.

[40]  Nirmalya Roy,et al.  Recent trends in machine learning for human activity recognition—A survey , 2018, WIREs Data Mining Knowl. Discov..

[41]  Tahmina Zebin,et al.  Human activity recognition with inertial sensors using a deep learning approach , 2016, 2016 IEEE SENSORS.

[42]  Franca Delmastro,et al.  Cognitive Training and Stress Detection in MCI Frail Older People Through Wearable Sensors and Machine Learning , 2020, IEEE Access.

[43]  Damla Arifoglu,et al.  Activity Recognition and Abnormal Behaviour Detection with Recurrent Neural Networks , 2017, FNC/MobiSPC.

[44]  Chunyan Miao,et al.  Robust human activity recognition using lesser number of wearable sensors , 2017, 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC).

[45]  Petia Radeva,et al.  Human Activity Recognition from Accelerometer Data Using a Wearable Device , 2011, IbPRIA.

[46]  Parisa Rashidi,et al.  Human Activity Recognition Using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey , 2020, IEEE Access.

[47]  Faranak Fotouhi,et al.  Deep learning-based motion activity recognition using smartphone sensors , 2020 .