Robust Human Activity Recognition using smartwatches and smartphones

Abstract Smart user devices are becoming increasingly ubiquitous and useful for detecting the user’s context and his/her current activity. This work analyzes and proposes several techniques to improve the robustness of a Human Activity Recognition (HAR) system that uses accelerometer signals from different smartwatches and smartphones. This analysis reveals some of the challenges associated with both device heterogeneity and the different use of smartwatches compared to smartphones. When using smartwatches to recognize whole body activities, the arm movements introduce additional variability giving rise to a significant degradation in HAR. In this analysis, we describe and evaluate several techniques which successfully address these challenges when using smartwatches and when training and testing with different devices and/or users.

[1]  Davide Anguita,et al.  Energy Efficient Smartphone-Based Activity Recognition Using Fixed-Point Arithmetic , 2013 .

[2]  Paul J. M. Havinga,et al.  Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors , 2016, Sensors.

[3]  Jesús Favela,et al.  Activity-Aware Computing for Healthcare , 2008, IEEE Pervasive Computing.

[4]  Jin Wang,et al.  Recognizing Human Daily Activities From Accelerometer Signal , 2011 .

[5]  Víctor Peláez,et al.  An automatic data mining method to detect abnormal human behaviour using physical activity measurements , 2014, Pervasive Mob. Comput..

[6]  Nigel H. Lovell,et al.  Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring , 2006, IEEE Transactions on Information Technology in Biomedicine.

[7]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[8]  I. Pigeot,et al.  Using Hidden Markov Models to Improve Quantifying Physical Activity in Accelerometer Data – A Simulation Study , 2014, PloS one.

[9]  Zebin Tahmina,et al.  Human activity recognition with inertial sensors using a deep learning approach , 2016 .

[10]  Angelo M. Sabatini,et al.  Machine Learning Methods for Classifying Human Physical Activity from On-Body Accelerometers , 2010, Sensors.

[11]  Faicel Chamroukhi,et al.  An Unsupervised Approach for Automatic Activity Recognition Based on Hidden Markov Model Regression , 2013, IEEE Transactions on Automation Science and Engineering.

[12]  Ivo Stani,et al.  Gesture recognition system for real-time mobile robot control based on inertial sensors and motion strings , 2017 .

[13]  Jeen-Shing Wang,et al.  Using acceleration measurements for activity recognition: An effective learning algorithm for constructing neural classifiers , 2008, Pattern Recognit. Lett..

[14]  Javier Bajo,et al.  Intelligent environment for monitoring Alzheimer patients, agent technology for health care , 2008, Decis. Support Syst..

[15]  Jianfeng Liu,et al.  An accelerometer-based gesture recognition algorithm and its application for 3D interaction , 2010, Comput. Sci. Inf. Syst..

[16]  Fabio Bellifemine,et al.  An agent-based signal processing in-node environment for real-time human activity monitoring based on wireless body sensor networks , 2011, Eng. Appl. Artif. Intell..

[17]  Gary M. Weiss,et al.  Activity recognition using cell phone accelerometers , 2011, SKDD.

[18]  Gerhard Tröster,et al.  The adARC pattern analysis architecture for adaptive human activity recognition systems , 2011, Journal of Ambient Intelligence and Humanized Computing.

[19]  Chris D. Nugent,et al.  A Knowledge-Driven Approach to Activity Recognition in Smart Homes , 2012, IEEE Transactions on Knowledge and Data Engineering.

[20]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[21]  Rubén San-Segundo-Hernández,et al.  Segmenting human activities based on HMMs using smartphone inertial sensors , 2016, Pervasive Mob. Comput..

[22]  Bernt Schiele,et al.  A tutorial on human activity recognition using body-worn inertial sensors , 2014, CSUR.

[23]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[24]  Steve Young,et al.  The HTK book , 1995 .

[25]  Seungjin Choi,et al.  Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[26]  Young-Koo Lee,et al.  Semi-Markov conditional random fields for accelerometer-based activity recognition , 2010, Applied Intelligence.

[27]  Fernando Fernández Martínez,et al.  Feature extraction from smartphone inertial signals for human activity segmentation , 2016, Signal Process..

[28]  Julius Hannink,et al.  Sensor-Based Gait Parameter Extraction With Deep Convolutional Neural Networks , 2016, IEEE Journal of Biomedical and Health Informatics.

[29]  Juan-Luis Gorricho,et al.  Activity Recognition from Accelerometer Data on a Mobile Phone , 2009, IWANN.

[30]  Deborah Estrin,et al.  Using mobile phones to determine transportation modes , 2010, TOSN.

[31]  Guang-Zhong Yang,et al.  A Deep Learning Approach to on-Node Sensor Data Analytics for Mobile or Wearable Devices , 2017, IEEE Journal of Biomedical and Health Informatics.

[32]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[33]  Rama Chellappa,et al.  Machine Recognition of Human Activities: A Survey , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Philippe Lenca,et al.  Unsupervised discovery of activities of daily living characterized by their periodicity and variability , 2015, Eng. Appl. Artif. Intell..

[35]  James A. Landay,et al.  The Mobile Sensing Platform: An Embedded Activity Recognition System , 2008, IEEE Pervasive Computing.