IoT Wearable Sensor and Deep Learning: An Integrated Approach for Personalized Human Activity Recognition in a Smart Home Environment

Human activity recognition (HAR) is currently recognized as a key element of a more general framework designed to perform continuous monitoring of human behaviors in the area of ambient assisted living (AAL), well-being management, medical diagnosis, elderly care, rehabilitation, entertainment, and surveillance in smart home environments. In this paper, an innovative HAR system, exploiting the potential of wearable devices integrated with the skills of deep learning techniques, is presented with the aim of recognizing the most common daily activities of a person at home. The designed wearable sensor embeds an inertial measurement unit (IMU) and a Wi-Fi section to send data on a cloud service and to allow direct connection to the Internet through a common home router so that the user themselves could manage the installation procedure. The sensor is coupled to a convolutional neural network (CNN) network designed to make inferences with the minimum possible resources to keep open the way of its implementation on low-cost or embedded devices. The system is conceived for daily activity monitor and nine different activities can be highlighted with an accuracy of 97%.

[1]  Sung-Bae Cho,et al.  Evaluation of Deep Convolutional Neural Network Architectures for Human Activity Recognition with Smartphone Sensors , 2015 .

[2]  Guang-Zhong Yang,et al.  Deep learning for human activity recognition: A resource efficient implementation on low-power devices , 2016, 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[3]  Davide Anguita,et al.  A Public Domain Dataset for Human Activity Recognition using Smartphones , 2013, ESANN.

[4]  Rama Chellappa,et al.  Machine Recognition of Human Activities: A Survey , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Athanasios Anastasiou,et al.  A New Method for Profile Generation in an Internet of Things Environment: An Application in Ambient-Assisted Living , 2015, IEEE Internet of Things Journal.

[7]  Liezel Cilliers,et al.  Mobile and wearable technologies in healthcare for the ageing population , 2018, Comput. Methods Programs Biomed..

[8]  Monica Mordonini,et al.  Food Image Recognition Using Very Deep Convolutional Networks , 2016, MADiMa @ ACM Multimedia.

[9]  Fei Wang,et al.  Deep learning for healthcare: review, opportunities and challenges , 2018, Briefings Bioinform..

[10]  Stefano Cagnoni,et al.  Sensor Fusion-Oriented Fall Detection for Assistive Technologies Applications , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[11]  Yuwen Chen,et al.  LSTM Networks for Mobile Human Activity Recognition , 2016 .

[12]  Sung-Bae Cho,et al.  Human activity recognition with smartphone sensors using deep learning neural networks , 2016, Expert Syst. Appl..

[13]  Bo Ding,et al.  Unsupervised Feature Learning for Human Activity Recognition Using Smartphone Sensors , 2014, MIKE.

[14]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[15]  Dong Yu,et al.  Recent progresses in deep learning based acoustic models , 2017, IEEE/CAA Journal of Automatica Sinica.

[16]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[17]  Valentina Bianchi,et al.  RSSI-Based Indoor Localization and Identification for ZigBee Wireless Sensor Networks in Smart Homes , 2019, IEEE Transactions on Instrumentation and Measurement.

[18]  Igor Bisio,et al.  Enabling IoT for In-Home Rehabilitation: Accelerometer Signals Classification Methods for Activity and Movement Recognition , 2017, IEEE Internet of Things Journal.

[19]  Stefano Cagnoni,et al.  Multi-sensor system for detection and classification of human activities , 2011, Journal of Ambient Intelligence and Humanized Computing.

[20]  Valentina Bianchi,et al.  An unobtrusive Wi-Fi system for human monitoring , 2017, 2017 IEEE 7th International Conference on Consumer Electronics - Berlin (ICCE-Berlin).

[21]  Jun Hu,et al.  Activity recognition based on inertial sensors for Ambient Assisted Living , 2016, 2016 19th International Conference on Information Fusion (FUSION).

[22]  Zhelong Wang,et al.  Using Body-Worn Sensors for Preliminary Rehabilitation Assessment in Stroke Victims With Gait Impairment , 2018, IEEE Access.

[23]  Claudia Eckert,et al.  Neural Network-Based User-Independent Physical Activity Recognition for Mobile Devices , 2015, IDEAL.

[24]  Jianbo Yang,et al.  Deep Learning for Human Activity Recognition , 2020 .

[25]  Wenyu Zhang,et al.  Multivariate Time Series Classification Using Dynamic Time Warping Template Selection for Human Activity Recognition , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[26]  Valentina Bianchi,et al.  CARDEAGate: Low-cost, ZigBee-based localization and identification for AAL purposes , 2015, 2015 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) Proceedings.

[27]  Sung-Bae Cho,et al.  Recognizing human activities from smartphone sensors using hierarchical continuous hidden Markov models , 2017, Int. J. Distributed Sens. Networks.

[28]  Valentina Bianchi,et al.  A BCI Platform Supporting AAL Applications , 2014, HCI.

[29]  Valentina Bianchi,et al.  An IoT Approach for an AAL Wi-Fi-Based Monitoring System , 2017, IEEE Transactions on Instrumentation and Measurement.

[30]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[31]  Subhas Chandra Mukhopadhyay,et al.  Wearable Sensors for Human Activity Monitoring: A Review , 2015, IEEE Sensors Journal.

[32]  Thomas Plötz,et al.  Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables , 2016, IJCAI.

[33]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[34]  Yu Zhao,et al.  Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors , 2017, Mathematical Problems in Engineering.

[35]  Thomas Plötz,et al.  Optimising sampling rates for accelerometer-based human activity recognition , 2016, Pattern Recognit. Lett..

[36]  Noel E. O'Connor,et al.  Toward Automatic Activity Classification and Movement Assessment During a Sports Training Session , 2015, IEEE Internet of Things Journal.

[37]  David Mendes,et al.  Deep Learning and IoT to Assist Multimorbidity Home Based Healthcare , 2017 .

[38]  Valentina Bianchi,et al.  MuSA: Wearable Multi Sensor Assistant for Human Activity Recognition and Indoor Localization , 2015 .

[39]  Krisjanis Nesenbergs,et al.  Inertial sensors and muscle electrical signals in human-computer interaction , 2017, 2017 6th International Conference on Information and Communication Technology and Accessibility (ICTA).

[40]  Thinagaran Perumal,et al.  Monitoring daily fitness activity using accelerometer sensor fusion , 2017, 2017 IEEE International Symposium on Consumer Electronics (ISCE).

[41]  Xiaoli Li,et al.  Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition , 2015, IJCAI.

[42]  Zhaozheng Yin,et al.  Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks , 2015, ACM Multimedia.

[43]  Yann Le Cun,et al.  A Theoretical Framework for Back-Propagation , 1988 .

[44]  Valentina Bianchi,et al.  A Plug and Play IoT Wi-Fi Smart Home System for Human Monitoring , 2018, Electronics.

[45]  Yuqing Chen,et al.  A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[46]  Fei Xie,et al.  A Survey on Smart Wearables in the Application of Fitness , 2017, 2017 IEEE 15th Intl Conf on Dependable, Autonomic and Secure Computing, 15th Intl Conf on Pervasive Intelligence and Computing, 3rd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech).

[47]  Jae-Young Pyun,et al.  Deep Recurrent Neural Networks for Human Activity Recognition , 2017, Sensors.

[48]  Andrea Szwajcer,et al.  A systematic review of the association between sedentary behaviors with frailty , 2018, Experimental Gerontology.

[49]  Valentina Bianchi,et al.  The HELICOPTER project: Wireless sensor network for multi-user behavioral monitoring , 2017, 2017 International Conference on Engineering, Technology and Innovation (ICE/ITMC).

[50]  Sung-Bae Cho,et al.  Human activity recognition using smartphone sensors with two-stage continuous hidden Markov models , 2014, 2014 10th International Conference on Natural Computation (ICNC).

[51]  Y. T. Zhou,et al.  Computation of optical flow using a neural network , 1988, IEEE 1988 International Conference on Neural Networks.

[52]  Takeshi Nishida,et al.  Deep recurrent neural network for mobile human activity recognition with high throughput , 2017, Artificial Life and Robotics.

[53]  Michele Rossi,et al.  Boosting the Battery Life of Wearables for Health Monitoring Through the Compression of Biosignals , 2017, IEEE Internet of Things Journal.