Deep-Learning-Enhanced Human Activity Recognition for Internet of Healthcare Things

Along with the advancement of several emerging computing paradigms and technologies, such as cloud computing, mobile computing, artificial intelligence, and big data, Internet of Things (IoT) technologies have been applied in a variety of fields. In particular, the Internet of Healthcare Things (IoHT) is becoming increasingly important in human activity recognition (HAR) due to the rapid development of wearable and mobile devices. In this article, we focus on the deep-learning-enhanced HAR in IoHT environments. A semisupervised deep learning framework is designed and built for more accurate HAR, which efficiently uses and analyzes the weakly labeled sensor data to train the classifier learning model. To better solve the problem of the inadequately labeled sample, an intelligent autolabeling scheme based on deep $Q$ -network (DQN) is developed with a newly designed distance-based reward rule which can improve the learning efficiency in IoT environments. A multisensor based data fusion mechanism is then developed to seamlessly integrate the on-body sensor data, context sensor data, and personal profile data together, and a long short-term memory (LSTM)-based classification method is proposed to identify fine-grained patterns according to the high-level features contextually extracted from the sequential motion data. Finally, experiments and evaluations are conducted to demonstrate the usefulness and effectiveness of the proposed method using real-world data.

[1]  Ling Pei,et al.  Weakly Supervised Human Activity Recognition From Wearable Sensors by Recurrent Attention Learning , 2019, IEEE Sensors Journal.

[2]  Paul Lukowicz,et al.  Wearable Sensing to Annotate Meeting Recordings , 2002, SEMWEB.

[3]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.

[4]  Yang Wang,et al.  Human Action Recognition by Semilatent Topic Models , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Tapio Seppänen,et al.  Recognizing human motion with multiple acceleration sensors , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[6]  Paul Lukowicz,et al.  Recognizing Workshop Activity Using Body Worn Microphones and Accelerometers , 2004, Pervasive.

[7]  Daniela Micucci,et al.  UniMiB SHAR: a new dataset for human activity recognition using acceleration data from smartphones , 2016, ArXiv.

[8]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[9]  Henry A. Kautz,et al.  Inferring High-Level Behavior from Low-Level Sensors , 2003, UbiComp.

[10]  Michael L. Littman,et al.  Activity Recognition from Accelerometer Data , 2005, AAAI.

[11]  Kenji Mase,et al.  Activity and Location Recognition Using Wearable Sensors , 2002, IEEE Pervasive Comput..

[12]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[13]  Bo Zhou,et al.  A Hybrid Hierarchical Framework for Gym Physical Activity Recognition and Measurement Using Wearable Sensors , 2019, IEEE Internet of Things Journal.

[14]  Urbashi Mitra,et al.  Multimodal Physical Activity Recognition by Fusing Temporal and Cepstral Information , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  Wei Lu,et al.  Wearable Computing for Internet of Things: A Discriminant Approach for Human Activity Recognition , 2019, IEEE Internet of Things Journal.

[16]  Yajie Miao,et al.  EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding , 2015, 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).

[17]  Sajal K. Das,et al.  HuMAn: Complex Activity Recognition with Multi-Modal Multi-Positional Body Sensing , 2019, IEEE Transactions on Mobile Computing.

[18]  Cheng Zhang,et al.  Blockchain Empowered Arbitrable Data Auditing Scheme for Network Storage as a Service , 2020, IEEE Transactions on Services Computing.

[19]  Nasser Kehtarnavaz,et al.  A survey of depth and inertial sensor fusion for human action recognition , 2015, Multimedia Tools and Applications.

[20]  Teh Ying Wah,et al.  Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions , 2019, Inf. Fusion.

[21]  Laurence T. Yang,et al.  An Efficient Deep Learning Model to Predict Cloud Workload for Industry Informatics , 2018, IEEE Transactions on Industrial Informatics.

[22]  Laurence T. Yang,et al.  A Big Data-as-a-Service Framework: State-of-the-Art and Perspectives , 2018, IEEE Transactions on Big Data.

[23]  Jiangchuan Liu,et al.  On Spatial Diversity in WiFi-Based Human Activity Recognition: A Deep Learning-Based Approach , 2019, IEEE Internet of Things Journal.

[24]  Laurence T. Yang,et al.  Big Data Real-Time Processing Based on Storm , 2013, 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications.

[25]  Timo Sztyler,et al.  Position-aware activity recognition with wearable devices , 2017, Pervasive Mob. Comput..

[26]  Bernt Schiele,et al.  Weakly Supervised Recognition of Daily Life Activities with Wearable Sensors , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Geyong Min,et al.  Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT , 2020, IEEE Internet of Things Journal.

[28]  Yang Xu,et al.  A Blockchain-Based Nonrepudiation Network Computing Service Scheme for Industrial IoT , 2019, IEEE Transactions on Industrial Informatics.

[29]  Zhenghua Chen,et al.  A Novel Semisupervised Deep Learning Method for Human Activity Recognition , 2019, IEEE Transactions on Industrial Informatics.

[30]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[31]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[32]  Tarek El-Ghazawi,et al.  Towards Accurate Prediction for High-Dimensional and Highly-Variable Cloud Workloads with Deep Learning , 2020, IEEE Transactions on Parallel and Distributed Systems.

[33]  Rama Chellappa,et al.  Machine Recognition of Human Activities: A Survey , 2008, IEEE Transactions on Circuits and Systems for Video Technology.