Shallow Convolutional Neural Networks for Human Activity Recognition Using Wearable Sensors

Due to rapid development of sensor technology, human activity recognition (HAR) using wearable inertial sensors has recently become a new research hotspot. Deep learning, especially convolutional neural network (CNN) that can automatically learn intricate activity features have gained a lot of attention in ubiquitous HAR task. Most existing CNNs process sensor input by extracting channel-wise features, and the information from each channel can be separately propagated in a hierarchical way from lower layers to higher layers. As a result, they typically overlook information exchange among channels within the same layer. In this article, we first propose a shallow CNN that considers cross-channel communication in HAR scenario, where all channels in the same layer have a comprehensive interaction to capture more discriminative features of sensor input. One channel can communicate with all other channels by graph neural network to remove redundant information accumulated among channels, which is more beneficial for deploying lightweight deep models. Extensive experiments are conducted on multiple benchmark HAR datasets, namely UCI-HAR, OPPORTUNITY, PAMAP2 and UniMib-SHAR, which indicates that the proposed method enables shallower CNNs to aggregate more useful information, and surpasses baseline deep networks and other competitive methods. The inference speed is evaluated via deploying the HAR systems on an embedded system.

[1]  Zhaozheng Yin,et al.  Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks , 2015, ACM Multimedia.

[2]  Thomas Plötz,et al.  Ensembles of Deep LSTM Learners for Activity Recognition using Wearables , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[3]  Mounib Khanafer,et al.  Applied AI in instrumentation and measurement: The deep learning revolution , 2020, IEEE Instrumentation & Measurement Magazine.

[4]  Jie Bao,et al.  Reliability index and Asian barrier option pricing formulas of the uncertain fractional first-hitting time model with Caputo type , 2020 .

[5]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[6]  Wenzhong Li,et al.  AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition , 2019, IJCAI.

[7]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.

[8]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[10]  Lei Zhang,et al.  Layer-Wise Training Convolutional Neural Networks With Smaller Filters for Human Activity Recognition Using Wearable Sensors , 2020, IEEE Sensors Journal.

[11]  Ming Zeng,et al.  Understanding and improving recurrent networks for human activity recognition by continuous attention , 2018, UbiComp.

[12]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[13]  Cheng-Zhong Xu,et al.  Dynamic Channel Pruning: Feature Boosting and Suppression , 2018, ICLR.

[14]  Zhiqiang Shen,et al.  Learning Efficient Convolutional Networks through Network Slimming , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Lei Zhang,et al.  The Layer-Wise Training Convolutional Neural Networks Using Local Loss for Sensor-Based Human Activity Recognition , 2020, IEEE Sensors Journal.

[16]  Jie Ding,et al.  Smartphone Sensor-Based Human Activity Recognition Using Feature Fusion and Maximum Full a Posteriori , 2020, IEEE Transactions on Instrumentation and Measurement.

[17]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[18]  Kaiming He,et al.  Group Normalization , 2018, ECCV.

[19]  Yi Li,et al.  Deformable Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Jinwoo Shin,et al.  Training CNNs with Selective Allocation of Channels , 2019, ICML.

[21]  Sung-Bae Cho,et al.  Human activity recognition with smartphone sensors using deep learning neural networks , 2016, Expert Syst. Appl..

[22]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Diane J. Cook,et al.  Keeping the Resident in the Loop: Adapting the Smart Home to the User , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[24]  Daniela Micucci,et al.  UniMiB SHAR: a new dataset for human activity recognition using acceleration data from smartphones , 2016, ArXiv.

[25]  Chuang Gan,et al.  Cross-channel Communication Networks , 2019, NeurIPS.

[26]  Bernt Schiele,et al.  A tutorial on human activity recognition using body-worn inertial sensors , 2014, CSUR.

[27]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[28]  Kimiaki Shirahama,et al.  Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors , 2018, Sensors.

[29]  Yiquan Du,et al.  Path Planning of Coastal Ships Based on Optimized DQN Reward Function , 2021, Journal of Marine Science and Engineering.

[30]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Xiangfeng Yang,et al.  RELIABILITY INDEX AND OPTION PRICING FORMULAS OF THE FIRST-HITTING TIME MODEL BASED ON THE UNCERTAIN FRACTIONAL-ORDER DIFFERENTIAL EQUATION WITH CAPUTO TYPE , 2020 .

[32]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[33]  Lei Zhang,et al.  Attention-Based Convolutional Neural Network for Weakly Labeled Human Activities’ Recognition With Wearable Sensors , 2019, IEEE Sensors Journal.

[34]  Mi Zhang,et al.  Human Daily Activity Recognition With Sparse Representation Using Wearable Sensors , 2013, IEEE Journal of Biomedical and Health Informatics.

[35]  Davide Anguita,et al.  Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine , 2012, IWAAL.

[36]  Tat-Seng Chua,et al.  SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Didier Stricker,et al.  Introducing a New Benchmarked Dataset for Activity Monitoring , 2012, 2012 16th International Symposium on Wearable Computers.

[38]  Sengul Dogan,et al.  An Automated Daily Sports Activities and Gender Recognition Method Based on Novel Multikernel Local Diamond Pattern Using Sensor Signals , 2020, IEEE Transactions on Instrumentation and Measurement.

[39]  Xiangyu Zhang,et al.  Channel Pruning for Accelerating Very Deep Neural Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[40]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[41]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[42]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.