ActivityGAN: generative adversarial networks for data augmentation in sensor-based human activity recognition

Label Scarcity and Data Augmentation have long been challenging problems in the research of Human-oriented Artificial Intelligence. Following the trends of Deep Learning, Human Activity Recognition (HAR) tasks have been significantly optimized in the recent decade with handful of industrial applications on health evaluation and security monitoring. Nevertheless, data acquisition on human activities has been increasingly problematic considering the limited sensor resources and high cost of investment on labor of human volunteers. In this paper, we propose a pioneering unified architecture of convolutional generative adversarial networks, namely ActivityGAN, to effectively generate sensor-based data simulating human physical activities. This architecture comprises of a generation model which is a stack of one-dimensional convolution (1D-convolution) and transposed convolution (1D-transposed convolution) layers, and a discrimination model which employs two-dimensional convolution networks (2D-convolution) with reshaped input of time series. We train the proposed architecture on a collection of activity data and evaluate the generator's output, namely synthetic data, with three approaches of visualization. We then assess the usability of synthetic data by evaluating the test accuracy of models trained with mixed real and synthetic data or with synthetic data that substitutes real data. The study's results show that our proposed architecture is able to generate sufficient synthetic data which are distinguishable by visualization techniques and trainable for HAR machine learning models.

[1]  Ying Wah Teh,et al.  Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges , 2018, Expert Syst. Appl..

[2]  Nobuhiko Nishio,et al.  HASC2011corpus: towards the common ground of human activity recognition , 2011, UbiComp '11.

[3]  Tanveer F. Syeda-Mahmood,et al.  Semi-supervised learning with generative adversarial networks for chest X-ray classification with ability of data domain adaptation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[4]  Andreas W. Kempa-Liehr,et al.  Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh - A Python package) , 2018, Neurocomputing.

[5]  Yun Lin,et al.  Semi-Supervised Learning with Generative Adversarial Networks on Digital Signal Modulation Classification , 2018 .

[6]  Jost Tobias Springenberg,et al.  Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks , 2015, ICLR.

[7]  Cesare Furlanello,et al.  Deep learning for automatic stereotypical motor movement detection using wearable sensors in autism spectrum disorders , 2017, Signal Process..

[8]  Andrey Ignatov,et al.  Real-time human activity recognition from accelerometer data using Convolutional Neural Networks , 2018, Appl. Soft Comput..

[9]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[10]  Lior Wolf,et al.  Language Generation with Recurrent Generative Adversarial Networks without Pre-training , 2017, ArXiv.

[11]  Zhaozheng Yin,et al.  Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks , 2015, ACM Multimedia.

[12]  Augustus Odena,et al.  Semi-Supervised Learning with Generative Adversarial Networks , 2016, ArXiv.

[13]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[14]  Quan Liu,et al.  Electrocardiogram generation with a bidirectional LSTM-CNN generative adversarial network , 2019, Scientific Reports.

[15]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[16]  Jin Wang,et al.  Semi-supervised Learning with Generative Adversarial Networks on Digital Signal Mod-ulation Classification , 2018 .

[17]  Dmitrijs Balabka,et al.  Semi-supervised learning for human activity recognition using adversarial autoencoders , 2019, UbiComp/ISWC Adjunct.

[18]  Lu Su,et al.  SenseGAN , 2018 .

[19]  Duc A. Tran,et al.  The 11th International Conference on Mobile Systems and Pervasive Computing (MobiSPC-2014) A Study on Human Activity Recognition Using Accelerometer Data from Smartphones , 2014 .

[20]  Antonio Bonafonte,et al.  SEGAN: Speech Enhancement Generative Adversarial Network , 2017, INTERSPEECH.

[21]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[22]  Mani B. Srivastava,et al.  SenseGen: A deep learning architecture for synthetic sensor data generation , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[23]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[24]  Héctor Pomares,et al.  Window Size Impact in Human Activity Recognition , 2014, Sensors.

[25]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.

[27]  David Berthelot,et al.  BEGAN: Boundary Equilibrium Generative Adversarial Networks , 2017, ArXiv.

[28]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[29]  Xiaoming Liu,et al.  Disentangled Representation Learning GAN for Pose-Invariant Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Yiqiang Chen,et al.  SensoryGANs: An Effective Generative Adversarial Framework for Sensor-based Human Activity Recognition , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).