SenseGAN

Recent proliferation of Internet of Things (IoT) devices with enhanced computing and sensing capabilities has revolutionized our everyday life. The massive data from these ubiquitous devices motivate the creation of intelligent IoT systems that can collectively learn. However, labelling data for learning purposes is extremely time-consuming, which greatly hinders deployment. In this paper, we describe a semi-supervised deep learning framework, called SenseGAN, that can leverage abundant unlabelled sensing data thereby minimizing the need for labelling effort. SenseGAN jointly trains three components with an adversarial game: (i) a classifier for predicting labels of input sensing data; (ii) a generator for generating sensing data samples based on the input labels; and (iii) a discriminator for differentiating the joint data/label distribution between real samples and partially generated samples from either the classifier or the generator. The classifier and the generator try to generate fake data/labels that can fool the discriminator. The adversarial game among the three components can mutually boost their performance, which helps the classifier learn to predict correct labels with unlabelled data in return. SenseGAN can effectively handle multimodal sensing inputs and easily stabilize the adversarial training process, which helps improve the performance of the classifier. Experiments on three IoT applications demonstrate the substantial improvements in accuracy and F1 score under SenseGAN, compared with supervised counterparts trained only on the labelled portion of the data, as well as other supervised and semi-supervised baselines. For these three applications, SenseGAN requires only 10% of the originally labelled data, to attain nearly the same accuracy as a deep learning classifier trained on the fully labelled dataset.

[1]  Ian Oakley,et al.  Indoor-ALPS: an adaptive indoor location prediction system , 2014, UbiComp.

[2]  Sunny Consolvo,et al.  ShutEye: encouraging awareness of healthy sleep recommendations with a mobile, peripheral display , 2012, CHI.

[3]  Liyuan Liu,et al.  TrioVecEvent: Embedding-Based Online Local Event Detection in Geo-Tagged Tweet Streams , 2017, KDD.

[4]  Nicholas D. Lane,et al.  DeepEar: robust smartphone audio sensing in unconstrained acoustic environments using deep learning , 2015, UbiComp.

[5]  A. J. Bernheim Brush,et al.  Health chair: implicitly sensing heart and respiratory rate , 2014, UbiComp.

[6]  Lars Vedel Kessing,et al.  Behavioral activities collected through smartphones and the association with illness activity in bipolar disorder , 2016, International journal of methods in psychiatric research.

[7]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[8]  Peter L. Bartlett,et al.  Variance Reduction Techniques for Gradient Estimates in Reinforcement Learning , 2001, J. Mach. Learn. Res..

[9]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[10]  James A. Landay,et al.  The Mobile Sensing Platform: An Embedded Activity Recognition System , 2008, IEEE Pervasive Computing.

[11]  Jörg Müller,et al.  Eye tracking for public displays in the wild , 2015, Personal and Ubiquitous Computing.

[12]  Diogo R. Ferreira,et al.  Preprocessing techniques for context recognition from accelerometer data , 2010, Personal and Ubiquitous Computing.

[13]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[14]  Peter Andras,et al.  On preserving statistical characteristics of accelerometry data using their empirical cumulative distribution , 2013, ISWC '13.

[15]  Hui-Shyong Yeo,et al.  WatchMI: pressure touch, twist and pan gesture input on unmodified smartwatches , 2016, MobileHCI.

[16]  Paul Lukowicz,et al.  Monitoring crowd condition in public spaces by tracking mobile consumer devices with wifi interface , 2016, UbiComp Adjunct.

[17]  Ig-Jae Kim,et al.  Indoor location sensing using geo-magnetism , 2011, MobiSys '11.

[18]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[19]  Shaowen Wang,et al.  Regions, Periods, Activities: Uncovering Urban Dynamics via Cross-Modal Representation Learning , 2017, WWW.

[20]  Tarek F. Abdelzaher,et al.  DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework , 2017, SenSys.

[21]  Shwetak N. Patel,et al.  HemaApp: Noninvasive Blood Screening of Hemoglobin Using Smartphone Cameras , 2017, GETMBL.

[22]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[23]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[24]  Aaron C. Courville,et al.  Adversarially Learned Inference , 2016, ICLR.

[25]  Charu C. Aggarwal,et al.  Recursive Ground Truth Estimator for Social Data Streams , 2016, 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[26]  Mahesh K. Marina,et al.  Towards multimodal deep learning for activity recognition on mobile devices , 2016, UbiComp Adjunct.

[27]  Frank Bentley,et al.  Reducing the Stress of Coordination: Sharing Travel Time Information Between Contacts on Mobile Phones , 2015, CHI.

[28]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[29]  Francisco Herrera,et al.  Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study , 2015, Knowledge and Information Systems.

[30]  Yann Dauphin,et al.  Language Modeling with Gated Convolutional Networks , 2016, ICML.

[31]  Rob Fergus,et al.  Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks , 2016, ArXiv.

[32]  Shaohan Hu,et al.  On Source Dependency Models for Reliable Social Sensing: Algorithms and Fundamental Error Bounds , 2016, 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS).

[33]  Nicholas D. Lane,et al.  Squeezing Deep Learning into Mobile and Embedded Devices , 2017, IEEE Pervasive Computing.

[34]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[35]  Shaohan Hu,et al.  Deep Learning for the Internet of Things , 2018, Computer.

[36]  Shen Li,et al.  RDeepSense: Reliable Deep Mobile Computing Models with Uncertainty Estimations , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[37]  Eric C. Larson,et al.  Design and learnability of vortex whistles for managing chronic lung function via smartphones , 2016, UbiComp.

[38]  Jun Zhu,et al.  Triple Generative Adversarial Nets , 2017, NIPS.

[39]  Lina Yao,et al.  Learning from less for better: semi-supervised activity recognition via shared structure discovery , 2016, UbiComp.

[40]  Shaohan Hu,et al.  DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing , 2016, WWW.

[41]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[42]  Shwetak N. Patel,et al.  AirLink: sharing files between multiple devices using in-air gestures , 2014, UbiComp.

[43]  Xin Pan,et al.  ARIEL: automatic wi-fi based room fingerprinting for indoor localization , 2012, UbiComp.

[44]  Augustus Odena,et al.  Semi-Supervised Learning with Generative Adversarial Networks , 2016, ArXiv.

[45]  Ramviyas Parasuraman,et al.  Wisture: RNN-based Learning of Wireless Signals for Gesture Recognition in Unmodified Smartphones , 2017, ArXiv.

[46]  Parameswaran Ramanathan,et al.  Leveraging directional antenna capabilities for fine-grained gesture recognition , 2014, UbiComp.

[47]  Yvonne Rogers,et al.  Best intentions: health monitoring technology and children , 2012, CHI.

[48]  Andrea Mannini,et al.  Activity recognition using a single accelerometer placed at the wrist or ankle. , 2013, Medicine and science in sports and exercise.

[49]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[50]  Mikkel Baun Kjærgaard,et al.  Smart Devices are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition , 2015, SenSys.

[51]  C. Villani Optimal Transport: Old and New , 2008 .

[52]  Arjan Kuijper,et al.  Platypus: Indoor Localization and Identification through Sensing of Electric Potential Changes in Human Bodies , 2016, MobiSys.