SWL-Adapt: An Unsupervised Domain Adaptation Model with Sample Weight Learning for Cross-User Wearable Human Activity Recognition

In practice, Wearable Human Activity Recognition (WHAR) models usually face performance degradation on the new user due to user variance. Unsupervised domain adaptation (UDA) becomes the natural solution to cross-user WHAR under annotation scarcity. Existing UDA models usually align samples across domains without differentiation, which ignores the difference among samples. In this paper, we propose an unsupervised domain adaptation model with sample weight learning (SWL-Adapt) for cross-user WHAR. SWL-Adapt calculates sample weights according to the classification loss and domain discrimination loss of each sample with a parameterized network. We introduce the meta-optimization based update rule to learn this network end-to-end, which is guided by meta-classification loss on the selected pseudo-labeled target samples. Therefore, this network can fit a weighting function according to the cross-user WHAR task at hand, which is superior to existing sample differentiation rules fixed for special scenarios. Extensive experiments on three public WHAR datasets demonstrate that SWL-Adapt achieves the state-of-the-art performance on the cross-user WHAR task, outperforming the best baseline by an average of 3.1% and 5.3% in accuracy and macro F1 score, respectively.

[1]  K. Mao,et al.  Calibrating Class Weights with Multi-Modal Information for Partial Video Domain Adaptation , 2022, ACM Multimedia.

[2]  H. Bischof,et al.  The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Hui Xue,et al.  Adversarial Spectral Kernel Matching for Unsupervised Time Series Domain Adaptation , 2021, IJCAI.

[4]  Lior Wolf,et al.  Sample Selection for Universal Domain Adaptation , 2021, AAAI.

[5]  Daniela Micucci,et al.  Personalized Models in Human Activity Recognition using Deep Learning , 2021, 2020 25th International Conference on Pattern Recognition (ICPR).

[6]  Avijoy Chakma,et al.  Activity recognition in wearables using adversarial multi-source domain adaptation , 2020 .

[7]  Rubén San-Segundo,et al.  Adaptation and Selection techniques based on Deep Learning for Human Activity Recognition using Inertial Sensors , 2020, Proceedings of 7th International Electronic Conference on Sensors and Applications.

[8]  Sozo Inoue,et al.  Improving activity data collection with on-device personalization using fine-tuning , 2020, UbiComp/ISWC Adjunct.

[9]  Jing Zhao,et al.  XHAR: Deep Domain Adaptation for Human Activity Recognition with Smart Devices , 2020, 2020 17th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON).

[10]  Diane J. Cook,et al.  Multi-Source Deep Domain Adaptation with Weak Supervision for Time-Series Sensor Data , 2020, KDD.

[11]  T. Abdelzaher,et al.  GIobalFusion: A Global Attentional Deep Learning Framework for Multisensor Information Fusion , 2020, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[12]  Junehwa Song,et al.  A Systematic Study of Unsupervised Domain Adaptation for Robust Human-Activity Recognition , 2020, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[13]  Ling Chen,et al.  METIER: A Deep Multi-Task Learning Based Activity and User Recognition Model Using Wearable Sensors , 2020, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[14]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[15]  Artem Molchanov,et al.  Generalized Inner Loop Meta-Learning , 2019, ArXiv.

[16]  Jianmin Wang,et al.  Transferable Curriculum for Weakly-Supervised Domain Adaptation , 2019, AAAI.

[17]  Han Zou,et al.  Consensus Adversarial Domain Adaptation , 2019, AAAI.

[18]  Majid Sarrafzadeh,et al.  Domain Adaptation in Children Activity Recognition , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[19]  Michael I. Jordan,et al.  Universal Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Michael I. Jordan,et al.  Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation , 2019, ICML.

[21]  Sergiu M. Dascalu,et al.  Spatiotemporal recursive hyperspheric classification with an application to dynamic gesture recognition , 2019, Artif. Intell..

[22]  Elnaz Soleimani,et al.  Cross-Subject Transfer Learning in Human Activity Recognition Systems using Generative Adversarial Networks , 2019, Neurocomputing.

[23]  Jianmin Wang,et al.  Learning to Transfer Examples for Partial Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Dianhui Chu,et al.  Empirical Study and Improvement on Deep Transfer Learning for Human Activity Recognition , 2018, Sensors.

[25]  Lina Yao,et al.  Distributionally Robust Semi-Supervised Learning for People-Centric Sensing , 2018, AAAI.

[26]  Jianmin Wang,et al.  Partial Adversarial Domain Adaptation , 2018, ECCV.

[27]  Jing Zhang,et al.  Importance Weighted Adversarial Nets for Partial Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Archan Misra,et al.  Scaling Human Activity Recognition via Deep Learning-based Domain Adaptation , 2018, 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[29]  Hassan Ghasemzadeh,et al.  Personalized Human Activity Recognition Using Convolutional Neural Networks , 2018, AAAI.

[30]  Tatsuya Harada,et al.  Maximum Classifier Discrepancy for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Koichi Shinoda,et al.  User adaptation of convolutional neural network for human activity recognition , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[32]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[33]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Michael I. Jordan,et al.  Deep Transfer Learning with Joint Adaptation Networks , 2016, ICML.

[35]  Pascal Fua,et al.  Beyond Sharing Weights for Deep Domain Adaptation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Timo Sztyler,et al.  On-body localization of wearable devices: An investigation of position-aware activity recognition , 2016, 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[37]  Jin-Hyuk Hong,et al.  Toward Personalized Activity Recognition Systems With a Semipopulation Approach , 2016, IEEE Transactions on Human-Machine Systems.

[38]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[39]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[40]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[41]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[42]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[43]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[44]  Ig-Jae Kim,et al.  Mobile health monitoring system based on activity recognition using accelerometer , 2010, Simul. Model. Pract. Theory.

[45]  Davide Anguita,et al.  Transition-Aware Human Activity Recognition Using Smartphones , 2016, Neurocomputing.

[46]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .