Digging deeper: towards a better understanding of transfer learning for human activity recognition

Transfer Learning is becoming increasingly important to the Human Activity Recognition community, as it enables algorithms to reuse what has already been learned from models. It promises shortened training times and increased classification results for new datasets and activity classes. However, the question of what exactly is transferred is not dealt with in detail in many of the recent publications, and it is furthermore often difficult to reproduce the presented results. Therefore we would like to contribute with this paper to the understanding of transfer learning for sensor-based human activity recognition. In our experiment use weight transfer to transfer models between two datasets, as well as between sensors from the same dataset. As source- and target- datasets PAMAP2 and Skoda Mini Checkpoint are used. The utilized network architecture is based on a DeepConvLSTM. The result of our investigation shows that transfer learning has to be considered in a very differentiated way, since the desired positive effects by applying the method depend very much on the data and also on the architecture used.

[1]  Didier Stricker,et al.  Introducing a New Benchmarked Dataset for Activity Monitoring , 2012, 2012 16th International Symposium on Wearable Computers.

[2]  Qiang Yang,et al.  Cross-domain activity recognition via transfer learning , 2011, Pervasive Mob. Comput..

[3]  Martin Gjoreski,et al.  Cross-dataset deep transfer learning for activity recognition , 2019, UbiComp/ISWC Adjunct.

[4]  Kristof Van Laerhoven,et al.  Using Wrist-Worn Activity Recognition for Basketball Game Analysis , 2018, iWOAR.

[5]  Yiqiang Chen,et al.  OCEAN: a new opportunistic computing model for wearable activity recognition , 2016, UbiComp Adjunct.

[6]  Florian Huber,et al.  Mcfly: Automated deep learning on time series , 2020, SoftwareX.

[7]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[8]  Daniel Roggen,et al.  Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations , 2016, SEMWEB.

[9]  Matjaz Gams,et al.  Context-based ensemble method for human energy expenditure estimation , 2015, Appl. Soft Comput..

[10]  Niall Twomey,et al.  Active transfer learning for activity recognition , 2016, ESANN.

[11]  Richard Walker,et al.  PD Disease State Assessment in Naturalistic Environments Using Deep Learning , 2015, AAAI.

[12]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[13]  Thomas Plötz,et al.  Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables , 2016, IJCAI.

[14]  Bertram Taetz,et al.  IMU-to-Segment Assignment and Orientation Alignment for the Lower Body Using Deep Learning , 2018, Sensors.

[15]  Jani Bizjak,et al.  Classical and deep learning methods for recognizing human activities and modes of transportation with smartphone sensors , 2020, Inf. Fusion.

[16]  Luca Benini,et al.  Network-Level Power-Performance Trade-Off in Wearable Activity Recognition: A Dynamic Sensor Selection Approach , 2012, TECS.

[17]  Sajal K. Das,et al.  A-Wristocracy: Deep learning on wrist-worn sensing for recognition of user complex activities , 2015, 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[18]  Netzahualcóyotl Hernández,et al.  Literature Review on Transfer Learning for Human Activity Recognition Using Mobile and Wearable Devices with Environmental Technology , 2020, SN Computer Science.

[19]  Paul Lukowicz,et al.  Sensor Placement Variations in Wearable Activity Recognition , 2014, IEEE Pervasive Computing.

[20]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[21]  Vincent S. Tseng,et al.  Transfer Learning on High Variety Domains for Activity Recognition , 2015, ASE BD&SI.

[22]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[23]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[24]  Kimiaki Shirahama,et al.  Deep Transfer Learning for Time Series Data Based on Sensor Modality Classification , 2020, Sensors.