Robust Human Activity Recognition based on Deep Metric Learning

In the domain of Activity Recognition, the proliferation of low-cost and sensor-enabled personal devices has led to significant heterogeneity in the data generated by users. Traditional approaches to this problem have previously relied on handcrafted features and template-matching methods, which have limited flexibility and performance with high variability. In this work we investigate the use of Deep Metric Learning in the domain of activity recognition. We use a deep Triplet Network to generate fixed-length descriptors from activity samples for purposes of classification. We carry out evaluation of our proposed method on five datasets from different sources with differing activities. We obtain classification accuracies of up to 96% in self-testing scenarios and up to 91% accuracy in cross-dataset testing without retraining. We also show that our method performs similarly to traditional Convolutional Neural Networks. The obtained results indicate the promise of this approach.

[1]  Anna M. Bianchi,et al.  User-Independent Recognition of Sports Activities From a Single Wrist-Worn Accelerometer: A Template-Matching-Based Approach , 2016, IEEE Transactions on Biomedical Engineering.

[2]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[3]  Samsu Sempena,et al.  Human action recognition using Dynamic Time Warping , 2011, Proceedings of the 2011 International Conference on Electrical Engineering and Informatics.

[4]  Billur Barshan,et al.  Investigating Inter-Subject and Inter-Activity Variations in Activity Recognition Using Wearable Motion Sensors , 2016, Comput. J..

[5]  Mikkel Baun Kjærgaard,et al.  Smart Devices are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition , 2015, SenSys.

[6]  Walid Gomaa,et al.  An LSTM-based Descriptor for Human Activities Recognition using IMU Sensors. , 2018 .

[7]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[9]  Billur Barshan,et al.  Comparative study on classifying human activities with miniature inertial and magnetic sensors , 2010, Pattern Recognit..

[10]  Homay Danaei Mehr,et al.  Resident activity recognition in smart homes by using artificial neural networks , 2016, 2016 4th International Istanbul Smart Grid Congress and Fair (ICSG).

[11]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[12]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[13]  Héctor Pomares,et al.  A benchmark dataset to evaluate sensor displacement in activity recognition , 2012, UbiComp.

[14]  Lucas Beyer,et al.  In Defense of the Triplet Loss for Person Re-Identification , 2017, ArXiv.

[15]  Lei Liu,et al.  Human Daily Activity Recognition for Healthcare Using Wearable and Visual Sensing Data , 2016, 2016 IEEE International Conference on Healthcare Informatics (ICHI).

[16]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[17]  Rajesh Kumar Dubey,et al.  Automatic speech recognition for connected words using DTW/HMM for English/ Hindi languages , 2015, 2015 Communication, Control and Intelligent Systems (CCIS).

[18]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[19]  Sajal K. Das,et al.  Multimodal Wearable Sensing for Fine-Grained Activity Recognition in Healthcare , 2015, IEEE Internet Computing.

[20]  Zhengping Che.  DECADE : A Deep Metric Learning Model for Multivariate Time Series , 2017 .

[21]  Shenghuo Zhu,et al.  Large Scale Strongly Supervised Ensemble Metric Learning, with Applications to Face Verification and Retrieval , 2012, ArXiv.

[22]  Seungjin Choi,et al.  Multi-modal Convolutional Neural Networks for Activity Recognition , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[23]  Sara Ashry Mohammed,et al.  ADL Classification Based on Autocorrelation Function of Inertial Signals , 2017, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[24]  Stephen Tyree,et al.  Non-linear Metric Learning , 2012, NIPS.

[25]  Wenyu Zhang,et al.  Multivariate Time Series Classification Using Dynamic Time Warping Template Selection for Human Activity Recognition , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[26]  Héctor Pomares,et al.  Dealing with the Effects of Sensor Displacement in Wearable Activity Recognition , 2014, Sensors.

[27]  Jiwen Lu,et al.  Discriminative Deep Metric Learning for Face Verification in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[29]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[30]  Davide Anguita,et al.  Transition-Aware Human Activity Recognition Using Smartphones , 2016, Neurocomputing.

[31]  Kimiaki Shirahama,et al.  Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors , 2018, Sensors.

[32]  Haomin Cui,et al.  A novel multi-metric scheme using dynamic time warping for similarity video clip search , 2013, 2013 IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC 2013).

[33]  Enamul Hoque,et al.  AALO: Activity recognition in smart homes using Active Learning in the presence of Overlapped activities , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[34]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).