Time Series Segmentation Using Neural Networks with Cross-Domain Transfer Learning

Searching for characteristic patterns in time series is a topic addressed for decades by the research community. Conventional subsequence matching techniques usually rely on the definition of a target template pattern and a searching method for detecting similar patterns. However, the intrinsic variability of time series introduces changes in patterns, either morphologically and temporally, making such techniques not as accurate as desired. Intending to improve segmentation performances, in this paper, we proposed a Mask-based Neural Network (NN) which is capable of extracting desired patterns of interest from long time series, without using any predefined template. The proposed NN has been validated, alongside a subsequence matching algorithm, in two datasets: clinical (electrocardiogram) and human activity (inertial sensors). Moreover, the reduced dimension of the data in the latter dataset led to the application of transfer learning and data augmentation techniques to reach model convergence. The results have shown the proposed model achieved better segmentation performances than the baseline one, in both domains, reaching average Precision and Recall scores of 99.0% and 97.5% (clinical domain), along with 77.0% and 71.4% (human activity domain), introducing Neural Networks and Transfer Learning as promising alternatives for pattern searching in time series.

[1]  Alberto Calatroni,et al.  Improving online gesture recognition with template matching methods in accelerometer data , 2012, 2012 12th International Conference on Intelligent Systems Design and Applications (ISDA).

[2]  Patrícia J. Bota,et al.  A Semi-Automatic Annotation Approach for Human Activity Recognition , 2019, Sensors.

[3]  Hugo Gamboa,et al.  Time Alignment Measurement for Time Series , 2018, Pattern Recognit..

[4]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  Hugo Gamboa,et al.  Robust Anomaly Detection in Time Series through Variational AutoEncoders and a Local Similarity Score , 2021, BIOSIGNALS.

[6]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Junliang Liu,et al.  Convolutional neural networks for time series classification , 2017 .

[8]  Hugo Gamboa,et al.  Explaining the Ergonomic Assessment of Human Movement in Industrial Contexts , 2020, BIOSIGNALS.

[9]  Parisa Rashidi,et al.  Human Activity Recognition Using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey , 2020, IEEE Access.

[10]  Cecilio Angulo,et al.  Generating Synthetic ECGs Using GANs for Anonymizing Healthcare Data , 2021 .

[11]  Marco Wiering,et al.  Deep Neural Networks with Intersection over Union Loss for Binary Image Segmentation , 2019, ICPRAM.

[12]  Ashfaqur Rahman,et al.  Convolutional Neural Network for Time Series Cattle Behaviour Classification , 2016, TSAA '16.

[13]  U. Rajendra Acharya,et al.  A deep convolutional neural network model to classify heartbeats , 2017, Comput. Biol. Medicine.

[14]  Sakorn Mekruksavanich,et al.  Biometric User Identification Based on Human Activity Recognition Using Wearable Sensors: An Experiment Using Deep Learning Models , 2021, Electronics.

[15]  Pietro Perona,et al.  Continuous dynamic time warping for translation-invariant curve alignment with applications to signature verification , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  Christian Igel,et al.  U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging , 2019, NeurIPS.

[17]  Annisa Darmawahyuni,et al.  Deep Learning-Based Stacked Denoising and Autoencoder for ECG Heartbeat Classification , 2020, Electronics.

[18]  Sergey Alekseev,et al.  ECG Segmentation by Neural Networks: Errors and Correction , 2018, 2019 International Joint Conference on Neural Networks (IJCNN).

[19]  Meinard Müller,et al.  Dynamic Time Warping , 2008 .

[20]  Shahrokh Valaee,et al.  Recent Advances in Recurrent Neural Networks , 2017, ArXiv.

[21]  Björn Eskofier,et al.  Subsequence dynamic time warping as a method for robust step segmentation using gyroscope signals of daily life activities , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[22]  Hugo Gamboa,et al.  Self-Similarity Matrix of Morphological Features for Motion Data Analysis in Manufacturing Scenarios , 2021, BIOSIGNALS.

[23]  Francesco Beritelli,et al.  Automatic ECG Diagnosis Using Convolutional Neural Network , 2020, Electronics.

[24]  Rui Zhang,et al.  A real time hybrid pattern matching scheme for stock time series , 2010, ADC.

[25]  Achilleas Zapranis,et al.  Dynamic Time Warping for Pattern Recognition , 2016 .

[26]  Simon Fong,et al.  A TIME-SERIES PRE-PROCESSING METHODOLOGY FOR BIOSIGNAL CLASSIFICATION USING STATISTICAL FEATURE EXTRACTION , 2013, BioMed 2013.

[27]  Li Deng,et al.  Deep Discriminative and Generative Models for speech Pattern Recognition , 2016, Handbook of Pattern Recognition and Computer Vision.

[28]  Alfredo Pulvirenti,et al.  Similarity Measures and Dimensionality Reduction Techniques for Time Series Data Mining , 2012 .

[29]  Vit Niennattrakul,et al.  Efficient Subsequence Search on Streaming Data Based on Time Warping Distance , 1970 .

[30]  C. Peng,et al.  Age-related alterations in the fractal scaling of cardiac interbeat interval dynamics. , 1996, The American journal of physiology.

[31]  M. Pontil Leave-one-out error and stability of learning algorithms with applications , 2002 .

[32]  William Robson Schwartz,et al.  ECG-based heartbeat classification for arrhythmia detection: A survey , 2016, Comput. Methods Programs Biomed..

[33]  G.B. Moody,et al.  The impact of the MIT-BIH Arrhythmia Database , 2001, IEEE Engineering in Medicine and Biology Magazine.

[34]  John Cristian Borges Gamboa,et al.  Deep Learning for Time-Series Analysis , 2017, ArXiv.

[35]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[36]  Nikolai Zolotykh,et al.  Deep Learning for ECG Segmentation , 2019, Studies in Computational Intelligence.

[37]  Huanhuan Chen,et al.  Dynamic State Warping , 2017, ArXiv.

[38]  Zehra Cataltepe,et al.  Link prediction using time series of neighborhood-based node similarity scores , 2015, Data Mining and Knowledge Discovery.

[39]  Jessica Lin,et al.  Pattern Recognition in Time Series , 2012 .

[40]  Volker Lohweg,et al.  Survey on time series motif discovery , 2017, WIREs Data Mining Knowl. Discov..

[41]  Loris Nanni,et al.  Handcrafted vs. non-handcrafted features for computer vision classification , 2017, Pattern Recognit..

[42]  Aeilko H. Zwinderman,et al.  Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG , 2000, IEEE Transactions on Biomedical Engineering.

[43]  Olufemi A. Omitaomu,et al.  Weighted dynamic time warping for time series classification , 2011, Pattern Recognit..

[44]  Chao Yang,et al.  A Survey on Deep Transfer Learning , 2018, ICANN.

[45]  Patrícia J. Bota,et al.  TSFEL: Time Series Feature Extraction Library , 2020, SoftwareX.

[46]  Philip de Chazal,et al.  Automatic classification of heartbeats using ECG morphology and heartbeat interval features , 2004, IEEE Transactions on Biomedical Engineering.

[47]  Seung Hwan Park,et al.  SSDTW: Shape segment dynamic time warping , 2020, Expert Syst. Appl..

[48]  Stanislaw Osowski,et al.  ECG beat recognition using fuzzy hybrid neural network , 2001, IEEE Trans. Biomed. Eng..

[49]  Serena Ivaldi,et al.  Human movement and ergonomics: An industry-oriented dataset for collaborative robotics , 2019, Int. J. Robotics Res..

[50]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[51]  Grigory V. Osipov,et al.  LUDB: A New Open-Access Validation Tool for Electrocardiogram Delineation Algorithms , 2020, IEEE Access.

[52]  Tomasz Rymarczyk,et al.  The Use of Time-Frequency Moments as Inputs of LSTM Network for ECG Signal Classification , 2020, Electronics.

[53]  Shengfa Miao,et al.  Predefined pattern detection in large time series , 2016, Inf. Sci..