Sequential data classification by dynamic state warping

The ubiquity of sequences in many domains enhances significant recent interest in sequence learning, for which a basic problem is how to measure the distance between sequences. Dynamic time warping (DTW) aligns two sequences by nonlinear local warping and returns a distance value. DTW shows superior ability in many applications, e.g. video, image, etc. However, in DTW, two points are paired essentially based on point-to-point comparisons without considering the autocorrelation of sequences. Thus, points with different semantic meanings, e.g. peaks and valleys, may be matched providing their coordinate values are similar. As a result, DTW may be sensitive to noise and poorly interpretable. This paper proposes an improved alignment method, dynamic state warping (DSW). DSW integrates the dynamic information of sequences into DTW by converting each time point into a latent state. Alignment is performed by using the state sequences. Thus, DSW is able to yield alignment that is semantically more interpretable than that of DTW. Using one nearest neighbour classifier, DSW shows significant improvement on classification accuracy in comparison with Euclidean distance (68/85 wins), DTW (70/85 wins) and its variants. We also empirically demonstrate that DSW is more robust and scales better to long sequences than Euclidean distance and DTW.

[1]  Vladimir Pavlovic,et al.  Robust time-series retrieval using probabilistic adaptive segmental alignment , 2015, Knowledge and Information Systems.

[2]  Carolyn Penstein Rosé,et al.  Time Series Analysis of Nursing Notes for Mortality Prediction via a State Transition Topic Model , 2015, CIKM.

[3]  Clu-istos Foutsos,et al.  Fast subsequence matching in time-series databases , 1994, SIGMOD '94.

[4]  Peter Tiño,et al.  Simple Deterministically Constructed Cycle Reservoirs with Regular Jumps , 2012, Neural Computation.

[5]  X. Yao,et al.  Model-based kernel for efficient time series analysis , 2013, KDD.

[6]  Seddik Hassene,et al.  Efficient speech de-noising applied to colored noise based dynamic low-pass filter supervised by cascade neural networks , 2013, 2013 International Conference on Electrical Engineering and Software Applications.

[7]  Eamonn J. Keogh,et al.  Searching and Mining Trillions of Time Series Subsequences under Dynamic Time Warping , 2012, KDD.

[8]  Eamonn J. Keogh,et al.  CID: an efficient complexity-invariant distance for time series , 2013, Data Mining and Knowledge Discovery.

[9]  Frank Höppner,et al.  Improving time series similarity measures by integrating preprocessing steps , 2017, Data Mining and Knowledge Discovery.

[10]  Fernando De la Torre,et al.  Generalized Canonical Time Warping , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Laurens van der Maaten,et al.  Modeling Time Series Similarity with Siamese Recurrent Networks , 2016, ArXiv.

[12]  Huanhuan Chen,et al.  Learning in the Model Space for Cognitive Fault Diagnosis , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Olufemi A. Omitaomu,et al.  Weighted dynamic time warping for time series classification , 2011, Pattern Recognit..

[14]  Nitish Srivastava,et al.  Unsupervised Learning of Video Representations using LSTMs , 2015, ICML.

[15]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[16]  Eamonn J. Keogh,et al.  Time series shapelets: a novel technique that allows accurate, interpretable and fast classification , 2010, Data Mining and Knowledge Discovery.

[17]  Eamonn J. Keogh,et al.  Time series shapelets: a new primitive for data mining , 2009, KDD.

[18]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[19]  Henry Markram,et al.  The "Liquid Computer": A Novel Strategy for Real-Time Computing on Time Series , 2002 .

[20]  Krisztian Buza,et al.  Projection-Based Person Identification , 2017, CORES.

[21]  James Large,et al.  The Great Time Series Classification Bake Off: An Experimental Evaluation of Recently Proposed Algorithms. Extended Version , 2016, ArXiv.

[22]  Diane J. Cook,et al.  A survey of methods for time series change point detection , 2017, Knowledge and Information Systems.

[23]  Petra Hermann,et al.  Resting State fMRI Functional Connectivity Analysis Using Dynamic Time Warping , 2017, Front. Neurosci..

[24]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[25]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[26]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[27]  Huanhuan Chen,et al.  Model Metric Co-Learning for Time Series Classification , 2015, IJCAI.

[28]  Jonathan Tompson,et al.  Unsupervised Learning of Spatiotemporally Coherent Metrics , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  Damien Garreau,et al.  Metric Learning for Temporal Sequence Alignment , 2014, NIPS.

[30]  Geoffrey I. Webb,et al.  Dynamic Time Warping Averaging of Time Series Allows Faster and More Accurate Classification , 2014, 2014 IEEE International Conference on Data Mining.

[31]  J A Kogan,et al.  Automated recognition of bird song elements from continuous recordings using dynamic time warping and hidden Markov models: a comparative study. , 1998, The Journal of the Acoustical Society of America.

[32]  Eamonn J. Keogh,et al.  Derivative Dynamic Time Warping , 2001, SDM.

[33]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[34]  Mohinder S. Grewal,et al.  Kalman Filtering , 2011, International Encyclopedia of Statistical Science.

[35]  Laurens van der Maaten,et al.  Learning Discriminative Fisher Kernels , 2011, ICML.

[36]  Daniel Lemire,et al.  Faster retrieval with a two-pass dynamic-time-warping lower bound , 2008, Pattern Recognit..

[37]  Milos Hauskrecht,et al.  An efficient pattern mining approach for event detection in multivariate temporal data , 2015, Knowledge and Information Systems.

[38]  Eamonn J. Keogh,et al.  Accelerating Dynamic Time Warping Clustering with a Novel Admissible Pruning Strategy , 2015, KDD.