Making Time-Series Classification More Accurate Using Learned Constraints

It has long been known that Dynamic Time Warping (DTW) is superior to Euclidean distance for classification and clustering of time series. However, until lately, most research has utilized Euclidean distance because it is more efficiently calculated. A recently introduced technique that greatly mitigates DTWs demanding CPU time has sparked a flurry of research activity. However, the technique and its many extensions still only allow DTW to be applied to moderately large datasets. In addition, almost all of the research on DTW has focused exclusively on speeding up its calculation; there has been little work done on improving its accuracy. In this work, we target the accuracy aspect of DTW performance and introduce a new framework that learns arbitrary constraints on the warping path of the DTW calculation. Apart from improving the accuracy of classification, our technique as a side effect speeds up DTW by a wide margin as well. We show the utility of our approach on datasets from diverse domains and demonstrate significant gains in accuracy and efficiency. E u clid ean D istan c e D yn am ic T im e W arp in g D is tan ce Figure 1: Note that while the two time series have an overall similar shape, they are not aligned in the time axis. Euclidean distance, which assumes the i point in one sequence is aligned with the i point in the other, will produce a pessimistic dissimilarity measure. The non-linear Dynamic Time Warped alignment allows a more intuitive distance measure to be calculated.

[1]  R. Manmatha,et al.  Word image matching using dynamic time warping , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[2]  Michael Clausen,et al.  An Efficient Indexing and Search Technique for Multimedia Databases , 2003 .

[3]  Kyuseok Shim,et al.  Fast Similarity Search in the Presence of Noise, Scaling, and Translation in Time-Series Databases , 1995, VLDB.

[4]  Tommi S. Jaakkola,et al.  A new approach to analyzing gene expression time series data , 2002, RECOMB '02.

[5]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[6]  Dennis Shasha,et al.  Query by Humming , 2004 .

[7]  Ada Wai-Chee Fu,et al.  Efficient time series matching by wavelets , 1999, Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337).

[8]  G. W. Hughes,et al.  Minimum Prediction Residual Principle Applied to Speech Recognition , 1975 .

[9]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[10]  Georges Hébrail,et al.  Interactive Interpretation of Kohonen Maps Applied to Curves , 1998, KDD.

[11]  Kim-Fung Man,et al.  Genetic Time Warping for Isolated Word Recognition , 1996, Int. J. Pattern Recognit. Artif. Intell..

[12]  Pietro Perona,et al.  Continuous dynamic time warping for translation-invariant curve alignment with applications to signature verification , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Eamonn J. Keogh,et al.  Probabilistic discovery of time series motifs , 2003, KDD '03.

[14]  Charles C. Tappert,et al.  Memory and time improvements in a dynamic programming algorithm for matching speech patterns , 1978 .

[15]  Dennis Shasha,et al.  Query by humming: in action with its technology revealed , 2003, SIGMOD '03.

[16]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[17]  Dipankar Dasgupta,et al.  Novelty detection in time series data using ideas from immunology , 1996 .

[18]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[19]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[20]  Dennis Shasha,et al.  Warping indexes with envelope transforms for query by humming , 2003, SIGMOD '03.

[21]  Zsolt Miklós Kovács-Vajna,et al.  A Fingerprint Verification System Based on Triangular Matching and Dynamic Time Warping , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Eamonn J. Keogh,et al.  On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration , 2002, Data Mining and Knowledge Discovery.

[23]  Man Hon Wong,et al.  Efficient subsequence matching for sequences databases under time warping , 2003, Seventh International Database Engineering and Applications Symposium, 2003. Proceedings..

[24]  George M. Church,et al.  Aligning gene expression time series with time warping algorithms , 2001, Bioinform..

[25]  Peter Clote,et al.  An Application of Time Warping to Functional Genomics , 2003 .

[26]  Christos Faloutsos,et al.  Fast Time Sequence Indexing for Arbitrary Lp Norms , 2000, VLDB.

[27]  S. Levinson,et al.  Considerations in dynamic time warping algorithms for discrete word recognition , 1978 .

[28]  Clement T. Yu,et al.  Haar Wavelets for Efficient Similarity Search of Time-Series: With and Without Time Warping , 2003, IEEE Trans. Knowl. Data Eng..

[29]  E. Caiani,et al.  Warped-average template technique to track on a cycle-by-cycle basis the cardiac filling phases on left ventricular volume , 1998, Computers in Cardiology 1998. Vol. 25 (Cat. No.98CH36292).

[30]  Aaron E. Rosenberg,et al.  Performance tradeoffs in dynamic time warping algorithms for isolated word recognition , 1980 .

[31]  Joseph B. Kruskal,et al.  Time Warps, String Edits, and Macromolecules , 1999 .

[32]  Juan José Rodríguez Diez,et al.  Applying Boosting to Similarity Literals for Time Series Classification , 2000, Multiple Classifier Systems.

[33]  Eamonn J. Keogh,et al.  Dimensionality Reduction for Fast Similarity Search in Large Time Series Databases , 2001, Knowledge and Information Systems.

[34]  Heikki Mannila,et al.  Rule Discovery from Time Series , 1998, KDD.

[35]  R. Manmatha,et al.  Indexing of Handwritten Historical Documents - Recent Progress , 2003 .

[36]  Eamonn J. Keogh,et al.  Iterative Deepening Dynamic Time Warping for Time Series , 2002, SDM.

[37]  Eamonn J. Keogh,et al.  Locally adaptive dimensionality reduction for indexing large time series databases , 2001, SIGMOD '01.

[38]  George Tzanetakis,et al.  Polyphonic audio matching and alignment for music retrieval , 2003, 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684).

[39]  Eamonn J. Keogh,et al.  Scaling up dynamic time warping for datamining applications , 2000, KDD '00.

[40]  Larry S. Davis,et al.  Towards 3-D model-based tracking and recognition of human movement: a multi-view approach , 1995 .

[41]  Paul R. Cohen,et al.  Learned models for continuous planning , 1999, AISTATS.

[42]  Shaogang Gong,et al.  Learning Intrinsic Video Content Using Levenshtein Distance in Graph Partitioning , 2002, ECCV.

[43]  Christos Faloutsos,et al.  Efficient retrieval of similar time sequences under time warping , 1998, Proceedings 14th International Conference on Data Engineering.

[44]  Clu-istos Foutsos,et al.  Fast subsequence matching in time-series databases , 1994, SIGMOD '94.

[45]  Christos Faloutsos,et al.  Efficiently supporting ad hoc queries in large datasets of time sequences , 1997, SIGMOD '97.

[46]  Mohammed Waleed Kadous,et al.  Learning Comprehensible Descriptions of Multivariate Time Series , 1999, ICML.

[47]  Dimitrios Gunopulos,et al.  Discovering similar multidimensional trajectories , 2002, Proceedings 18th International Conference on Data Engineering.

[48]  Klaus Gollmer,et al.  Detection of distorted pattern using dynamic time warping algorithm and application for supervision , 1995 .