Combining discrete SVM and fixed cardinality warping distances for multivariate time series classification

Time series classification is a supervised learning problem aimed at labeling temporally structured multivariate sequences of variable length. The most common approach reduces time series classification to a static problem by suitably transforming the set of multivariate input sequences into a rectangular table composed by a fixed number of columns. Then, one of the alternative efficient methods for classification is applied for predicting the class of new temporal sequences. In this paper, we propose a new classification method, based on a temporal extension of discrete support vector machines, that benefits from the notions of warping distance and softened variable margin. Furthermore, in order to transform a temporal dataset into a rectangular shape, we also develop a new method based on fixed cardinality warping distances. Computational tests performed on both benchmark and real marketing temporal datasets indicate the effectiveness of the proposed method in comparison to other techniques.

[1]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[2]  Andrea Omicini,et al.  Proceedings of the 2004 ACM Symposium on Applied Computing (SAC 2004) , 2004 .

[3]  Claude Sammut,et al.  Classification of Multivariate Time Series and Structured Data Using Constructive Induction , 2005, Machine Learning.

[4]  Anne H. Anderson,et al.  Proceedings of Eurospeech , 2003, ISCA 2003.

[5]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[6]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[7]  Stefan Irnich,et al.  Shortest Path Problems with Resource Constraints , 2005 .

[8]  Robert Sedgewick,et al.  Algorithms in Java, Part 5: Graph Algorithms , 2003 .

[9]  Sebastiano Impedovo,et al.  Frontiers in Handwriting Recognition , 1994 .

[10]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[11]  Carlo Vercellis,et al.  Accurately learning from few examples with a polyhedral classifier , 2007, Comput. Optim. Appl..

[12]  Panos M. Pardalos,et al.  On the time series support vector machine using dynamic time warping kernel for brain activity classification , 2008 .

[13]  Dimitrios Gunopulos,et al.  Indexing Multidimensional Time-Series , 2004, The VLDB Journal.

[14]  Carlo Vercellis,et al.  Multivariate classification trees based on minimum features discrete support vector machines , 2003 .

[15]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[16]  Li Wei,et al.  Fast time series classification using numerosity reduction , 2006, ICML.

[17]  Shigeki Sagayama,et al.  Support vector machine with dynamic time-alignment kernel for speech recognition , 2001, INTERSPEECH.

[18]  Robert Sedgewick,et al.  Algorithms in Java , 2003 .

[19]  Nicos Christofides,et al.  An algorithm for the resource constrained shortest path problem , 1989, Networks.

[20]  Yannis Manolopoulos,et al.  Feature-based classification of time-series data , 2001 .

[21]  Claus Bahlmann,et al.  Online handwriting recognition with support vector machines - a kernel approach , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[22]  Tomoko Matsui,et al.  A Kernel for Time Series Based on Global Alignments , 2006, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[23]  Juan José Rodríguez Diez,et al.  Interval and dynamic time warping-based decision trees , 2004, SAC '04.

[24]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[25]  Carlo Vercellis,et al.  Discrete support vector decision trees via tabu search , 2004, Comput. Stat. Data Anal..

[26]  Edward Y. Chang,et al.  Distance-function design and fusion for sequence data , 2004, CIKM '04.

[27]  Carlo Vercellis,et al.  Multicategory classification via discrete support vector machines , 2009, Comput. Manag. Sci..

[28]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[29]  Bernhard Schölkopf,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[30]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[31]  C. Berg,et al.  Harmonic Analysis on Semigroups: Theory of Positive Definite and Related Functions , 1984 .