Nearest Subspace with Discriminative Regularization for Time Series Classification

For time series classification (TSC) problem, many studies focus on elastic distance measures for comparing time series and complete the task with the help of Nearest Neighbour (NN) classifier. This is mainly due to the fact that the order of variables is a crucial factor for time series. Unlike the NN classifier only considers one training sample, in this paper, we propose an improved Nearest Subspace (NS) classifier to classify new time series. By adding a discriminative regularization item, the improved NS classifier takes full advantage of all training time series of one class. Two kinds of discriminative regularization items are employed in our method. One is directly calculated based on Euclidean distance of time series. For the other, we obtain the regularization items from a lower-dimensional subspace. Two well-known dimensional reduction methods, Generalized Eigenvector Method (GEM) and Local Fisher Discriminant Analysis (LFDA), are employed to complete this task. Furthermore, we combine these improved NS classifiers through ensemble schemes to accommodate different time series datasets. Through extensive experiments on all UCR and UEA datasets, we demonstrate that the proposed method can gain better performance than NN classifiers with different elastic distance measures and other classifiers.

[1]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[2]  Tomasz Górecki,et al.  Using derivatives in time series classification , 2012, Data Mining and Knowledge Discovery.

[3]  Eamonn J. Keogh,et al.  Three Myths about Dynamic Time Warping Data Mining , 2005, SDM.

[4]  Shuzhi Sam Ge,et al.  $k$-NS: A Classifier by the Distance to the Nearest Subspace , 2011, IEEE Transactions on Neural Networks.

[5]  Eamonn J. Keogh,et al.  Addressing Big Data Time Series: Mining Trillions of Time Series Subsequences Under Dynamic Time Warping , 2013, TKDD.

[6]  Paul Mineiro,et al.  Discriminative Features via Generalized Eigenvectors , 2013, ICML.

[7]  Fatih Murat Porikli,et al.  Connecting the dots in multi-class classification: From nearest subspace to collaborative representation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Eamonn Keogh Exact Indexing of Dynamic Time Warping , 2002, VLDB.

[9]  Eamonn J. Keogh,et al.  The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances , 2016, Data Mining and Knowledge Discovery.

[10]  Gautam Das,et al.  The Move-Split-Merge Metric for Time Series , 2013, IEEE Transactions on Knowledge and Data Engineering.

[11]  Lei Chen,et al.  On The Marriage of Lp-norms and Edit Distance , 2004, VLDB.

[12]  Eamonn J. Keogh,et al.  Derivative Dynamic Time Warping , 2001, SDM.

[13]  Jason Lines,et al.  Time-Series Classification with COTE: The Collective of Transformation-Based Ensembles , 2015, IEEE Trans. Knowl. Data Eng..

[14]  Masashi Sugiyama,et al.  Local Fisher discriminant analysis for supervised dimensionality reduction , 2006, ICML.

[15]  Giandomenico Spezzano,et al.  A single pass algorithm for clustering evolving data streams based on swarm intelligence , 2011, Data Mining and Knowledge Discovery.

[16]  John C. Platt,et al.  Translingual Document Representations from Discriminative Projections , 2010, EMNLP.

[17]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[18]  Sergio A. Velastin,et al.  Local Fisher Discriminant Analysis for Pedestrian Re-identification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Pierre-François Marteau,et al.  Time Warp Edit Distance with Stiffness Adjustment for Time Series Matching , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Lei Chen,et al.  Robust and fast similarity search for moving object trajectories , 2005, SIGMOD '05.

[22]  Eamonn J. Keogh,et al.  A Complexity-Invariant Distance Measure for Time Series , 2011, SDM.

[23]  Tomasz Górecki,et al.  Non-isometric transforms in time series classification using DTW , 2014, Knowl. Based Syst..

[24]  Lothar Reichel,et al.  A new Tikhonov regularization method , 2011, Numerical Algorithms.

[25]  James E. Fowler,et al.  Nearest Regularized Subspace for Hyperspectral Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Jason Lines,et al.  Time series classification with ensembles of elastic distance measures , 2015, Data Mining and Knowledge Discovery.

[27]  Olufemi A. Omitaomu,et al.  Weighted dynamic time warping for time series classification , 2011, Pattern Recognit..

[28]  Eamonn J. Keogh,et al.  CID: an efficient complexity-invariant distance for time series , 2013, Data Mining and Knowledge Discovery.

[29]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..