Adaptive State Continuity-Based Sparse Inverse Covariance Clustering for Multivariate Time Series

Compared with univariate time series clustering, multivariate time series (MTS) clustering has become a challenging research topic on the data mining of time series. In this paper, a novel model-based approach Adaptive State Continuity-Based Sparse Inverse Covariance Clustering (ASCSICC) is proposed for MTS clustering. Specifically, the state continuity is introduced to make the traditional Gaussian mixture model (GMM) applicable to time series clustering. To prevent overfitting, the alternating direction method of multipliers (ADMM) is applied to optimize the parameter of GMM inverse covariance. In addition, the proposed approach simultaneously segments and clusters the time series. Technically, first, the adaptive state continuity is estimated based on the distance similarity of adjacent time series data. Then, a dynamic programming algorithm of cluster assignment by adaptive state continuity is taken as the E-step, and the ADMM for solving sparse inverse covariance is taken as the M-step. E-step and M-step are combined into an Expectation-Maximization (EM) algorithm to conduct the clustering process. Finally, we show the effectiveness of the proposed approach by comparing the ASC-SICC with several state-of-the-art approaches in experiments on two datasets from real applications.

[1]  Jean Pierre Delmas,et al.  Asymptotic Generalized Eigenvalue Distribution of Block Multilevel Toeplitz Matrices , 2009, IEEE Transactions on Signal Processing.

[2]  Andrew K. C. Wong,et al.  Discovery of Temporal Associations in Multivariate Time Series , 2014, IEEE Transactions on Knowledge and Data Engineering.

[3]  Ying Wah Teh,et al.  Time-series clustering - A decade review , 2015, Inf. Syst..

[4]  Dapeng Tao,et al.  Clustering Multivariate Time Series Data via Multi-Nonnegative Matrix Factorization in Multi-Relational Networks , 2018, IEEE Access.

[5]  Didier Stricker,et al.  Exploring and extending the boundaries of physical activity recognition , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[6]  Hailin Li,et al.  Multivariate time series clustering based on common principal component analysis , 2019, Neurocomputing.

[7]  Jessika Weiss,et al.  Graphical Models In Applied Multivariate Statistics , 2016 .

[8]  Robert Jenssen,et al.  Time Series Cluster Kernel for Learning Similarities between Multivariate Time Series with Missing Data , 2017, Pattern Recognit..

[9]  Eamonn J. Keogh,et al.  Classification of streaming time series under more realistic assumptions , 2015, Data Mining and Knowledge Discovery.

[10]  Beibei Zhang,et al.  Clustering time series based on dependence structure , 2018, PloS one.

[11]  Witold Pedrycz,et al.  Time-series clustering based on linear fuzzy information granules , 2018, Appl. Soft Comput..

[12]  Hubert Cardot,et al.  SOM time series clustering and prediction with recurrent neural networks , 2011, Neurocomputing.

[13]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[14]  A. Raftery,et al.  Model-based Gaussian and non-Gaussian clustering , 1993 .

[15]  Chengqi Zhang,et al.  Salient Subsequence Learning for Time Series Clustering , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[17]  Hong He,et al.  Pattern Clustering of Hysteresis Time Series With Multivalued Mapping Using Tensor Decomposition , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[18]  Xiaozhe Wang,et al.  Characteristic-Based Clustering for Time Series Data , 2006, Data Mining and Knowledge Discovery.

[19]  Stephen P. Boyd,et al.  Toeplitz Inverse Covariance-Based Clustering of Multivariate Time Series Data , 2017, KDD.

[20]  Witold Pedrycz,et al.  Clustering of interval-valued time series of unequal length based on improved dynamic time warping , 2019, Expert Syst. Appl..

[21]  Marcelo Embiruçu,et al.  A wavelet-based clustering of multivariate time series using a Multiscale SPCA approach , 2016, Comput. Ind. Eng..

[22]  Hongqing Zhu,et al.  Merging Student's-t and Rayleigh distributions regression mixture model for clustering time-series , 2017, Neurocomputing.

[23]  T. Warren Liao,et al.  Clustering of time series data - a survey , 2005, Pattern Recognit..

[24]  Andrew L. Janke,et al.  Maximum Pseudolikelihood Estimation for Model-Based Clustering of Time Series Data , 2016, Neural Computation.