Time Series Analysis using a Kernel based Multi-Modal Uncertainty Decomposition Framework

This paper proposes a kernel based information theoretic framework that provides a sensitive multi-modal quantification of time series uncertainty by leveraging a quantum physical description of the projected feature space of data in a Reproducing Kernel Hilbert Space (RKHS). We specifically modify the kernel mean embedding, which yields an intuitive physical interpretation of the signal structure, to produce a data based “dynamic potential field”. This results in a new energy based formulation that exploits the mathematics of quantum theory and facilitates a multi-modal physics based uncertainty representation of the signal at each data sample. We demonstrate in this paper that such uncertainty features provide a better ability for online detection of statistical change points in time series data when compared to existing non-parametric and unsupervised methods. We also demonstrate a better ability of the framework in clustering time series sequences when compared to discrete wavelet transform features on a subset of VidTIMIT speaker recognition corpus.

[1]  Ryan P. Adams,et al.  Bayesian Online Changepoint Detection , 2007, 0710.3742.

[2]  Mahesh Kumar,et al.  Clustering seasonality patterns in the presence of errors , 2002, KDD.

[3]  Kenji Yamanishi,et al.  A unifying framework for detecting outliers and change points from time series , 2006, IEEE Transactions on Knowledge and Data Engineering.

[4]  E. Villaseñor Introduction to Quantum Mechanics , 2008, Nature.

[5]  K. Kosmelj,et al.  Cross-sectional approach for clustering time varying data , 1990 .

[6]  Brian C. Lovell,et al.  Multi-Region Probabilistic Histograms for Robust and Scalable Identity Inference , 2009, ICB.

[7]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[8]  Yoram Singer,et al.  The Hierarchical Hidden Markov Model: Analysis and Applications , 1998, Machine Learning.

[9]  E. Parzen On Estimation of a Probability Density Function and Mode , 1962 .

[10]  Thomas J. Watson,et al.  An empirical study of the naive Bayes classifier , 2001 .

[11]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[12]  J. Neyman Contributions to the theory of statistics , 1956 .

[13]  Kwangwon Ahn,et al.  Modeling stock return distributions with a quantum harmonic oscillator , 2017 .

[14]  Diane J. Cook,et al.  A survey of methods for time series change point detection , 2017, Knowledge and Information Systems.

[15]  J. Tierney,et al.  A study of LPC analysis of speech in additive noise , 1980 .

[16]  Jian-Wei Zhang,et al.  Quantum Brownian motion model for the stock market , 2014, 1405.3512.

[17]  Rishabh Singh,et al.  Towards a Kernel based Physical Interpretation of Model Uncertainty , 2020, ArXiv.

[18]  Ying Wah Teh,et al.  Time-series clustering - A decade review , 2015, Inf. Syst..

[19]  Robert Savit,et al.  Stationarity and nonstationarity in time series analysis , 1996 .

[20]  Peter Harremoës,et al.  Rényi Divergence and Kullback-Leibler Divergence , 2012, IEEE Transactions on Information Theory.

[21]  Robert H. Shumway,et al.  Discrimination and Clustering for Multivariate Time Series , 1998 .

[22]  T. Warren Liao,et al.  Clustering of time series data - a survey , 2005, Pattern Recognit..

[23]  G. Crooks On Measures of Entropy and Information , 2015 .

[24]  N. Aronszajn Theory of Reproducing Kernels. , 1950 .

[25]  Frank Nielsen,et al.  An Information-Geometric Characterization of Chernoff Information , 2013, IEEE Signal Processing Letters.

[26]  Jose C. Principe,et al.  Information Theoretic Learning - Renyi's Entropy and Kernel Perspectives , 2010, Information Theoretic Learning.

[27]  Bernhard Schölkopf,et al.  Kernel Mean Embedding of Distributions: A Review and Beyonds , 2016, Found. Trends Mach. Learn..

[28]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[29]  Christopher Heil,et al.  Continuous and Discrete Wavelet Transforms , 1989, SIAM Rev..

[30]  Deniz Erdogmus,et al.  Information Theoretic Learning , 2005, Encyclopedia of Artificial Intelligence.