Multivariate time-series analysis and diffusion maps

Dimensionality reduction in multivariate time series analysis has broad applications, ranging from financial data analysis to biomedical research. However, high levels of ambient noise and various interferences result in nonstationary signals, which may lead to inefficient performance of conventional methods. In this paper, we propose a nonlinear dimensionality reduction framework using diffusion maps on a learned statistical manifold, which gives rise to the construction of a low-dimensional representation of the high-dimensional nonstationary time series. We show that diffusion maps, with affinity kernels based on the Kullback-Leibler divergence between the local statistics of samples, allow for efficient approximation of pairwise geodesic distances. To construct the statistical manifold, we estimate time-evolving parametric distributions by designing a family of Bayesian generative models. The proposed framework can be applied to problems in which the time-evolving distributions (of temporally localized data), rather than the samples themselves, are driven by a low-dimensional underlying process. We provide efficient parameter estimation and dimensionality reduction methodologies, and apply them to two applications: music analysis and epileptic-seizure prediction. Author-HighlightsWe build a class of Bayesian models to learn the evolving statistics of time series.We construct diffusion maps based on the time-evolving distributional information.The proposed method recovers the underlying process controlling the time series.The proposed framework is applied to the analysis of music and icEEG recordings.

[1]  Ronald R. Coifman,et al.  Intrinsic modeling of stochastic dynamical systems using empirical geometry , 2015 .

[2]  D. Dunson,et al.  Efficient Gaussian process regression for large datasets. , 2011, Biometrika.

[3]  Andrew Herzog,et al.  Neuroendocrinological aspects of epilepsy: Important issues and trends in future research , 2011, Epilepsy & Behavior.

[4]  R. Coifman,et al.  Anisotropic diffusion on sub-manifolds with application to Earth structure classification , 2012 .

[5]  Mike West,et al.  Autoregressive Models for Variance Matrices: Stationary Inverse Wishart Processes , 2011, 1107.5239.

[6]  Jean-Christophe Nebel,et al.  Temporal Extension of Laplacian Eigenmaps for Unsupervised Dimensionality Reduction of Time Series , 2010, 2010 20th International Conference on Pattern Recognition.

[7]  S. Frühwirth-Schnatter Data Augmentation and Dynamic Linear Models , 1994 .

[8]  Ian T. Jolliffe,et al.  Estimating common trends in multivariate time series using dynamic factor analysis , 2003 .

[9]  Alfred O. Hero,et al.  Bayesian inference of the number of factors in gene-expression analysis: application to human virus challenge studies , 2010, BMC Bioinformatics.

[10]  C. Carvalho,et al.  A sparse factor analytic probit model for congressional voting patterns , 2012 .

[11]  Stéphane Mallat,et al.  Manifold Learning for Latent Variable Inference in Dynamical Systems , 2015, IEEE Transactions on Signal Processing.

[12]  Tobias Loddenkemper,et al.  Seizure detection, seizure prediction, and closed-loop warning systems in epilepsy , 2014, Epilepsy & Behavior.

[13]  Ruey S. Tsay,et al.  Analysis of Financial Time Series , 2005 .

[14]  Alfred O. Hero,et al.  Information-Geometric Dimensionality Reduction , 2011, IEEE Signal Processing Magazine.

[15]  Michel Le Van Quyen,et al.  Probing cortical excitability using cross-frequency coupling in intracranial EEG recordings: A new method for seizure prediction , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  Ronald R. Coifman,et al.  Diffusion Maps for Signal Processing: A Deeper Look at Manifold-Learning Techniques Based on Kernels and Graphs , 2013, IEEE Signal Processing Magazine.

[17]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[18]  Ronen Talmon,et al.  Empirical intrinsic geometry for nonlinear modeling and time series filtering , 2013, Proceedings of the National Academy of Sciences.

[19]  Maja J. Mataric,et al.  A spatio-temporal extension to Isomap nonlinear dimension reduction , 2004, ICML.

[20]  Ronald R. Coifman,et al.  Graph Laplacian Tomography From Unknown Random Projections , 2008, IEEE Transactions on Image Processing.

[21]  David B. Dunson,et al.  Bayesian nonparametric covariance regression , 2011, J. Mach. Learn. Res..

[22]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[23]  Alfred O. Hero,et al.  FINE: Fisher Information Nonparametric Embedding , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Matthias Hein Intrinsic Dimensionality Estimation of Submanifolds in R , 2005 .

[25]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[26]  Klaus Lehnertz,et al.  Controversies in epilepsy: Debates held during the Fourth International Workshop on Seizure Prediction , 2010, Epilepsy & Behavior.

[27]  R. Coifman,et al.  Empirical Intrinsic Modeling of Signals and Information Geometry , 2012 .

[28]  Ronald R. Coifman,et al.  Supervised Graph-Based Processing for Sequential Transient Interference Suppression , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[29]  David B. Dunson,et al.  The Kernel Beta Process , 2011, NIPS.

[30]  R. Dahlhaus On the Kullback-Leibler information divergence of locally stationary processes , 1996 .

[31]  Ryan P. Adams,et al.  Slice sampling covariance hyperparameters of latent Gaussian models , 2010, NIPS.

[32]  Ronald R. Coifman,et al.  Texture separation via a reference set , 2014 .

[33]  J. S. Rao,et al.  Spike and slab variable selection: Frequentist and Bayesian strategies , 2005, math/0505633.

[34]  Ronald R. Coifman,et al.  Diffusion maps for changing data , 2012, ArXiv.

[35]  L. Bauwens,et al.  Multivariate GARCH Models: A Survey , 2003 .

[36]  R. Coifman,et al.  Non-linear independent component analysis with diffusion maps , 2008 .

[37]  Nuno Vasconcelos,et al.  A Kullback-Leibler Divergence Based Kernel for SVM Classification in Multimedia Applications , 2003, NIPS.

[38]  Stephen M. Myers,et al.  Seizure prediction: Methods , 2011, Epilepsy & Behavior.

[39]  Ronen Talmon,et al.  Identifying preseizure state in intracranial EEG data using diffusion kernels. , 2013, Mathematical biosciences and engineering : MBE.

[40]  Carol Alexander Moving Average Models for Volatility and Correlation, and Covariance Matrices , 2008 .

[41]  Frank P. Ferrie,et al.  A Note on Metric Properties for Some Divergence Measures: The Gaussian Case , 2012, ACML.

[42]  Mark Frei Seizure detection , 2013, Scholarpedia.

[43]  J. Gotman,et al.  Seizure prediction in patients with mesial temporal lobe epilepsy using EEG measures of state similarity , 2013, Clinical Neurophysiology.

[44]  Jaeyong Lee,et al.  GENERALIZED DOUBLE PARETO SHRINKAGE. , 2011, Statistica Sinica.

[45]  Ali H. Shoeb,et al.  Application of Machine Learning To Epileptic Seizure Detection , 2010, ICML.

[46]  Frank P. Ferrie,et al.  Relaxed Exponential Kernels for Unsupervised Learning , 2011, DAGM-Symposium.

[47]  Carol Alexander Chair and Moving Average Models for Volatility and Correlation, and Covariance Matrices , 2008 .

[48]  B. Nadler,et al.  Diffusion maps, spectral clustering and reaction coordinates of dynamical systems , 2005, math/0503445.

[49]  Daniele Durante,et al.  Locally adaptive Bayesian covariance regression , 2012 .

[50]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[51]  Timo Terasvirta,et al.  Multivariate GARCH Models , 2008 .

[52]  Fan Chung,et al.  Spectral Graph Theory , 1996 .