Manifold Learning for Latent Variable Inference in Dynamical Systems

We study the inference of latent intrinsic variables of dynamical systems from output signal measurements. The primary focus is the construction of an intrinsic distance between signal measurements, which is independent of the measurement device. This distance enables us to infer the latent intrinsic variables through the solution of an eigenvector problem with a Laplace operator based on a kernel. The signal geometry and its dynamics are represented with nonlinear observers. An analysis of the properties of the observers that allow for accurate recovery of the latent variables is given, and a way to test whether these properties are satisfied from the measurements is proposed. Scattering and window Fourier transform observers are compared. Applications are shown on simulated data, and on real intracranial Electroencephalography (EEG) signals of epileptic patients recorded prior to seizures.

[1]  Matthias Hein.  Intrinsic Dimensionality Estimation of Submanifolds in R , 2005 .

[2]  Ronen Talmon,et al.  Empirical intrinsic geometry for nonlinear modeling and time series filtering , 2013, Proceedings of the National Academy of Sciences.

[3]  Ronald R. Coifman,et al.  Intrinsic modeling of stochastic dynamical systems using empirical geometry , 2015 .

[4]  V. Araújo.  Random Dynamical Systems , 2006, math/0608162.

[5]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[6]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Ronald R. Coifman,et al.  Diffusion Maps for Signal Processing: A Deeper Look at Manifold-Learning Techniques Based on Kernels and Graphs , 2013, IEEE Signal Processing Magazine.

[8]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[9]  Nahum Shimkin,et al.  Nonlinear Control Systems , 2008 .

[10]  Anima Anandkumar,et al.  Tensor decompositions for learning latent variable models , 2012, J. Mach. Learn. Res..

[11]  Andreas Ruttor,et al.  Approximate inference in continuous time Gaussian-Jump processes , 2010, NIPS.

[12]  Stephen M. Myers,et al.  Seizure prediction: Methods , 2011, Epilepsy & Behavior.

[13]  Ronen Talmon,et al.  Identifying preseizure state in intracranial EEG data using diffusion kernels. , 2013, Mathematical biosciences and engineering : MBE.

[14]  Ann B. Lee,et al.  Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[16]  R.W. Schafer,et al.  Digital representations of speech signals , 1975, Proceedings of the IEEE.

[17]  Thomas Quatieri,et al.  Discrete-Time Speech Signal Processing: Principles and Practice , 2001 .

[18]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[19]  Klaus Lehnertz,et al.  Controversies in epilepsy: Debates held during the Fourth International Workshop on Seizure Prediction , 2010, Epilepsy & Behavior.

[20]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[21]  R. Coifman,et al.  Empirical Intrinsic Modeling of Signals and Information Geometry , 2012 .

[22]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[23]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[24]  Stéphane Mallat,et al.  Group Invariant Scattering , 2011, ArXiv.

[25]  Anima Anandkumar,et al.  A Spectral Algorithm for Latent Dirichlet Allocation , 2012, Algorithmica.

[26]  A. Krener,et al.  Nonlinear observers with linearizable error dynamics , 1985 .

[27]  Matthias Hein,et al.  Intrinsic dimensionality estimation of submanifolds in Rd , 2005, ICML.

[28]  Ronald R. Coifman,et al.  Multivariate time-series analysis and diffusion maps , 2015, Signal Process..

[29]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[30]  Israel Cohen,et al.  Supervised source localization using diffusion kernels , 2011, 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA).

[31]  W. Hauser,et al.  Descriptive epidemiology of epilepsy: contributions of population-based studies from Rochester, Minnesota. , 1996, Mayo Clinic proceedings.

[32]  Michael I. Jordan,et al.  Nonparametric Bayesian Learning of Switching Linear Dynamical Systems , 2008, NIPS.

[33]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[34]  Tobias Loddenkemper,et al.  Seizure detection, seizure prediction, and closed-loop warning systems in epilepsy , 2014, Epilepsy & Behavior.

[35]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[36]  Chinmay Hegde,et al.  LIE operators for compressive sensing , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  Ronald R. Coifman,et al.  Parametrization of Linear Systems Using Diffusion Kernels , 2012, IEEE Transactions on Signal Processing.

[38]  Ronald R. Coifman,et al.  Graph Laplacian Tomography From Unknown Random Projections , 2008, IEEE Transactions on Image Processing.

[39]  Amit Singer,et al.  Detecting intrinsic slow variables in stochastic dynamical systems by anisotropic diffusion maps , 2009, Proceedings of the National Academy of Sciences.

[40]  Patrick E. McSharry,et al.  Prediction of epileptic seizures: are nonlinear methods relevant? , 2003, Nature Medicine.

[41]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[42]  Joakim Andén,et al.  Deep Scattering Spectrum , 2013, IEEE Transactions on Signal Processing.

[43]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[44]  John D. Lafferty,et al.  Dynamic topic models , 2006, ICML.

[45]  D. Donoho,et al.  Hessian Eigenmaps : new locally linear embedding techniques for high-dimensional data , 2003 .

[46]  Israel Cohen,et al.  Single-Channel Transient Interference Suppression With Diffusion Maps , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[47]  Dan Cornford,et al.  Variational Inference for Diffusion Processes , 2007, NIPS.

[48]  G. Vachtsevanos,et al.  Epileptic Seizures May Begin Hours in Advance of Clinical Onset A Report of Five Patients , 2001, Neuron.

[49]  R. Coifman,et al.  Non-linear independent component analysis with diffusion maps , 2008 .

[50]  A. Krener,et al.  Nonlinear controllability and observability , 1977 .

[51]  Klaus Lehnertz,et al.  Testing the null hypothesis of the nonexistence of a preseizure state. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.