Graph Gamma Process Linear Dynamical Systems

We introduce graph gamma process (GGP) linear dynamical systems to model real-valued multivariate time series. GGP generates S latent states that are shared by K different communities, each of which is characterized by its own pattern of activation probabilities imposed on a S×S directed sparse graph, and allow both S and K to grow without bound. For temporal pattern discovery, the latent representation under the model is used to decompose the time series into a parsimonious set of multivariate sub-sequences generated by formed communities. In each sub-sequence, different data dimensions often share similar temporal patterns but may exhibit distinct magnitudes, and hence allowing the superposition of all sub-sequences to exhibit diverse behaviors at different data dimensions. On both synthetic and real-world time series, the proposed nonparametric Bayesian dynamic models, which are initialized at random, consistently exhibit good predictive performance in comparison to a variety of baseline models, revealing interpretable latent state transition patterns and decomposing the time series into distinctly behaved sub-sequences.

[1]  John P. Cunningham,et al.  A Novel Variational Family for Hidden Nonlinear Markov Models , 2018, ArXiv.

[2]  Feras Saad,et al.  Temporally-Reweighted Chinese Restaurant Process Mixtures for Clustering, Imputing, and Forecasting Multivariate Time Series , 2017, AISTATS.

[3]  T. J. Mitchell,et al.  Bayesian Variable Selection in Linear Regression , 1988 .

[4]  Alex Simpkins,et al.  System Identification: Theory for the User, 2nd Edition (Ljung, L.; 1999) [On the Shelf] , 2012, IEEE Robotics & Automation Magazine.

[5]  Guokun Lai,et al.  Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks , 2017, SIGIR.

[6]  Joydeep Ghosh,et al.  Nonparametric Bayesian sparse graph linear dynamical systems , 2018, AISTATS.

[7]  Lawrence Carin,et al.  Negative Binomial Process Count and Mixture Modeling , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Scott W. Linderman,et al.  Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems , 2017, AISTATS.

[9]  Ryan P. Adams,et al.  Composing graphical models with neural networks for structured representations and fast inference , 2016, NIPS.

[10]  Michael A. West,et al.  Bayesian Forecasting and Dynamic Models (2nd edn) , 1997, J. Oper. Res. Soc..

[11]  J. S. Rao,et al.  Spike and slab variable selection: Frequentist and Bayesian strategies , 2005, math/0505633.

[12]  Kevin Murphy,et al.  Switching Kalman Filters , 1998 .

[13]  Sach Mukherjee,et al.  Penalized estimation in high-dimensional hidden Markov models with state-specific graphical models , 2012, 1208.4989.

[14]  Fernando Pérez-Cruz,et al.  Infinite Factorial Dynamical Model , 2015, NIPS.

[15]  Michael I. Jordan,et al.  Nonparametric Bayesian Learning of Switching Linear Dynamical Systems , 2008, NIPS.

[16]  Alberto Contreras-Cristán,et al.  A Bayesian Nonparametric Approach for Time Series Clustering , 2014 .

[17]  Ulisses Braga-Neto,et al.  Particle filters for partially-observed Boolean dynamical systems , 2018, Autom..

[18]  Benjamin Letham,et al.  Forecasting at Scale , 2018, PeerJ Prepr..

[19]  Guillermo Sapiro,et al.  Non-Parametric Bayesian Dictionary Learning for Sparse Image Representations , 2009, NIPS.

[20]  Karl J. Friston,et al.  Multivariate Autoregressive Modelling of fMRI time series , 2003 .

[21]  Valentin Flunkert,et al.  DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks , 2017, International Journal of Forecasting.

[22]  Richard A. Davis,et al.  Sparse Vector Autoregressive Modeling , 2012, 1207.0520.

[23]  Byron Boots,et al.  Reduced-Rank Hidden Markov Models , 2009, AISTATS.

[24]  Carlos M. Carvalho,et al.  Simulation-based sequential analysis of Markov switching stochastic volatility models , 2007, Comput. Stat. Data Anal..

[25]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[26]  Milos Hauskrecht,et al.  A Regularized Linear Dynamical System Framework for Multivariate Time Series Analysis , 2015, AAAI.

[27]  James P Turley,et al.  Time series analysis as input for clinical predictive modeling: Modeling cardiac arrest in a pediatric ICU , 2011, Theoretical Biology and Medical Modelling.

[28]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[29]  Arnaud Doucet,et al.  Bayesian Inference for Linear Dynamic Models With Dirichlet Process Mixtures , 2007, IEEE Transactions on Signal Processing.

[30]  Justin K. Romberg,et al.  Sparsity penalties in dynamical system estimation , 2011, 2011 45th Annual Conference on Information Sciences and Systems.

[31]  Zoubin Ghahramani,et al.  Learning Nonlinear Dynamical Systems Using an EM Algorithm , 1998, NIPS.

[32]  Emily B. Fox,et al.  Sparse graphs using exchangeable random measures , 2014, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[33]  Matthew J. Johnson,et al.  Bayesian nonparametric hidden semi-Markov models , 2012, J. Mach. Learn. Res..

[34]  Shreyas Sundaram,et al.  Sensor selection for Kalman filtering of linear dynamical systems: Complexity, limitations and greedy algorithms , 2017, Autom..

[35]  Ute Beyer,et al.  Bayesian Forecasting And Dynamic Models , 2016 .

[36]  Michael I. Jordan,et al.  Bayesian Nonparametric Inference of Switching Dynamic Linear Models , 2010, IEEE Transactions on Signal Processing.

[37]  John P. Cunningham,et al.  Linear dynamical neural population models through nonlinear embeddings , 2016, NIPS.

[38]  Geoffrey E. Hinton,et al.  Parameter estimation for linear dynamical systems , 1996 .

[39]  R. Kálmán Mathematical description of linear dynamical systems , 1963 .

[40]  Alessandro Chiuso,et al.  Learning sparse dynamic linear systems using stable spline kernels and exponential hyperpriors , 2010, NIPS.

[41]  Shinsuke Koyama Projection smoothing for continuous and continuous-discrete stochastic dynamic systems , 2018, Signal Process..

[42]  Tengyu Ma,et al.  Gradient Descent Learns Linear Dynamical Systems , 2016, J. Mach. Learn. Res..

[43]  Mingyuan Zhou,et al.  Infinite Edge Partition Models for Overlapping Community Detection and Link Prediction , 2015, AISTATS.

[44]  John W. Fisher,et al.  Nonparametric Object and Parts Modeling With Lie Group Dynamics , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[46]  Neil D. Lawrence,et al.  Sparse Convolved Gaussian Processes for Multi-output Regression , 2008, NIPS.

[47]  David Barber,et al.  Bayesian Time Series Models , 2011 .

[48]  Jianhua Xiao,et al.  Application of time-series autoregressive integrated moving average model in predicting the epidemic situation of newcastle disease , 2010, 2010 World Automation Congress.

[49]  Mónica F. Bugallo,et al.  Tree-Structured Recurrent Switching Linear Dynamical Systems for Multi-Scale Modeling , 2018, ICLR.