Time Varying Dynamic Bayesian Network for Nonstationary Events Modeling and Online Inference

This paper presents a novel time varying dynamic Bayesian network (TVDBN) model for the analysis of nonstationary sequences which are of interest in many fields. The changing network structure and parameter in TVDBN are treated as random processes whose values at each time epoch determine a stationary DBN model; this DBN model is then used to specify the distribution of data sequence at the time epoch. Under such a hierarchical formulation, the changing state of network can be incorporated into the Bayesian framework straightforwardly. The network state is assumed to transit smoothly in the joint space of numerical parameter and graphical topology so that we can achieve robust online network learning even without abundant observations. Particle filtering is employed to dynamically update current network state as well as infer hidden data values. We implement our time varying model for data sequences of multinomial and Gaussian distributions, while the general model framework can be used for any other distribution. Simulations on synthetic data and evaluations on video sequences both demonstrate that the proposed TVDBN is effective in modeling nonstationary sequences. Comprehensive comparisons have been made against existing nonstationary models, and our proposed model is shown to be the top performer.

[1]  Le Song,et al.  Sparsistent Learning of Varying-coefficient Models with Structural Changes , 2009, NIPS.

[2]  P. Priouret,et al.  On recursive estimation for time varying autoregressive processes , 2005, math/0603047.

[3]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[4]  Vladimir Pavlovic,et al.  Learning Switching Linear Models of Human Motion , 2000, NIPS.

[5]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[6]  Le Song,et al.  Estimating time-varying networks , 2008, ISMB 2008.

[7]  Patrick Pérez,et al.  Color-Based Probabilistic Tracking , 2002, ECCV.

[8]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[9]  Michael I. Jordan,et al.  Nonparametric Bayesian Learning of Switching Linear Dynamical Systems , 2008, NIPS.

[10]  Carlo S. Regazzoni,et al.  Interaction Analysis with a Bayesian Trajectory Model , 2010, IEEE Intelligent Systems.

[11]  Christophe Andrieu,et al.  Bayesian curve fitting using MCMC with applications to signal segmentation , 2002, IEEE Trans. Signal Process..

[12]  Le Song,et al.  Dynamic mixed membership blockmodel for evolving networks , 2009, ICML '09.

[13]  Michael A. West,et al.  Bayesian time-varying autoregressions: Theory, methods and applications , 2000 .

[14]  Alexander J. Hartemink,et al.  Non-stationary dynamic Bayesian networks , 2008, NIPS.

[15]  Le Song,et al.  Time-Varying Dynamic Bayesian Networks , 2009, NIPS.

[16]  Geoffrey E. Hinton,et al.  Variational Learning for Switching State-Space Models , 2000, Neural Computation.

[17]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[18]  Paul Fearnhead,et al.  Exact and efficient Bayesian inference for multiple changepoint problems , 2006, Stat. Comput..

[19]  Marco Grzegorczyk,et al.  Non-stationary continuous dynamic Bayesian networks , 2009, NIPS.

[20]  P. Green Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[21]  Alfred O. Hero,et al.  Inferring Time-Varying Network Topologies from Gene Expression Data , 2007, EURASIP J. Bioinform. Syst. Biol..

[22]  David Heckerman,et al.  Learning Gaussian Networks , 1994, UAI.

[23]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[24]  Thomas D. Nielsen,et al.  Adapting Bayes Network Structures to Non-stationary Domains , 2006, Probabilistic Graphical Models.

[25]  Kevin P. Murphy,et al.  Modeling changing dependency structure in multivariate time series , 2007, ICML '07.

[26]  Diego Sona,et al.  Hierarchical Dirichlet model for document classification , 2005, ICML.

[27]  Eli Upfal,et al.  Stochastic models for the Web graph , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[28]  Narsingh Deo,et al.  Preferential deletion in dynamic models of web-like networks , 2007, Inf. Process. Lett..

[29]  Kevin P. Murphy,et al.  Learning the Structure of Dynamic Probabilistic Networks , 1998, UAI.

[30]  Padhraic Smyth,et al.  Adaptive event detection with time-varying poisson processes , 2006, KDD '06.

[31]  Aysin Ertüzün,et al.  Modeling non-Gaussian time-varying vector autoregressive processes by particle filtering , 2010, Multidimens. Syst. Signal Process..

[32]  James M. Rehg,et al.  Learning and Inferring Motion Patterns using Parametric Segmental Switching Linear Dynamic Systems , 2008, International Journal of Computer Vision.

[33]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[34]  Michael Isard,et al.  Learning Multi-Class Dynamics , 1998, NIPS.

[35]  Christophe Andrieu,et al.  Efficient particle filtering for jump Markov systems. Application to time-varying autoregressions , 2003, IEEE Trans. Signal Process..

[36]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[37]  Lingfeng Shi,et al.  Adaptive learning of dynamic Bayesian networks with changing structures by detecting geometric structures of time series , 2008, Knowledge and Information Systems.

[38]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[39]  Rama Chellappa,et al.  Locally time-invariant models of human activities using trajectories on the grassmannian , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Michael A. West,et al.  Evaluation and Comparison of EEG Traces: Latent Structure in Nonstationary Time Series , 1999 .

[41]  Allan Tucker,et al.  A Bayesian network approach to explaining time series with changing structure , 2004, Intell. Data Anal..

[42]  Wenjie Fu,et al.  Recovering temporally rewiring networks: a model-based approach , 2007, ICML '07.

[43]  Lorne D. Johnson,et al.  Maximizing equity market sector predictability in a Bayesian time-varying parameter model , 2008, Comput. Stat. Data Anal..

[44]  Robert T. Collins,et al.  An Open Source Tracking Testbed and Evaluation Web Site , 2005 .