Learning graphical models for stationary time series

Probabilistic graphical models can be extended to time series by considering probabilistic dependencies between entire time series. For stationary Gaussian time series, the graphical model semantics can be expressed naturally in the frequency domain, leading to interesting families of structured time series models that are complementary to families defined in the time domain. In this paper, we present an algorithm to learn the structure from data for directed graphical models for stationary Gaussian time series. We describe an algorithm for efficient forecasting for stationary Gaussian time series whose spectral densities factorize in a graphical model. We also explore the relationships between graphical model structure and sparsity, comparing and contrasting the notions of sparsity in the time domain and the frequency domain. Finally, we show how to make use of Mercer kernels in this setting, allowing our ideas to be extended to nonlinear models.

[1]  N. Wiener,et al.  The prediction theory of multivariate stochastic processes , 1957 .

[2]  N. Wiener,et al.  The prediction theory of multivariate stochastic processes, II , 1958 .

[3]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[4]  E. J. Hannan,et al.  Multiple time series , 1970 .

[5]  C. Chatfield,et al.  Fourier Analysis of Time Series: An Introduction , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[6]  P. Papantoni-Kazakos,et al.  Spectral distance measures between Gaussian processes , 1980, ICASSP.

[7]  David R. Brillinger,et al.  Time Series: Data Analysis and Theory. , 1982 .

[8]  Gene H. Golub,et al.  Matrix computations , 1983 .

[9]  E. Parzen.  Proceedings of Time Series Analysis of Irregularly Observed Data Held at College Station, Texas on February 10-13, 1983, , 1983 .

[10]  Emanuel Parzen.  Time Series Analysis of Irregularly Observed Data , 1984 .

[11]  T. Speed,et al.  Gaussian Markov Distributions over Finite Graphs , 1986 .

[12]  Keiji Kanazawa,et al.  A model for reasoning about persistence and causation , 1989 .

[13]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[14]  Michael C. Horsch,et al.  Dynamic Bayesian networks , 1990 .

[15]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[16]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[17]  R. Chan.  Toeplitz Preconditioners for Toeplitz Systems with Nonnegative Generating Functions , 1991 .

[18]  J. Nagy,et al.  FFT-based preconditioners for Toeplitz-block least squares problems , 1993 .

[19]  David Heckerman,et al.  Learning Gaussian Networks , 1994, UAI.

[20]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[21]  David Maxwell Chickering,et al.  Learning Bayesian Networks is NP-Complete , 2016, AISTATS.

[22]  Raymond H. Chan,et al.  Conjugate Gradient Methods for Toeplitz Systems , 1996, SIAM Rev..

[23]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[24]  Steffen L. Lauritzen,et al.  Graphical models in R , 1996 .

[25]  D. Brillinger.  Remarks Concerning Graphical Models for Time Series and Point Processes , 1996 .

[26]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[27]  R. Dahlhaus.  Graphical interaction models for multivariate time series1 , 2000 .

[28]  D. Brillinger.  Time series - data analysis and theory , 1981, Classics in applied mathematics.

[29]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[30]  Bernhard Schölkopf,et al.  Kernel Dependency Estimation , 2002, NIPS.

[31]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[32]  Michael I. Jordan,et al.  Learning Graphical Models with Mercer Kernels , 2002, NIPS.

[33]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[34]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[35]  Paolo Giudici,et al.  Improving Markov Chain Monte Carlo Model Search for Data Mining , 2004, Machine Learning.

[36]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.