Learning Causal Relations in Multivariate Time Series Data

Many applications naturally involve time series data and the vector autoregression (VAR), and the structural VAR (SVAR) are dominant tools to investigate relations between variables in time series. In the first part of this work, we show that the SVAR method is incapable of identifying contemporaneous causal relations for Gaussian process. In addition, least squares estimators become unreliable when the scales of the problems are large and observations are limited. In the remaining part, we propose an approach to apply Bayesian network learning algorithms to identify SVARs from time series data in order to capture both temporal and contemporaneous causal relations, and avoid high-order statistical tests. The difficulty of applying Bayesian network learning algorithms to time series is that the sizes of the networks corresponding to time series tend to be large, and high-order statistical tests are required by Bayesian network learning algorithms in this case. To overcome the difficulty, we show that the search space of conditioning sets d-separating two vertices should be a subset of the Markov blankets. Based on this fact, we propose an algorithm enabling us to learn Bayesian networks locally, and make the largest order of statistical tests independent of the scales of the problems. Empirical results show that our algorithm outperforms existing methods in terms of both efficiency and accuracy.

[1]  Naoki Abe,et al.  Grouped graphical Granger modeling methods for temporal causal modeling , 2009, KDD.

[2]  Constantin F. Aliferis,et al.  Time and sample efficient discovery of Markov blankets and direct causal relations , 2003, KDD '03.

[3]  K. Hoover,et al.  Searching for the Causal Structure of a Vector Autoregression , 2003 .

[4]  P. Spirtes,et al.  Latent variables, causal models and overidentifying constraints , 1988 .

[5]  Charles E. Heckler,et al.  Applied Multivariate Statistical Analysis , 2005, Technometrics.

[6]  Aapo Hyvärinen,et al.  A Linear Non-Gaussian Acyclic Model for Causal Discovery , 2006, J. Mach. Learn. Res..

[7]  C. Chiarella,et al.  Keynesian Macrodynamics and the Phillips Curve: An Estimated Baseline Macromodel for the U.S. Economy , 2006 .

[8]  Bernhard Schölkopf,et al.  Regression by dependence minimization and its application to causal inference in additive noise models , 2009, ICML '09.

[9]  J. Stock,et al.  The NAIRU, Unemployment and Monetary Policy , 1997 .

[10]  Hai Yang,et al.  ACM Transactions on Intelligent Systems and Technology - Special Section on Urban Computing , 2014 .

[11]  Kevin D. Hoover AUTOMATIC INFERENCE OF THE CONTEMPORANEOUS CAUSAL ORDER OF A SYSTEM OF EQUATIONS , 2005, Econometric Theory.

[12]  M. Eichler Granger causality and path diagrams for multivariate time series , 2007 .

[13]  P. Dhrymes Topics in Advanced Econometrics , 1989 .

[14]  P. Spirtes,et al.  From probability to causality , 1991 .

[15]  Paul Humphreys,et al.  Are There Algorithms That Discover Causal Structure? , 1999, Synthese.

[16]  Allan Leck Jensen,et al.  MIDAS: An Influence Diagram for Management of Mildew in Winter Wheat , 1996, UAI.

[17]  Steffen L. Lauritzen,et al.  Independence properties of directed markov fields , 1990, Networks.

[18]  Daniel Aaronson Price Pass-Through and the Minimum Wage , 2001, Review of Economics and Statistics.

[19]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[20]  R. Dahlhaus Graphical interaction models for multivariate time series1 , 2000 .

[21]  Norman R. Swanson,et al.  Impulse Response Functions Based on a Causal Approach to Residual Orthogonalization in Vector Autoregressions , 1997 .

[22]  Peter Spirtes,et al.  Graphical Models for the Identification of Causal Structures in Multivariate Time Series Models , 2006, JCIS.

[23]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[24]  Y. P. Mehra Unit Labor Costs and the Price Level , 1993 .

[25]  M. Watson,et al.  Are Business Cycles All Alike? , 1984 .

[26]  Christopher Meek,et al.  Strong completeness and faithfulness in Bayesian networks , 1995, UAI.

[27]  A. W. Phillips,et al.  A. W. H. Phillips: Collected Works in Contemporary Perspective: The relation between unemployment and the rate of change of money wage rates in the United Kingdom, 1861-1957 , 2000 .

[28]  C. Granger Testing for causality: a personal viewpoint , 1980 .

[29]  P. Spirtes,et al.  An Algorithm for Fast Recovery of Sparse Causal Graphs , 1991 .

[30]  C. Sims Are forecasting models usable for policy analysis , 1986 .

[31]  Aapo Hyvärinen,et al.  Causal modelling combining instantaneous and lagged effects: an identifiable model based on non-Gaussianity , 2008, ICML '08.

[32]  André Elisseeff,et al.  Using Markov Blankets for Causal Structure Learning , 2008, J. Mach. Learn. Res..

[33]  Lai-Wan Chan,et al.  An efficient causal discovery algorithm for linear models , 2010, KDD.

[34]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[35]  Nancy Cartwright,et al.  What is Wrong with Bayes Nets , 2001 .

[36]  Bernhard Schölkopf,et al.  Detecting the direction of causal time series , 2009, ICML '09.

[37]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[38]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[39]  Judea Pearl,et al.  A Theory of Inferred Causation , 1991, KR.

[40]  Stuart J. Russell,et al.  Adaptive Probabilistic Networks with Hidden Variables , 1997, Machine Learning.

[41]  C. Sims MACROECONOMICS AND REALITY , 1977 .

[42]  R. Shibata,et al.  PARTIAL CORRELATION AND CONDITIONAL CORRELATION AS MEASURES OF CONDITIONAL INDEPENDENCE , 2004 .

[43]  Jiji Zhang,et al.  Adjacency-Faithfulness and Conservative Causal Inference , 2006, UAI.

[44]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[45]  Lai-Wan Chan,et al.  Learning bayesian networks from Markov random fields: An efficient algorithm for linear models , 2012, TKDD.

[46]  R. R. Hocking The analysis and selection of variables in linear regression , 1976 .

[47]  Michael I. Jordan,et al.  Learning graphical models for stationary time series , 2004, IEEE Transactions on Signal Processing.

[48]  Sebastian Thrun,et al.  Bayesian Network Induction via Local Neighborhoods , 1999, NIPS.

[49]  P. Spirtes,et al.  Using Path Diagrams as a Structural Equation Modeling Tool , 1998 .

[50]  Pu Chen,et al.  Learning Causal Relations in Multivariate Time Series Data , 2007 .

[51]  Ole Winther,et al.  Bayesian Sparse Factor Models and DAGs Inference and Comparison , 2009, NIPS.

[52]  David A. Bell,et al.  Learning Bayesian networks from data: An information-theory based approach , 2002, Artif. Intell..

[53]  Stefan Haufe,et al.  Sparse Causal Discovery in Multivariate Time Series , 2008, NIPS Causality: Objectives and Assessment.

[54]  Constantin F. Aliferis,et al.  Algorithms for Large Scale Markov Blanket Discovery , 2003, FLAIRS.

[55]  Khalifa H. Ghali Wage Growth and the Inflation Process: A Multivariate Cointegration Analysis , 1999 .

[56]  M. Schweitzer,et al.  Does Wage Inflation Cause Price Inflation? , 2000 .

[57]  André Elisseeff,et al.  A Partial Correlation-Based Algorithm for Causal Structure Discovery with Continuous Variables , 2007, IDA.

[58]  R. Fisher FREQUENCY DISTRIBUTION OF THE VALUES OF THE CORRELATION COEFFIENTS IN SAMPLES FROM AN INDEFINITELY LARGE POPU;ATION , 1915 .

[59]  Richard A. Johnson,et al.  Applied Multivariate Statistical Analysis , 1983 .

[60]  Yan Liu,et al.  Learning Temporal Causal Graphs for Relational Time-Series Analysis , 2010, ICML.