Overlapping decomposition for causal graphical modeling

Causal graphical models are developed to detect the dependence relationships between random variables and provide intuitive explanations for the relationships in complex systems. Most of existing work focuses on learning a single graphical model for all the variables. However, a single graphical model cannot accurately characterize the complicated causal relationships for a relatively large graph. In this paper, we propose the problem of estimating an overlapping decomposition for Gaussian graphical models of a large scale to generate overlapping sub-graphical models. Specifically, we formulate an objective function for the overlapping decomposition problem and propose an approximate algorithm for it. A key theory of the algorithm is that the problem of solving a κ+1 node graphical model can be reduced to the problem of solving a one-step regularization based on a solved κ node graphical model. Based on this theory, a greedy expansion algorithm is proposed to generate the overlapping subgraphs. We evaluate the effectiveness of our model on both synthetic datasets and real traffic dataset, and the experimental results show the superiority of our method.

[1]  Pradeep Ravikumar,et al.  BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables , 2013, NIPS.

[2]  Mark W. Schmidt,et al.  Fast Optimization Methods for L1 Regularization: A Comparative Study and Two New Approaches , 2007, ECML.

[3]  Trevor J. Hastie,et al.  Exact Covariance Thresholding into Connected Components for Large-Scale Graphical Lasso , 2011, J. Mach. Learn. Res..

[4]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[5]  Yan Liu,et al.  Temporal causal modeling with graphical granger methods , 2007, KDD '07.

[6]  Martin A. Nowak,et al.  Inferring Cellular Networks Using Probabilistic Graphical Models , 2004 .

[7]  Le Song,et al.  Time-Varying Dynamic Bayesian Networks , 2009, NIPS.

[8]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[9]  Yan Liu,et al.  Spatial-temporal causal modeling for climate change attribution , 2009, KDD.

[10]  Pradeep Ravikumar,et al.  A Divide-and-Conquer Method for Sparse Inverse Covariance Estimation , 2012, NIPS.

[11]  J. N. R. Jeffers,et al.  Graphical Models in Applied Multivariate Statistics. , 1990 .

[12]  Pei Wang,et al.  Partial Correlation Estimation by Joint Sparse Regression Models , 2008, Journal of the American Statistical Association.

[13]  Ruoming Jin,et al.  A Sparsification Approach for Temporal Graphical Model Decomposition , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[14]  Pradeep Ravikumar,et al.  Sparse inverse covariance matrix estimation using quadratic approximation , 2011, MLSLP.

[15]  Yan Liu,et al.  Learning dynamic temporal graphs for oil-production equipment monitoring system , 2009, KDD.

[16]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[17]  Eric P. Xing,et al.  On Time Varying Undirected Graphs , 2011, AISTATS.

[18]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[19]  Shang-Hua Teng,et al.  A Local Clustering Algorithm for Massive Graphs and Its Application to Nearly Linear Time Graph Partitioning , 2008, SIAM J. Comput..

[20]  Larry A. Wasserman,et al.  Time varying undirected graphs , 2008, Machine Learning.

[21]  E. Levina,et al.  Joint estimation of multiple graphical models. , 2011, Biometrika.

[22]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[23]  Le Song,et al.  Sparsistent Learning of Varying-coefficient Models with Structural Changes , 2009, NIPS.

[24]  Jorge Nocedal,et al.  Newton-Like Methods for Sparse Inverse Covariance Estimation , 2012, NIPS.

[25]  Xi Chen,et al.  Learning Spatial-Temporal Varying Graphs with Applications to Climate Data Analysis , 2010, AAAI.

[26]  Dimitris Samaras,et al.  Multi-Task Learning of Gaussian Graphical Models , 2010, ICML.

[27]  Fergal Reid,et al.  Detecting highly overlapping community structure by greedy clique expansion , 2010, KDD 2010.

[28]  Steffen L. Lauritzen,et al.  Graphical models in R , 1996 .

[29]  Yan Liu,et al.  Temporal Graphical Models for Cross-Species Gene Regulatory Network Discovery , 2011, J. Bioinform. Comput. Biol..

[30]  Le Song,et al.  Estimating time-varying networks , 2008, ISMB 2008.

[31]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .