Overlapping Decomposition for Gaussian Graphical Modeling

Correlation based graphical models are developed to detect the dependence relationships among random variables and provide intuitive explanations for these relationships in complex systems. Most of the existing works focus on learning a single correlation based graphical model for all the random variables. However, it is difficult to understand and interpret the massive dependencies of the variables learned from a single graphical model at a global level especially when the graph is large. In order to provide a clearer understanding for the dependence relationships among a large number of random variables, in this paper, we propose the problem of estimating an overlapping decomposition for the Gaussian graphical model of a large scale to generate overlapping sub-graphical models, where strong and meaningful correlations remain in each subgraph with a small scale. Specifically, we propose a greedy algorithm to achieve the overlapping decomposition for the Gaussian graphical model. A key technique of the algorithm is that the problem of solving a (k + 1)-node Gaussian graphical model can be approximately reduced to the problem of solving a one-step vector regularization problem based on a solved k-node Gaussian graphical model with theoretical guarantee. Based on this technique, a greedy expansion algorithm is proposed to generate the overlapping subgraphs. Moreover, we extend the proposed method to deal with dynamic graphs where the dependence relationships among random variables vary with the time. We evaluate the proposed methods on synthetic dataset and a real-life traffic dataset, and the experimental results show the superiority of the proposed methods.

[1]  Jorge Nocedal,et al.  Newton-Like Methods for Sparse Inverse Covariance Estimation , 2012, NIPS.

[2]  Xi Chen,et al.  Learning Spatial-Temporal Varying Graphs with Applications to Climate Data Analysis , 2010, AAAI.

[3]  Pradeep Ravikumar,et al.  Sparse inverse covariance matrix estimation using quadratic approximation , 2011, MLSLP.

[4]  Yan Liu,et al.  Learning dynamic temporal graphs for oil-production equipment monitoring system , 2009, KDD.

[5]  Le Song,et al.  Sparsistent Learning of Varying-coefficient Models with Structural Changes , 2009, NIPS.

[6]  Trevor J. Hastie,et al.  Exact Covariance Thresholding into Connected Components for Large-Scale Graphical Lasso , 2011, J. Mach. Learn. Res..

[7]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[8]  Dimitris Samaras,et al.  Multi-Task Learning of Gaussian Graphical Models , 2010, ICML.

[9]  Fergal Reid,et al.  Detecting highly overlapping community structure by greedy clique expansion , 2010, KDD 2010.

[10]  Yan Liu,et al.  Temporal Graphical Models for Cross-Species Gene Regulatory Network Discovery , 2011, J. Bioinform. Comput. Biol..

[11]  Le Song,et al.  Estimating time-varying networks , 2008, ISMB 2008.

[12]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[13]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[14]  Yan Liu,et al.  Temporal causal modeling with graphical granger methods , 2007, KDD '07.

[15]  Pradeep Ravikumar,et al.  BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables , 2013, NIPS.

[16]  Yan Liu,et al.  Spatial-temporal causal modeling for climate change attribution , 2009, KDD.

[17]  Le Song,et al.  Time-Varying Dynamic Bayesian Networks , 2009, NIPS.

[18]  Pei Wang,et al.  Partial Correlation Estimation by Joint Sparse Regression Models , 2008, Journal of the American Statistical Association.

[19]  Ruoming Jin,et al.  A Sparsification Approach for Temporal Graphical Model Decomposition , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[20]  Mark W. Schmidt,et al.  Fast Optimization Methods for L1 Regularization: A Comparative Study and Two New Approaches , 2007, ECML.

[21]  Jessika Weiss,et al.  Graphical Models In Applied Multivariate Statistics , 2016 .

[22]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[23]  Pradeep Ravikumar,et al.  A Divide-and-Conquer Method for Sparse Inverse Covariance Estimation , 2012, NIPS.

[24]  Nir Friedman,et al.  Inferring Cellular Networks Using Probabilistic Graphical Models , 2004, Science.

[25]  Shang-Hua Teng,et al.  A Local Clustering Algorithm for Massive Graphs and Its Application to Nearly Linear Time Graph Partitioning , 2008, SIAM J. Comput..

[26]  Larry A. Wasserman,et al.  Time varying undirected graphs , 2008, Machine Learning.

[27]  E. Levina,et al.  Joint estimation of multiple graphical models. , 2011, Biometrika.

[28]  Eran Treister,et al.  A Block-Coordinate Descent Approach for Large-scale Sparse Inverse Covariance Estimation , 2014, NIPS.

[29]  Lei Han,et al.  Overlapping decomposition for causal graphical modeling , 2012, KDD.

[30]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[31]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.