Learning Directed Acyclic Graphs with Hidden Variables via Latent Gaussian Graphical Model Selection

We introduce a new method to estimate the Markov equivalence class of a directed acyclic graph (DAG) in the presence of hidden variables, in settings where the underlying DAG among the observed variables is sparse, and there are a few hidden variables that have a direct effect on many of the observed ones. Building on the so-called low rank plus sparse framework, we suggest a two-stage approach which first removes unwanted variation using latent Gaussian graphical model selection, and then estimates the Markov equivalence class of the underlying DAG by applying GES. This approach is consistent in certain high-dimensional regimes and performs favourably when compared to the state of the art, both in terms of graphical structure recovery and total causal effect estimation.

[1]  D. Madigan,et al.  A characterization of Markov equivalence classes for acyclic digraphs , 1997 .

[2]  Bin Yu,et al.  Model Selection in Gaussian Graphical Models: High-Dimensional Consistency of boldmathell_1-regularized MLE , 2008, NIPS 2008.

[3]  S. Geer,et al.  $\ell_0$-penalized maximum likelihood for sparse directed acyclic graphs , 2012, 1205.5473.

[4]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[5]  Kenneth A. Bollen,et al.  Structural Equations with Latent Variables , 1989 .

[6]  R. Durbin,et al.  Using probabilistic estimation of expression residuals (PEER) to obtain increased power and interpretability of gene expression analyses , 2012, Nature Protocols.

[7]  Diego Colombo,et al.  Order-independent constraint-based causal structure learning , 2012, J. Mach. Learn. Res..

[8]  A. Willsky,et al.  Latent variable graphical model selection via convex optimization , 2010, 1008.1290.

[9]  Jiji Zhang,et al.  On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias , 2008, Artif. Intell..

[10]  James M. Robins,et al.  Sparse Nested Markov models with Log-linear Parameters , 2013, UAI.

[11]  Naftali Harris,et al.  PC algorithm for nonparanormal graphical models , 2013, J. Mach. Learn. Res..

[12]  P. Spirtes,et al.  Ancestral graph Markov models , 2002 .

[13]  D. Tweardy,et al.  Targeting transcription factors for cancer therapy. , 2005, Current pharmaceutical design.

[14]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[15]  Peter Bühlmann,et al.  Estimating High-Dimensional Directed Acyclic Graphs with the PC-Algorithm , 2007, J. Mach. Learn. Res..

[16]  Thomas S. Richardson,et al.  Learning high-dimensional directed acyclic graphs with latent and selection variables , 2011, 1104.5617.

[17]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[18]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[19]  Shiqian Ma,et al.  Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection , 2012, Neural Computation.

[20]  Jung Eun Shim,et al.  TRRUST: a reference database of human transcriptional regulatory interactions , 2015, Scientific Reports.

[21]  P. Bühlmann,et al.  Sparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana , 2004, Genome Biology.

[22]  Judea Pearl,et al.  Equivalence and Synthesis of Causal Models , 1990, UAI.

[23]  M. Maathuis,et al.  Estimating the effect of joint interventions from observational data in sparse high-dimensional settings , 2014, 1407.2451.

[24]  Peter Bühlmann,et al.  Causal Inference Using Graphical Models with the R Package pcalg , 2012 .

[25]  John D. Storey,et al.  Capturing Heterogeneity in Gene Expression Studies by Surrogate Variable Analysis , 2007, PLoS genetics.

[26]  Thomas S. Richardson,et al.  Causal Inference in the Presence of Latent Variables and Selection Bias , 1995, UAI.

[27]  Trevor Hastie,et al.  Statistical Learning with Sparsity: The Lasso and Generalizations , 2015 .

[28]  Marloes H. Maathuis,et al.  Structure Learning in Graphical Modeling , 2016, 1606.02359.

[29]  Alain Hauser,et al.  High-dimensional consistency in score-based and hybrid structure learning , 2015, The Annals of Statistics.

[30]  Myun-Seok Cheon,et al.  Estimation of Directed Acyclic Graphs Through Two-Stage Adaptive Lasso for Gene Network Inference , 2016, Journal of the American Statistical Association.

[31]  Rina Foygel,et al.  Extended Bayesian Information Criteria for Gaussian Graphical Models , 2010, NIPS.

[32]  P. Spirtes,et al.  MARKOV EQUIVALENCE FOR ANCESTRAL GRAPHS , 2009, 0908.3605.