Exact estimation of multiple directed acyclic graphs

This paper considers structure learning for multiple related directed acyclic graph (DAG) models. Building on recent developments in exact estimation of DAGs using integer linear programming (ILP), we present an ILP approach for joint estimation over multiple DAGs. Unlike previous work, we do not require that the vertices in each DAG share a common ordering. Furthermore, we allow for (potentially unknown) dependency structure between the DAGs. Results are presented on both simulated data and fMRI data obtained from multiple subjects.

[1]  B. Maher.  ENCODE: The human encyclopaedia , 2012, Nature.

[2]  K. Sachs,et al.  Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data , 2005, Science.

[3]  J. Peters,et al.  Identifiability of Gaussian structural equation models with equal error variances , 2012, 1205.2536.

[4]  James Cussens,et al.  Bayesian network learning with cutting planes , 2011, UAI.

[5]  Brandon M. Malone,et al.  Predicting the Hardness of Learning Bayesian Networks , 2014, AAAI.

[6]  Tobias Achterberg,et al.  SCIP: solving constraint integer programs , 2009, Math. Program. Comput..

[7]  Takashi Washio,et al.  Learning a common substructure of multiple graphical Gaussian models , 2012, Neural Networks.

[8]  A. Dawid,et al.  Hyper Markov Laws in the Statistical Analysis of Decomposable Graphical Models , 1993 .

[9]  R. Scheines,et al.  Bayesian estimation and testing of structural equation models , 1999 .

[10]  W. Wong,et al.  Learning Causal Bayesian Network Structures From Experimental Data , 2008 .

[11]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[12]  Bo Thiesson,et al.  Learning Mixtures of Bayesian Networks , 1997, UAI 1997.

[13]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[14]  Peter Bühlmann,et al.  Structural Intervention Distance for Evaluating Causal Graphs , 2015, Neural Computation.

[15]  Joe W. Gray,et al.  Joint estimation of multiple networks from time course data , 2013 .

[16]  Martin J. McKeown,et al.  Dynamic Bayesian network modeling of fMRI: A comparison of group-analysis methods , 2008, NeuroImage.

[17]  Jirí Vomlel,et al.  A geometric view on learning Bayesian network structures , 2010, Int. J. Approx. Reason..

[18]  Michael A. West,et al.  Bayesian Forecasting and Dynamic Models (2nd edn) , 1997, J. Oper. Res. Soc..

[19]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[20]  Thomas E. Nichols,et al.  Searching Multiregression Dynamic Models of Resting-State fMRI Networks Using Integer Programming , 2015, 1505.06832.

[21]  James Cussens,et al.  Maximum likelihood pedigree reconstruction using integer programming , 2010, WCB@ICLP.

[22]  G. Nemhauser,et al.  Integer Programming , 2020 .

[23]  David Maxwell Chickering,et al.  Optimal Structure Identification With Greedy Search , 2002, J. Mach. Learn. Res..

[24]  Laurence A. Wolsey,et al.  Integer and Combinatorial Optimization , 1988, Wiley interscience series in discrete mathematics and optimization.

[25]  A. Philip Dawid,et al.  Beware of the DAG! , 2008, NIPS Causality: Objectives and Assessment.

[26]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[27]  Christopher A. Penfold,et al.  Nonparametric Bayesian inference for perturbed and orthologous gene regulatory networks , 2012, Bioinform..

[28]  James Cussens,et al.  Improved maximum likelihood reconstruction of complex multi-generational pedigrees. , 2014, Theoretical population biology.

[29]  F. Markowetz,et al.  The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups , 2012, Nature.

[30]  Tatsuro Kaminaga,et al.  Interindividual uniformity and variety of the “Writing center”: A functional MRI study , 2006, NeuroImage.

[31]  Terence P. Speed,et al.  Bayesian Inference of Signaling Network Topology in a Cancer Cell Line , 2012, Bioinform..

[32]  Chris J. Oates,et al.  Joint Structure Learning of Multiple Non-Exchangeable Networks , 2014, AISTATS.

[33]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[34]  Qiang Ji,et al.  Efficient Structure Learning of Bayesian Networks using Constraints , 2011, J. Mach. Learn. Res..

[35]  David Heckerman,et al.  Knowledge Representation and Inference in Similarity Networks and Bayesian Multinets , 1996, Artif. Intell..

[36]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[37]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[38]  Po-Ling Loh,et al.  Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses , 2012, NIPS.

[39]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[40]  Robert G Cowell,et al.  Efficient maximum likelihood pedigree reconstruction. , 2009, Theoretical population biology.

[41]  S. Miyano,et al.  Finding Optimal Bayesian Network Given a Super-Structure , 2008 .

[42]  Casper J. Albers,et al.  Intervention and Causality: Forecasting Traffic Flows Using a Dynamic Bayesian Network , 2009 .

[43]  Peter Müller,et al.  Detecting differential patterns of interaction in molecular pathways. , 2015, Biostatistics.

[44]  Ted K. Ralphs,et al.  Integer and Combinatorial Optimization , 2013 .

[45]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[46]  Guido Consonni,et al.  Moment priors for Bayesian model choice with applications to directed acyclic graphs , 2011 .

[47]  Bernhard Schölkopf,et al.  Identifiability of Causal Graphs using Functional Models , 2011, UAI.

[48]  Chris J. Oates,et al.  Toward a Multisubject Analysis of Neural Connectivity , 2014, Neural Computation.

[49]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[50]  Christophe Ambroise,et al.  Inferring multiple graphical structures , 2009, Stat. Comput..

[51]  Catriona M. Queen,et al.  Multiregression dynamic models , 1993 .

[52]  Clive G Bowsher.  STOCHASTIC KINETIC MODELS: DYNAMIC INDEPENDENCE, MODULARITY AND GRAPHS. , 2010, Annals of statistics.

[53]  Terran Lane,et al.  Leveraging Domain Knowledge in Multitask Bayesian Network Structure Learning , 2012, AAAI.

[54]  Tom Heskes,et al.  A Bayesian Approach to Constraint Based Causal Inference , 2012, UAI.

[55]  Sik-Yum Lee.  Structural Equation Modeling: A Bayesian Approach , 2007 .

[56]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[57]  Bo Thiesson,et al.  Learning Mixtures of DAG Models , 1998, UAI.

[58]  Tobias Achterberg,et al.  Constraint integer programming , 2007 .

[59]  Dirk Husmeier,et al.  Gene Regulatory Network Reconstruction by Bayesian Integration of Prior Knowledge and/or Different Experimental Conditions , 2008, J. Bioinform. Comput. Biol..

[60]  Richard E. Neapolitan.  Learning Bayesian Network Structure , 2009 .

[61]  Brandon M. Malone,et al.  Learning Optimal Bounded Treewidth Bayesian Networks via Maximum Satisfiability , 2014, AISTATS.

[62]  Rich Caruana,et al.  Inductive Transfer for Bayesian Network Structure Learning , 2007, ICML Unsupervised and Transfer Learning.

[63]  Karsten M. Borgwardt,et al.  Whole-genome sequencing of multiple Arabidopsis thaliana populations , 2011, Nature Genetics.

[64]  Adam A. Margolin,et al.  The Cancer Cell Line Encyclopedia enables predictive modeling of anticancer drug sensitivity , 2012, Nature.

[65]  Changhe Yuan,et al.  Learning Optimal Bayesian Networks: A Shortest Path Perspective , 2013, J. Artif. Intell. Res..

[66]  Sach Mukherjee,et al.  Network clustering: probing biological heterogeneity by sparse graphical models , 2011, Bioinform..

[67]  Bin Yu,et al.  Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs , 2012, ArXiv.

[68]  Karl J. Friston.  Functional and Effective Connectivity: A Review , 2011, Brain Connect..

[69]  Jukka Corander,et al.  Labeled directed acyclic graphs: a generalization of context-specific independence in directed graphical models , 2013, Data Mining and Knowledge Discovery.

[70]  Simon E. F. Spencer,et al.  Quantifying the multi-scale performance of network inference algorithms , 2014, Statistical applications in genetics and molecular biology.

[71]  Sik-Yum Lee,et al.  Structural equation modelling: A Bayesian approach. , 2007 .

[72]  Luis Enrique Sucar,et al.  Inductive transfer for learning Bayesian networks , 2010, Machine Learning.

[73]  T. Ideker,et al.  Differential network biology , 2012, Molecular systems biology.

[74]  Ruth Nussinov,et al.  Structure and dynamics of molecular networks: A novel paradigm of drug discovery. A comprehensive review , 2012, Pharmacology & therapeutics.

[75]  James G. Scott,et al.  Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem , 2010, 1011.2333.

[76]  Mark Girolami,et al.  The Controlled Thermodynamic Integral for Bayesian Model Comparison , 2014, 1404.5053.

[77]  Milan Studený,et al.  On Polyhedral Approximations of Polytopes for Learning Bayesian Networks , 2013 .

[78]  Dario L Ringach,et al.  Spontaneous and driven cortical activity: implications for computation , 2009, Current Opinion in Neurobiology.

[79]  Sach Mukherjee,et al.  Network inference using informative priors , 2008, Proceedings of the National Academy of Sciences.

[80]  Luis M. de Campos,et al.  Searching for Bayesian Network Structures in the Space of Restricted Acyclic Partially Directed Graphs , 2011, J. Artif. Intell. Res..

[81]  J. Peters,et al.  Structural Intervention Distance (SID) for Evaluating Causal Graphs , 2013, 1306.1043.

[82]  Peter Wonka,et al.  Fused Multiple Graphical Lasso , 2012, SIAM J. Optim..

[83]  Tomi Silander,et al.  A Simple Approach for Finding the Globally Optimal Bayesian Network Structure , 2006, UAI.

[84]  A. Mahajan,et al.  Presolving Mixed–Integer Linear Programs , 2011 .

[85]  M. Maathuis,et al.  Estimating high-dimensional intervention effects from observational data , 2008, 0810.4214.

[86]  M. Girolami,et al.  Inferring Signaling Pathway Topologies from Multiple Perturbation Measurements of Specific Biochemical Species , 2010, Science Signaling.

[87]  Karl J. Friston,et al.  Effective Connectivity and Intersubject Variability: Using a Multisubject Network to Test Differences and Commonalities , 2002, NeuroImage.

[88]  Terran Lane,et al.  Bayesian Discovery of Multiple Bayesian Networks via Transfer Learning , 2013, 2013 IEEE 13th International Conference on Data Mining.

[89]  James Cussens,et al.  Advances in Bayesian Network Learning using Integer Programming , 2013, UAI.

[90]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[91]  Qiang Ji,et al.  Advances in Learning Bayesian Networks of Bounded Treewidth , 2014, NIPS.

[92]  Essa Yacoub,et al.  The WU-Minn Human Connectome Project: An overview , 2013, NeuroImage.

[93]  Jens Lagergren,et al.  Learning Bounded Tree-width Bayesian Networks using Integer Linear Programming , 2014, AISTATS.

[94]  Tommi S. Jaakkola,et al.  Learning Bayesian Network Structure using LP Relaxations , 2010, AISTATS.

[95]  James Cussens,et al.  Maximum Likelihood Pedigree Reconstruction Using Integer Linear Programming , 2013, Genetic epidemiology.