Optimal experimental design via Bayesian optimization: active causal structure learning for Gaussian process networks

We study the problem of causal discovery through targeted interventions. Starting from few observational measurements, we follow a Bayesian active learning approach to perform those experiments which, in expectation with respect to the current model, are maximally informative about the underlying causal structure. Unlike previous work, we consider the setting of continuous random variables with non-linear functional relationships, modelled with Gaussian process priors. To address the arising problem of choosing from an uncountable set of possible interventions, we propose to use Bayesian optimisation to efficiently maximise a Monte Carlo estimate of the expected information gain.

[1]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[2]  F. Harary.  New directions in the theory of graphs , 1973 .

[3]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[4]  Jonas Mockus,et al.  On Bayesian Methods for Seeking the Extremum , 1974, Optimization Techniques.

[5]  J. Mockus.  Bayesian Approach to Global Optimization: Theory and Applications , 1989 .

[6]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[7]  David Heckerman,et al.  A Bayesian Approach to Learning Causal Networks , 1995, UAI.

[8]  A. Gopnik.  The Scientist as Child , 1996, Philosophy of Science.

[9]  Gregory F. Cooper,et al.  Causal Discovery from a Mixture of Experimental and Observational Data , 1999, UAI.

[10]  Nir Friedman,et al.  Gaussian Process Networks , 2000, UAI.

[11]  Daphne Koller,et al.  Active Learning for Structure in Bayesian Networks , 2001, IJCAI.

[12]  Jin Tian,et al.  Causal Discovery from Changes , 2001, UAI.

[13]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[14]  Mikko Koivisto,et al.  Exact Bayesian Structure Discovery in Bayesian Networks , 2004, J. Mach. Learn. Res..

[15]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[16]  Frederick Eberhardt,et al.  N-1 Experiments Suffice to Determine the Causal Relations Among N Variables , 2006 .

[17]  Kevin Murphy,et al.  Active Learning of Causal Bayes Net Structure , 2006 .

[18]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[19]  Michael I. Jordan,et al.  Variational inference for Dirichlet process mixtures , 2006 .

[20]  D. Heckerman,et al.  A Bayesian Approach to Causal Discovery , 2006 .

[21]  Kevin P. Murphy,et al.  Exact Bayesian structure learning from uncertain interventions , 2007, AISTATS.

[22]  Frederick Eberhardt,et al.  Almost Optimal Intervention Sets for Causal Discovery , 2008, UAI.

[23]  Bernhard Schölkopf,et al.  Nonlinear causal discovery with additive noise models , 2008, NIPS.

[24]  Yangbo He,et al.  Active Learning of Causal Networks with Intervention Experiments and Optimal Designs , 2008 .

[25]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[26]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[27]  Bernhard Schölkopf,et al.  Information-geometric approach to inferring causal directions , 2012, Artif. Intell..

[28]  Bernhard Schölkopf,et al.  Causal discovery with continuous additive noise models , 2013, J. Mach. Learn. Res..

[29]  Peter Bühlmann,et al.  Two optimal strategies for active learning of causal models from interventional data , 2012, Int. J. Approx. Reason..

[30]  Bernhard Schölkopf,et al.  Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks , 2014, J. Mach. Learn. Res..

[31]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[32]  Bonnie Berger,et al.  Reconstructing Causal Biological Networks through Active Learning , 2016, PloS one.

[33]  Bernhard Schölkopf,et al.  Probabilistic Active Learning of Functions in Structural Causal Models , 2017, ArXiv.

[34]  Olga Vitek,et al.  A Bayesian Active Learning Experimental Design for Inferring Signaling Networks , 2017, RECOMB.

[35]  Tamara Broderick,et al.  Minimal I-MAP MCMC for Scalable Structure Discovery in Causal DAG Models , 2018, ICML.

[36]  Elias Bareinboim,et al.  Budgeted Experiment Design for Causal Structure Learning , 2017, ICML.

[37]  Chandler Squires,et al.  ABCD-Strategy: Budgeted Experimental Design for Targeted Causal Structure Discovery , 2019, AISTATS.