Multi-Domain Sampling With Applications to Structural Inference of Bayesian Networks

When a posterior distribution has multiple modes, unconditional expectations, such as the posterior mean, may not offer informative summaries of the distribution. Motivated by this problem, we propose to decompose the sample space of a multimodal distribution into domains of attraction of local modes. Domain-based representations are defined to summarize the probability masses of and conditional expectations on domains of attraction, which are much more informative than the mean and other unconditional expectations. A computational method, the multi-domain sampler, is developed to construct domain-based representations for an arbitrary multimodal distribution. The multi-domain sampler is applied to structural learning of protein-signaling networks from high-throughput single-cell data, where a signaling network is modeled as a causal Bayesian network. Not only does our method provide a detailed landscape of the posterior distribution but also improves the accuracy and the predictive power of estimated networks. This article has supplementary material online.

[1]  J. Robins.  A graphical approach to the identification and estimation of causal parameters in mortality studies with sustained exposure periods. , 1987, Journal of chronic diseases.

[2]  L. Darrell Whitley,et al.  Serial and Parallel Genetic Algorithms as Function Optimizers , 1993, ICGA.

[3]  J. Pearl.  Graphical Models, Causality, and Intervention , 2011 .

[4]  D. Landau,et al.  Efficient, multiple-range random walk algorithm to calculate the density of states. , 2000, Physical review letters.

[5]  Jun S. Liu,et al.  The Wang-Landau algorithm in general state spaces: Applications and convergence analysis , 2010 .

[6]  C. Andrieu,et al.  On the ergodicity properties of some adaptive MCMC algorithms , 2006, math/0610317.

[7]  David J Wales,et al.  Potential energy and free energy landscapes. , 2006, The journal of physical chemistry. B.

[8]  J. Robins.  A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect , 1986 .

[9]  J. Doye,et al.  Global Optimization by Basin-Hopping and the Lowest Energy Structures of Lennard-Jones Clusters Containing up to 110 Atoms , 1997, cond-mat/9803344.

[10]  R. Tweedie,et al.  Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms , 1996 .

[11]  A. Shapiro.  Monte Carlo Sampling Methods , 2003 .

[12]  Walter R. Gilks,et al.  Adaptive Direction Sampling , 1994 .

[13]  J. Pearl.  [Bayesian Analysis in Expert Systems]: Comment: Graphical Models, Causality and Intervention , 1993 .

[14]  Wing Hung Wong,et al.  Reconstructing the Energy Landscape of a Distribution from Monte Carlo Samples , 2008, 0901.3999.

[15]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[16]  Jun S. Liu,et al.  The Multiple-Try Method and Local Optimization in Metropolis Sampling , 2000 .

[17]  Donald B. Rubin,et al.  Bayesian Inference for Causal Effects: The Role of Randomization , 1978 .

[18]  W. Kolch,et al.  Raf-1 and B-Raf promote protein kinase C θ interaction with BAD , 2007 .

[19]  F. Liang.  On the use of stochastic approximation Monte Carlo for Monte Carlo integration , 2009 .

[20]  M. Karplus,et al.  The topology of multidimensional potential energy surfaces: Theory and application to peptide structure and kinetics , 1997 .

[21]  R. Carroll,et al.  Stochastic Approximation in Monte Carlo Computation , 2007 .

[22]  W. Wong,et al.  Learning Causal Bayesian Network Structures From Experimental Data , 2008 .

[23]  F. Liang.  A Generalized Wang–Landau Algorithm for Monte Carlo Computation , 2005 .

[24]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[25]  Gregory F. Cooper,et al.  Causal Discovery from a Mixture of Experimental and Observational Data , 1999, UAI.

[26]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[27]  Terence P. Speed,et al.  Introductory Remarks on Neyman (1923) , 1990 .

[28]  W. Michael Conklin,et al.  Monte Carlo Methods in Bayesian Computation , 2001, Technometrics.

[29]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Eric Moulines,et al.  Stability of Stochastic Approximation under Verifiable Conditions , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[31]  Jia Su,et al.  Wang-Landau Algorithm , 2008 .

[32]  K. Sachs,et al.  Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data , 2005, Science.

[33]  Judea Pearl,et al.  Causal Inference , 2010 .

[34]  Faming Liang,et al.  Author's Personal Copy Computational Statistics and Data Analysis Learning Bayesian Networks for Discrete Data , 2022 .

[35]  P. Spirtes,et al.  Causation, Prediction, and Search, 2nd Edition , 2001 .

[36]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[37]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[38]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[39]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[40]  J. Pearl.  The Logic of Counterfactuals in Causal Inference , 2000 .

[41]  Judea Pearl.  Causal Inference Without Counterfactuals: Comment , 2000 .

[42]  Qing Zhou.  Random walk over basins of attraction to construct ising energy landscapes. , 2011, Physical review letters.

[43]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[44]  H. Scheraga,et al.  Monte Carlo-minimization approach to the multiple-minima problem in protein folding. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[45]  P. Holland.  CAUSAL INFERENCE, PATH ANALYSIS AND RECURSIVE STRUCTURAL EQUATIONS MODELS , 1988 .