Variational Probabilistic Inference and the QMR-DT Network

We describe a variational approximation method for efficient inference in large-scale probabilistic models. Variational methods are deterministic procedures that provide approximations to marginal and conditional probabilities of interest. They provide alternatives to approximate inference methods based on stochastic sampling or search. We describe a variational approach to the problem of diagnostic inference in the "Quick Medical Reference" (QMR) network. The QMR network is a large-scale probabilistic graphical model built on statistical and expert knowledge. Exact probabilistic inference is infeasible in this model for all but a small set of cases. We evaluate our variational inference algorithm on a large set of diagnostic test cases, comparing the algorithm to a state-of-the-art stochastic sampling method.

[1]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[2]  Donald B. Rubin,et al.  Max-imum Likelihood from Incomplete Data , 1972 .

[3]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[4]  Gregory F. Cooper,et al.  NESTOR: A Computer-Based Medical Diagnostic Aid That Integrates Causal and Probabilistic Knowledge. , 1984 .

[5]  F E Masarie,et al.  Quick medical reference (QMR) for diagnostic assistance. , 1986, M.D.Computing.

[6]  Yun Peng,et al.  A Probabilistic Causal Model for Diagnostic Problem Solving Part II: Diagnostic Strategy , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[7]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[8]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[9]  David Heckerman,et al.  A Tractable Inference Algorithm for Diagnosing Multiple Diseases , 2013, UAI.

[10]  Ross D. Shachter,et al.  Simulation Approaches to General Probabilistic Inference on Belief Networks , 2013, UAI.

[11]  Eric Horvitz,et al.  Bounded Conditioning: Flexible Inference for Decisions under Scarce Resources , 2013, UAI 1989.

[12]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[13]  Kuo-Chu Chang,et al.  Weighing and Integrating Evidence for Stochastic Simulation in Bayesian Networks , 2013, UAI.

[14]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[15]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[16]  D. Heckerman,et al.  Probabilistic diagnosis using a reformulation of the INTERNIST-1/QMR knowledge base. II. Evaluation of diagnostic performance. , 1991, Methods of information in medicine.

[17]  Max Henrion,et al.  Search-Based Methods to Bound Diagnostic Probabilities in Very Large Belief Nets , 1991, UAI.

[18]  Gregory F. Cooper,et al.  An Empirical Analysis of Likelihood-Weighting Simulation on a Large, Multiply-Connected Belief Network , 1991, Computers and biomedical research, an international journal.

[19]  D. Heckerman,et al.  ,81. Introduction , 2022 .

[20]  Prakash P. Shenoy,et al.  Valuation-Based Systems for Bayesian Decision Analysis , 1992, Oper. Res..

[21]  Eric Horvitz,et al.  Reformulating Inference Problems Through Selective Conditioning , 1992, UAI.

[22]  Bruce D'Ambrosio,et al.  Incremental Probabilistic Inference , 1993, UAI.

[23]  Michael Luby,et al.  Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard , 1993, Artif. Intell..

[24]  Eric Horvitz,et al.  A Bayesian analysis of simulation algorithms for inference in belief networks , 1993, Networks.

[25]  Denise Draper,et al.  Localized Partial Evaluation of Belief Networks , 1994, UAI.

[26]  Bruce D'Ambrosio,et al.  Symbolic Probabilistic Inference in Large BN20 Networks , 1994, UAI.

[27]  Uffe Kjærulff,et al.  Blocking Gibbs sampling in very large probabilistic expert systems , 1995, Int. J. Hum. Comput. Stud..

[28]  Finn Verner Jensen,et al.  Introduction to Bayesian Networks , 2008, Innovations in Bayesian Networks.

[29]  Michael I. Jordan,et al.  Recursive Algorithms for Approximating Probabilities in Graphical Models , 1996, NIPS.

[30]  Rina Dechter,et al.  Bucket elimination: A unifying framework for probabilistic inference , 1996, UAI.

[31]  Rina Dechter,et al.  Mini-Buckets: A General Scheme for Generating Approximations in Automated Reasoning , 1997, IJCAI.

[32]  Michael I. Jordan,et al.  Variational methods for inference and estimation in graphical models , 1997 .

[33]  David Poole,et al.  Probabilistic Partial Evaluation: Exploiting Rule Structure in Probabilistic Inference , 1997, IJCAI.

[34]  ModelsbyTommi S. Jaakkola.  Variational Methods for Inference and Estimation inGraphical , 1997 .

[35]  David J. C. Mackay,et al.  Introduction to Monte Carlo Methods , 1998, Learning in Graphical Models.

[36]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.