A Lagrangian Relaxation Algorithm for Finding the MAP Configuration in QMR-DT

The quick medical reference decision-theoretic (QMR-DT) network is a large two-layer Bayesian network (BN) [consisting of 571 diseases (ldquofailure sourcesrdquo) and 4075 findings (ldquotest outcomesrdquo)] based on expert and statistical knowledge in internal medicine. The maximum a posteriori (MAP) diagnosis (configuration) based on QMR-DT constitutes an intractable inference problem for all, but a small set of, cases. Consequently, we consider near-optimal algorithms for finding the most likely set of diseases given a set of findings. A computationally efficient algorithm that can handle cases with hundreds of positive findings, i.e., the Lagrangian relaxation algorithm (LRA), is presented. By relaxing the original problem via a set of Lagrange multipliers, the LRA generates an upper bound for the objective function. The near-optimal diagnosis (configuration) is found by minimizing the duality gap via a subgradient method. Numerical experiments show that the LRA is promising in achieving highly accurate diagnosis, and that it is computationally very efficient in solving MAP configuration problems in large and dense two-layer BNs with noisy-OR (BN2O) nodes and containing undirected loops (cycles), such as the QMR-DT network.

[1]  Rina Dechter,et al.  Mini-buckets: A general scheme for bounded inference , 2003, JACM.

[2]  P. Vontobel,et al.  On the Relationship between Linear Programming Decoding and Min-Sum Algorithm Decoding , 2004 .

[3]  Wim Wiegerinck,et al.  Variational Approximations between Mean Field Theory and the Junction Tree Algorithm , 2000, UAI.

[4]  Mark A. Kramer,et al.  GALGO: A Genetic ALGOrithm Decision Support Tool for Complex Uncertain Systems Modeled with Bayesian Belief Networks , 1993, UAI.

[5]  Rina Dechter,et al.  Stochastic local search for Bayesian network , 1999, AISTATS.

[6]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[7]  Dariush Divsalar,et al.  Soft-Output Decoding Algorithms in Iterative Decoding of Turbo Codes , 1996 .

[8]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[9]  Yair Weiss,et al.  Correctness of Local Probability Propagation in Graphical Models with Loops , 2000, Neural Computation.

[10]  J. Beasley,et al.  A genetic algorithm for the set covering problem , 1996 .

[11]  Quaid Morris,et al.  Recognition Networks for Approximate Inference in BN20 Networks , 2001, UAI.

[12]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[13]  Krishna R. Pattipati,et al.  Optimal and near-optimal algorithms for multiple fault diagnosis with unreliable tests , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[14]  James A. Ritcey,et al.  Iterative Channel Estimation and LDPC Decoding over Flat-Fading Channels: A Factor Graph Approach , 2003 .

[15]  Ashraf M. Abdelbar,et al.  Approximating MAPs for Belief Networks is NP-Hard and Other Theorems , 1998, Artif. Intell..

[16]  William T. Freeman,et al.  Learning to Estimate Scenes from Images , 1998, NIPS.

[17]  Martin J. Wainwright,et al.  MAP estimation via agreement on (hyper)trees: Message-passing and linear programming , 2005, ArXiv.

[18]  H R Warner.  Iliad: moving medical decision-making into new frontiers. , 1989, Methods of information in medicine.

[19]  J. Reggia,et al.  Abductive Inference Models for Diagnostic Problem-Solving , 1990, Symbolic Computation.

[20]  David Heckerman,et al.  A Tractable Inference Algorithm for Diagnosing Multiple Diseases , 2013, UAI.

[21]  Adnan Darwiche,et al.  Solving MAP Exactly using Systematic Search , 2002, UAI.

[22]  J. Beasley.  A lagrangian heuristic for set‐covering problems , 1990 .

[23]  M. Opper,et al.  From Naive Mean Field Theory to the TAP Equations , 2001 .

[24]  Michael I. Jordan,et al.  Variational Probabilistic Inference and the QMR-DT Network , 2011, J. Artif. Intell. Res..

[25]  Dimitri P. Bertsekas,et al.  Convex Analysis and Optimization , 2003 .

[26]  A. P. Dawid,et al.  Applications of a general propagation algorithm for probabilistic expert systems , 1992 .

[27]  Gregory M. Provan,et al.  Knowledge Engineering for Large Belief Networks , 1994, UAI.

[28]  Changhe Yuan,et al.  Annealed MAP , 2004, UAI.

[29]  Edzard S. Gelsema,et al.  Abductive reasoning in Bayesian belief networks using a genetic algorithm , 1995, Pattern Recognit. Lett..

[30]  R. Dechter,et al.  Stochastic Local Search for Bayesian Networks , 1999 .

[31]  Yun Peng,et al.  A Neural Network Approach to Approximating Map in Belief Networks , 2002, Int. J. Neural Syst..

[32]  Michael I. Jordan,et al.  Variational methods and the QMR-DT database , 1998 .

[33]  R. Kikuchi.  A Theory of Cooperative Phenomena , 1951 .

[34]  Francis J. Vasko,et al.  Using a facility location algorithm to solve large set covering problems , 1984 .

[35]  Wayne E. Stark,et al.  Unified design of iterative receivers using factor graphs , 2001, IEEE Trans. Inf. Theory.

[36]  D. Heckerman,et al.  ,81. Introduction , 2022 .

[37]  Michael I. Jordan,et al.  Exploiting Tractable Substructures in Intractable Networks , 1995, NIPS.

[38]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[39]  Y. Bar-Shalom,et al.  m-best S-D assignment algorithm with application to multitarget tracking , 2001 .

[40]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[41]  Krishna R. Pattipati,et al.  Multiple disease (fault) diagnosis with applications to the QMR-DT problem , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[42]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[43]  William T. Freeman,et al.  On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs , 2001, IEEE Trans. Inf. Theory.

[44]  G. Barnett,et al.  DXplain. An evolving diagnostic decision-support system. , 1987, JAMA.

[45]  Timothy M. Franz,et al.  Enhancement of clinicians' diagnostic reasoning by computer-based consultation: a multisite study of 2 systems. , 1999, JAMA.

[46]  J. Beasley.  An algorithm for set covering problem , 1987 .

[47]  Krishna R. Pattipati,et al.  Computationally efficient algorithms for multiple fault diagnosis in large graph-based systems , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[48]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[49]  Yair Weiss,et al.  Approximate Inference and Protein-Folding , 2002, NIPS.

[50]  Steffen L. Lauritzen,et al.  Bayesian updating in causal probabilistic networks by local computations , 1990 .

[51]  José A. Gámez,et al.  Partial abductive inference in Bayesian belief networks - an evolutionary computation approach by using problem-specific genetic operators , 2002, IEEE Trans. Evol. Comput..

[52]  Solomon Eyal Shimony,et al.  Finding MAPs for Belief Networks is NP-Hard , 1994, Artif. Intell..

[53]  Marshall L. Fisher,et al.  The Lagrangian Relaxation Method for Solving Integer Programming Problems , 2004, Manag. Sci..

[54]  Rina Dechter,et al.  Mini-buckets: a general scheme for approximating inference , 2002 .