Annealed MAP

Maximum a Posteriori assignment (MAP) is the problem of finding the most probable instantiation of a set of variables given the partial evidence on the other variables in a Bayesian network. MAP has been shown to be a NP-hard problem [22], even for constrained networks, such as polytrees [18]. Hence, previous approaches often fail to yield any results for MAP problems in large complex Bayesian networks. To address this problem, we propose ANNEALEDMAP algorithm, a simulated annealing-based MAP algorithm. The ANNEALEDMAP algorithm simulates a non-homogeneous Markov chain whose invariant function is a probability density that concentrates itself on the modes of the target density. We tested this algorithm on several real Bayesian networks. The results show that, while maintaining good quality of the MAP solutions, the ANNEALEDMAP algorithm is also able to solve many problems that are beyond the reach of previous approaches.

[1]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[2]  José A. Gámez,et al.  Partial abductive inference in Bayesian belief networks using a genetic algorithm , 1999, Pattern Recognit. Lett..

[3]  David Heckerman,et al.  Decision-theoretic troubleshooting , 1995, CACM.

[4]  Kristian Kristensen,et al.  The use of a Bayesian network in the design of a decision support system for growing malting barley without use of pesticides , 2002 .

[5]  Adnan Darwiche,et al.  Solving MAP Exactly using Systematic Search , 2002, UAI.

[6]  Yan Lin,et al.  Relevance-Based Sequential Evidence Processing in Bayesian Networks , 1998, FLAIRS.

[7]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[8]  A. H. Murphy,et al.  Hailfinder: A Bayesian system for forecasting severe weather , 1996 .

[9]  Ewart R. Carson,et al.  A Model-Based Approach to Insulin Adjustment , 1991, AIME.

[10]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[11]  Donald Geman,et al.  Gibbs distributions and the bayesian restoration of images , 1984 .

[12]  Rina Dechter,et al.  Mini-buckets: a general scheme for approximating inference , 2002 .

[13]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[14]  Gregory M. Provan,et al.  Knowledge Engineering for Large Belief Networks , 1994, UAI.

[15]  James D. Park,et al.  MAP Complexity Results and Approximation Methods , 2002, UAI.

[16]  Geoffrey C. Fox,et al.  A Comparison of Annealing Techniques for Academic Course Scheduling , 1997, PATAT.

[17]  G. Bao.  AIR FORCE OFFICE OF SCIENTIFIC RESEARCH , 1999 .

[18]  David Heckerman,et al.  Probabilistic similarity networks , 1991, Networks.

[19]  José A. Gámez,et al.  Partial abductive inference in Bayesian belief networks by simulated annealing , 2001, Int. J. Approx. Reason..

[20]  Adnan Darwiche,et al.  Approximating MAP using Local Search , 2001, UAI.

[21]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[22]  Finn Verner Jensen,et al.  MUNIN: an expert EMG assistant , 1988 .

[23]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Rina Dechter,et al.  Mini-buckets: A general scheme for bounded inference , 2003, JACM.

[25]  Solomon Eyal Shimony,et al.  Finding MAPs for Belief Networks is NP-Hard , 1994, Artif. Intell..