Probabilistic decision graphs for optimization under uncertainty

This paper provides a survey on probabilistic decision graphs for modeling and solving decision problems under uncertainty. We give an introduction to influence diagrams, which is a popular framework for representing and solving sequential decision problems with a single decision maker. As the methods for solving influence diagrams can scale rather badly in the length of the decision sequence, we present a couple of approaches for calculating approximate solutions. The modeling scope of the influence diagram is limited to so-called symmetric decision problems. This limitation has motivated the development of alternative representation languages, which enlarge the class of decision problems that can be modeled efficiently. We present some of these alternative frameworks and demonstrate their expressibility using several examples. Finally, we provide a list of software systems that implement the frameworks described in the paper.

[1]  Serafín Moral,et al.  Mixtures of Truncated Exponentials in Hybrid Bayesian Networks , 2001, ECSQARU.

[2]  Thomas D. Nielsen,et al.  Probabilistic decision graphs for optimization under uncertainty , 2013, Ann. Oper. Res..

[3]  Thomas D. Nielsen,et al.  Sensitivity analysis in influence diagrams , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[4]  Ross D. Shachter,et al.  Dynamic programming in in uence diagrams with decision circuits , 2010, UAI.

[5]  Anders L. Madsen,et al.  Lazy Evaluation of Symmetric Bayesian Decision Problems , 1999, UAI.

[6]  Prakash P. Shenoy,et al.  Mixtures of Polynomials in Hybrid B ayesian Networks with Deterministic Variables , 2009 .

[7]  Prakash P. Shenoy,et al.  Probability propagation , 1990, Annals of Mathematics and Artificial Intelligence.

[8]  H. Raiffa,et al.  Applied Statistical Decision Theory. , 1961 .

[9]  Anders L. Madsen,et al.  Solving Influence Diagrams using HUGIN, Shafer-Shenoy and Lazy Propagation , 2001, UAI.

[10]  Michael C. Horsch,et al.  An Anytime Algorithm for Decision Making under Uncertainty , 1998, UAI.

[11]  Steffen L. Lauritzen,et al.  Representing and Solving Decision Problems with Limited Information , 2001, Manag. Sci..

[12]  Francisco Javier Díez,et al.  Variable elimination for influence diagrams with super value nodes , 2010, Int. J. Approx. Reason..

[13]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[14]  Marek J. Druzdzel,et al.  An Efficient Exhaustive Anytime Sampling Algorithm for Influence Diagrams , 2007 .

[15]  Finn Verner Jensen,et al.  Unconstrained Influence Diagrams , 2002, UAI.

[16]  Anders L. Madsen,et al.  Solving linear-quadratic conditional Gaussian influence diagrams , 2005, Int. J. Approx. Reason..

[17]  Ya'akov Gal,et al.  A language for modeling agents' decision making processes in games , 2003, AAMAS '03.

[18]  Concha Bielza,et al.  Sensitivity Analysis in IctNeo , 2000 .

[19]  Changhe Yuan,et al.  Solving Multistage Influence Diagrams using Branch-and-Bound Search , 2010, UAI.

[20]  Howard Raiffa,et al.  Applied Statistical Decision Theory. , 1961 .

[21]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[22]  Ross D. Shachter,et al.  Decision Making Using Probabilistic Inference Methods , 1992, UAI.

[23]  Ross D. Shachter.  Efficient Value of Information Computation , 1999, UAI.

[24]  Prakash P. Shenoy,et al.  Sequential influence diagrams: A unified asymmetry framework , 2006, Int. J. Approx. Reason..

[25]  Edmund H. Durfee,et al.  A decision-theoretic approach to coordinating multiagent interactions , 1991, IJCAI 1991.

[26]  Ross D. Shachter.  Evaluating Influence Diagrams , 1986, Oper. Res..

[27]  Kevin B. Korb,et al.  Bayesian Artificial Intelligence , 2004, Computer science and data analysis series.

[28]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[29]  Anders L. Madsen,et al.  Bayesian networks and influence diagrams , 2007 .

[30]  Prakash P. Shenoy,et al.  Multistage Monte Carlo Method for Solving Influence Diagrams Using Local Computation , 2004, Manag. Sci..

[31]  Thomas D. Nielsen.  Decomposition of influence diagrams , 2001, J. Appl. Non Class. Logics.

[32]  Ross D. Shachter,et al.  Evaluating influence diagrams with decision circuits , 2007, UAI.

[33]  Gregory F. Cooper,et al.  A Method for Using Belief Networks as Influence Diagrams , 2013, UAI 1988.

[34]  Barry R. Cobb.  Continuous Decision MTE Influence Diagrams , 2006, Probabilistic Graphical Models.

[35]  Thomas D. Nielsen,et al.  Well-Defined Decision Scenarios , 1999 .

[36]  Thomas D. Nielsen,et al.  A comparison of two approaches for solving unconstrained influence diagrams , 2009, Int. J. Approx. Reason..

[37]  Gordon B. Hazen,et al.  Sensitivity Analysis and the Expected Value of Perfect Information , 1998, Medical decision making : an international journal of the Society for Medical Decision Making.

[38]  Howard Raiffa,et al.  Applied Statistical Decision Theory. , 1961 .

[39]  Qiang Ji,et al.  Efficient non-myopic value-of-information computation for influence diagrams , 2008, Int. J. Approx. Reason..

[40]  Jacinto Martín,et al.  Approximate Solutions of Complex Influence Diagrams through MCMC Methods , 2002, Probabilistic Graphical Models.

[41]  Jonathan Lawry,et al.  Symbolic and Quantitative Approaches to Reasoning with Uncertainty , 2009 .

[42]  Yijing Li,et al.  Solving Hybrid Influence Diagrams with Deterministic Variables , 2010, UAI.

[43]  Finn Verner,et al.  Information enhancement for approximate representation of optimal strategies from in uence diagrams , 2010 .

[44]  Daphne Koller,et al.  Multi-Agent Influence Diagrams for Representing and Solving Games , 2001, IJCAI.

[45]  Adnan Darwiche,et al.  A differential approach to inference in Bayesian networks , 2000, JACM.

[46]  Ross D. Shachter,et al.  Sensitivity analysis in decision circuits , 2008, UAI.

[47]  Prakash P. Shenoy,et al.  Valuation network representation and solution of asymmetric decision problems , 2000, Eur. J. Oper. Res..

[48]  Prakash P. Shenoy,et al.  Decision making with hybrid influence diagrams using mixtures of truncated exponentials , 2004, Eur. J. Oper. Res..

[49]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[50]  Changhe Yuan,et al.  Solving influence diagrams using heuristic search , 2010, ISAIM.

[51]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[52]  A. Madsen,et al.  New Methods for Marginalization in Lazy Propagation , 2008 .

[53]  Frank Jensen,et al.  From Influence Diagrams to junction Trees , 1994, UAI.

[54]  Simon Parsons.  Bayesian Artificial Intelligence by Kevin B. Korb and Ann E. Nicholson, Chapman and Hall, 369 pp., $79.95, ISBN 1-58488-387-1 , 2004, Knowl. Eng. Rev..

[55]  Adnan Darwiche,et al.  Modeling and Reasoning with Bayesian Networks , 2009 .

[56]  Prakash P. Shenoy,et al.  Valuation-Based Systems for Bayesian Decision Analysis , 1992, Oper. Res..

[57]  Nevin Lianwen Zhang,et al.  Probabilistic Inference in Influence Diagrams , 1998, Comput. Intell..

[58]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[59]  Andrés Cano,et al.  A forward-backward Monte Carlo method for solving influence diagrams , 2006, Int. J. Approx. Reason..

[60]  Steffen L. Lauritzen,et al.  Evaluating Influence Diagrams using LIMIDs , 2000, UAI.

[61]  Rina Dechter,et al.  A New Perspective on Algorithms for Optimizing Policies under Uncertainty , 2000, AIPS.

[62]  Thomas D. Nielsen,et al.  Welldefined Decision Scenarios , 1999, UAI.

[63]  Finn Verner Jensen,et al.  Myopic Value of Information in Influence Diagrams , 1997, UAI.

[64]  Rina Dechter,et al.  An Anytime Approximation For Optimizing Policies Under Uncertainty , 2000 .

[65]  Ya'akov Gal,et al.  Networks of Influence Diagrams: A Formalism for Representing Agents' Beliefs and Decision-Making Processes , 2008, J. Artif. Intell. Res..