A POMDP Approach to Influence Diagram Evaluation

We propose a node-removal/arc-reversal algorithm for influence diagram evaluation that includes reductions that allow an influence diagram to be solved by a generalization of the dynamic programming approach to solving partially observable Markov decision processes (POMDPs). Among its potential advantages, the algorithm allows a more flexible ordering of node removals, and a POMDP-inspired approach to optimizing over hidden state variables, which can improve the scalability of influence diagram evaluation in solving complex, multi-stage problems. It also finds a more compact representation of an optimal strategy.

[1]  Scott M. Olmsted.  On representing and solving decision problems , 1983 .

[2]  H. Brachinger,et al.  Decision analysis , 1997 .

[3]  Ross D. Shachter,et al.  Decision Making Using Probabilistic Inference Methods , 1992, UAI.

[4]  Ross D. Shachter.  Efficient Value of Information Computation , 1999, UAI.

[5]  Zhengzhu Feng,et al.  Dynamic Programming for POMDPs Using a Factored State Representation , 2000, AIPS.

[6]  Thomas D. Nielsen,et al.  Well-Defined Decision Scenarios , 1999 .

[7]  George E. Monahan,et al.  A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 2007 .

[8]  David L. Poole,et al.  A NEW METHOD FOR INFLUENCE DIAGRAM EVALUATION , 1993, Comput. Intell..

[9]  Gregory F. Cooper,et al.  A Method for Using Belief Networks as Influence Diagrams , 2013, UAI 1988.

[10]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[11]  Changhe Yuan,et al.  Solving Multistage Influence Diagrams using Branch-and-Bound Search , 2010, UAI.

[12]  Michael L. Littman,et al.  Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes , 1997, UAI.

[13]  Michael Höhle,et al.  Computing bounds on expected utilities based on limited information , 2001 .

[14]  Rina Dechter,et al.  A New Perspective on Algorithms for Optimizing Policies under Uncertainty , 2000, AIPS.

[15]  J. Satia,et al.  Markovian Decision Processes with Probabilistic Observation of States , 1973 .

[16]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[17]  Thomas D. Nielsen.  Decomposition of Influence Diagrams , 2001, ECSQARU.

[18]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[19]  Ross D. Shachter,et al.  Dynamic programming and influence diagrams , 1990, IEEE Trans. Syst. Man Cybern..

[20]  Nevin Lianwen Zhang,et al.  Probabilistic Inference in Influence Diagrams , 1998, Comput. Intell..

[21]  Craig Boutilier,et al.  uting Optimal Policies for Compact Representations , 1996 .

[22]  Richard Washington,et al.  BI-POMDP: Bounded, Incremental, Partially-Observable Markov-Model Planning , 1997, ECP.

[23]  Frank Jensen,et al.  From Influence Diagrams to junction Trees , 1994, UAI.

[24]  Steffen L. Lauritzen,et al.  Representing and Solving Decision Problems with Limited Information , 2001, Manag. Sci..

[25]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[26]  Thomas D. Nielsen,et al.  Welldefined Decision Scenarios , 1999, UAI.

[27]  Ross D. Shachter.  Evaluating Influence Diagrams , 1986, Oper. Res..