LIMIDs of decision problems

We introduce the notion of LImited Memory Innuence Diagram (LIMID) to describe multi-stage decision problems where the traditional assumption of no forgetting is relaxed. This can be relevant in situations with multiple decision makers or when decisions must be prescribed under memory constraints, such as e.g. in partially observed Markov decision processes (POMDPs). We give an algorithm for improving any given strategy by local computation of single policy updates. We investigate conditions for the resulting strategy to be optimal. As a consequence we also obtain an eecient algorithm for solving traditional innuence diagrams.

[1]  Scott M. Olmsted.  On representing and solving decision problems , 1983 .

[2]  Thomas D. Nielsen,et al.  Welldeened Decision Scenarios , 1999 .

[3]  Nevin Lianwen Zhang,et al.  Probabilistic Inference in Innuence Diagrams , 1998 .

[4]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[5]  Uue Kjjrull.  Optimal Decomposition of Probabilistic Networks by Simulated Annealing E Optimal Decomposition of Probabilistic Networks by Simulated Annealing , 1992 .

[6]  Frank Jensen,et al.  From Influence Diagrams to junction Trees , 1994, UAI.

[7]  Ross D. Shachter,et al.  Decision Making Using Probabilistic Inference Methods , 1992, UAI.

[8]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[9]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[10]  Judea Pearl,et al.  A Constraint-Propagation Approach to Probabilistic Reasoning , 1985, UAI.

[11]  Gregory F. Cooper,et al.  A Method for Using Belief Networks as Influence Diagrams , 2013, UAI 1988.

[12]  Judea Pearl,et al.  Causal networks: semantics and expressiveness , 2013, UAI.

[13]  Jim Q. Smith.  Influence diagrams for Bayesian decision analysis , 1989 .

[14]  Prakash P. Shenoy,et al.  Valuation-Based Systems for Bayesian Decision Analysis , 1992, Oper. Res..

[15]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[16]  Prakash P. Shenoy,et al.  Axioms for probability and belief-function proagation , 1990, UAI.

[17]  Ross D. Shachter.  Bayes-Ball: The Rational Pastime (for Determining Irrelevance and Requisite Information in Belief Networks and Influence Diagrams) , 1998, UAI.

[18]  Michael I. Jordan.  Graphical Models , 2003 .

[19]  Anders L. Madsen,et al.  Lazy Propagation in Junction Trees , 1998, UAI.

[20]  Ross D. Shachter.  Efficient Value of Information Computation , 1999, UAI.

[21]  Howard Raiffa,et al.  Applied Statistical Decision Theory. , 1961 .

[22]  Uffe Kjærulff.  Optimal decomposition of probabilistic networks by simulated annealing , 1992 .

[23]  J. Neumann,et al.  The Theory of Games and Economic Behaviour , 1944 .

[24]  Nevin Lianwen ZhangDepartment,et al.  Probabilistic Inference in In uence Diagrams , 2003 .