Possibilistic Markov decision processes

Abstract In this article we propose a synthesis of recent works concerning a qualitative approach, based on possibility theory, to multi-stage decision under uncertainty. Our framework is a qualitative possibilistic counterpart to Markov decision processes (MDP), for which we propose dynamic programming-like algorithms. The classical MDP algorithms and their possibilistic counterparts are then experimentally compared on a family of benchmark examples. Finally, we also explore the case of partial observability, thus providing qualitative counterparts to the partially observable Markov decision processes framework.

[1]  Jérôme Lang,et al.  Planning with graded nondeterministic actions: A possibilistic approach , 1997 .

[2]  J. Baldwin,et al.  Dynamic programming for fuzzy systems with fuzzy environment , 1982 .

[3]  W. Lovejoy.  A survey of algorithmic methods for partially observed Markov decision processes , 1991 .

[4]  Ronen I. Brafman,et al.  A Heuristic Variable Grid Solution Method for POMDPs , 1997, AAAI/IAAI.

[5]  Didier Dubois,et al.  A survey of belief revision and updating rules in various uncertainty models , 1994, Int. J. Intell. Syst..

[6]  M. Sniedovich,et al.  Sequential Decision Making in Fuzzy Environment , 1998 .

[7]  R. Słowiński.  Fuzzy sets in decision analysis, operations research and statistics , 1999 .

[8]  Celia Da Costa Pereira.  Planification d'actions en environnement incertain : une approche fondee sur la theorie des possibilites , 1998 .

[9]  Jon Doyle,et al.  Principles of knowledge representation and reasoning : proceedings of the fourth international conference (KR'94), Bonn, Germany, May 24-27, 1994 , 1994 .

[10]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[11]  Régis Sabbadin,et al.  Empirical Comparison of Probabilistic and Possibilistic Markov Decision Processes Algorithms , 2000, ECAI.

[12]  Didier Dubois,et al.  Possibility Theory as a Basis for Qualitative Decision Theory , 1995, IJCAI.

[13]  Didier Dubois,et al.  The logical view of conditioning and its application to possibility and evidence theories , 1990, Int. J. Approx. Reason..

[14]  Jérôme Lang,et al.  Towards qualitative approaches to multi-stage decision making , 1998, Int. J. Approx. Reason..

[15]  D. Dubois,et al.  On Possibility/Probability Transformations , 1993 .

[16]  G. Monahan.  State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[17]  L. J. Savage,et al.  The Foundations of Statistics , 1955 .

[18]  Richard Bellman,et al.  Decision-making in fuzzy environment , 2012 .

[19]  Célia da Costa Pereira,et al.  Possibilistic Planning: Representation and Complexity , 1997, ECP.

[20]  E. Hisdal.  Conditional possibilities independence and noninteraction , 1978 .

[21]  Craig Boutilier,et al.  Toward a Logic for Qualitative Decision Theory , 1994, KR.

[22]  Didier Dubois,et al.  Qualitative Decision Theory with Sugeno Integrals , 1998, UAI.

[23]  Daniel S. Weld.  Recent Advances in AI Planning , 1999, AI Mag..

[24]  Harold F. Smiddy,et al.  Evolution of a “Science of Managing” in America , 1954 .

[25]  Judea Pearl,et al.  Qualitative Decision Theory , 1994, AAAI.

[26]  Janusz Kacprzyk,et al.  Fuzzy dynamic programming , 1999 .

[27]  Leslie Pack Kaelbling,et al.  Acting Optimally in Partially Observable Stochastic Domains , 1994, AAAI.

[28]  R. Yager.  On the specificity of a possibility distribution , 1992 .

[29]  Milos Hauskrecht,et al.  Incremental Methods for Computing Bounds in Partially Observable Markov Decision Processes , 1997, AAAI/IAAI.

[30]  Stuart J. Russell,et al.  Approximating Optimal Policies for Partially Observable Stochastic Domains , 1995, IJCAI.

[31]  R. Bellman.  Dynamic programming. , 1957, Science.

[32]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[33]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[34]  Régis Sabbadin,et al.  A Possibilistic Model for Qualitative Sequential Decision Problems under Uncertainty in Partially Observable Environments , 1999, UAI.

[35]  M. Roubens,et al.  Fuzzy logic : state of the art , 1993 .