A Possibilistic Model for Qualitative Sequential Decision Problems under Uncertainty in Partially Observable Environments

In this article we propose a qualitative (ordinal) counterpart for the Partially Observable Markov Decision Processes model (POMDP) in which the uncertainty, as well as the preferences of the agent, are modeled by possibility distributions. This qualitative counterpart of the POMDP model relies on a possibilistic theory of decision under uncertainty, recently developed. One advantage of such a qualitative framework is its ability to escape from the classical obstacle of stochastic POMDPs, in which even with a finite state space, the obtained belief state space of the POMDP is infinite. Instead, in the possibilistic framework even if exponentially larger than the state space, the belief state space remains finite.

[1]  R. Bellman.  Dynamic programming. , 1957, Science.

[2]  Jérôme Lang,et al.  Towards qualitative approaches to multi-stage decision making , 1998, Int. J. Approx. Reason..

[3]  Ronen I. Brafman,et al.  A Heuristic Variable Grid Solution Method for POMDPs , 1997, AAAI/IAAI.

[4]  Didier Dubois,et al.  A survey of belief revision and updating rules in various uncertainty models , 1994, Int. J. Intell. Syst..

[5]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[6]  Didier Dubois,et al.  Qualitative Decision Theory with Sugeno Integrals , 1998, UAI.

[7]  W. Lovejoy.  A survey of algorithmic methods for partially observed Markov decision processes , 1991 .

[8]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[9]  E. Hisdal.  Conditional possibilities independence and noninteraction , 1978 .

[10]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[11]  Milos Hauskrecht,et al.  Incremental Methods for Computing Bounds in Partially Observable Markov Decision Processes , 1997, AAAI/IAAI.

[12]  Stuart J. Russell,et al.  Approximating Optimal Policies for Partially Observable Stochastic Domains , 1995, IJCAI.

[13]  Leslie Pack Kaelbling,et al.  Acting Optimally in Partially Observable Stochastic Domains , 1994, AAAI.

[14]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[15]  Régis Sabbadin.  Decision As Abduction? , 1998, ECAI.

[16]  Didier Dubois,et al.  Possibility Theory as a Basis for Qualitative Decision Theory , 1995, IJCAI.

[17]  Didier Dubois,et al.  The logical view of conditioning and its application to possibility and evidence theories , 1990, Int. J. Approx. Reason..

[18]  G. Monahan.  State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[19]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .