Quasi-Deterministic Partially Observable Markov Decision Processes

We study a subclass of pomdps, called quasi-deterministic pomdps (qDet- pomdps), characterized by deterministic actions and stochastic observations. While this framework does not model the same general problems as pomdps, they still capture a number of interesting and challenging problems and, in some cases, have interesting properties. By studying the observability available in this subclass, we show that qDet- pomdps may fall many steps in the complexity classes of polynomial hierarchy.

[1]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[2]  Michael L. Littman,et al.  Algorithms for Sequential Decision Making , 1996 .

[3]  Paolo Traverso,et al.  Automated Planning: Theory & Practice , 2004 .

[4]  Larry J. Stockmeyer,et al.  The Polynomial-Time Hierarchy , 1976, Theor. Comput. Sci..

[5]  Blai Bonet,et al.  Deterministic POMDPs Revisited , 2009, UAI.

[6]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[7]  Anne Condon,et al.  On the Undecidability of Probabilistic Planning and Infinite-Horizon Partially Observable Markov Decision Problems , 1999, AAAI/IAAI.

[8]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[9]  Robert P. Goldman,et al.  Expressive Planning and Explicit Knowledge , 1996, AIPS.

[10]  K. Pattipati,et al.  Application of heuristic search and information theory to sequential fault diagnosis , 1988, Proceedings IEEE International Symposium on Intelligent Control 1988.

[11]  Hector Geffner,et al.  From Conformant into Classical Planning: Efficient Translations that May Be Complete Too , 2007, ICAPS.

[12]  Krishna R. Pattipati,et al.  Application of heuristic search and information theory to sequential fault diagnosis , 1990, IEEE Trans. Syst. Man Cybern..

[13]  Lawrence Carin,et al.  Nonmyopic Multiaspect Sensing With Partially Observable Markov Decision Processes , 2007, IEEE Transactions on Signal Processing.

[14]  Eyal Amir,et al.  Learning Partially Observable Deterministic Action Models , 2005, IJCAI.