Planning with Incomplete Information as Heuristic Search in Belief Space

The formulation of planning as heuristic search with heuristics derived from problem representations has turned out to be a fruitful approach for classical planning. In this paper, we pursue a similar idea in the context planning with incomplete information. Planning with incomplete information can be formulated as a problem of search in belief space, where belief states can be either sets of states or more generally probability distribution over states. While the formulation (as the formulation of classical planning as heuristic search) is not particularly novel, the contribution of this paper is to make it explicit, to test it over a number of domains, and to extend it to tasks like planning with sensing where the standard search algorithms do not apply. The resulting planner appears to be competitive with the most recent conformant and contingent planners (e.g., CGP, SGP, and CMBP) while at the same time is more general as it can handle probabilistic actions and sensing, different action costs, and epistemic goals.

[1]  Karl Johan Åström,et al.  Optimal control of Markov processes with incomplete state information , 1965 .

[2]  Edward J. Sondik,et al.  The optimal control of par-tially observable Markov processes , 1971 .

[3]  E. J. Sondik,et al.  The Optimal Control of Partially Observable Markov Decision Processes. , 1971 .

[4]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[5]  Nils J. Nilsson,et al.  Principles of Artificial Intelligence , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Richard E. Korf,et al.  Real-Time Heuristic Search , 1990, Artif. Intell..

[7]  William S. Lovejoy,et al.  Computationally Feasible Bounds for Partially Observed Markov Decision Processes , 1991, Oper. Res..

[8]  Lonnie Chrisman,et al.  Reinforcement Learning with Perceptual Aliasing: The Perceptual Distinctions Approach , 1992, AAAI.

[9]  Illah R. Nourbakhsh,et al.  Time-Saving Tips for Problem Solving with Incomplete Information , 1993, AAAI.

[10]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[11]  Nicholas Kushmerick,et al.  An Algorithm for Probabilistic Planning , 1995, Artif. Intell..

[12]  Avrim Blum,et al.  Fast Planning Through Planning Graph Analysis , 1995, IJCAI.

[13]  Stuart J. Russell,et al.  Approximating Optimal Policies for Partially Observable Stochastic Domains , 1995, IJCAI.

[14]  Ronald Fagin,et al.  Reasoning about knowledge , 1995 .

[15]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[16]  Bart Selman,et al.  Pushing the Envelope: Planning, Propositional Logic and Stochastic Search , 1996, AAAI/IAAI, Vol. 2.

[17]  Hector J. Levesque,et al.  What Is Planning in the Presence of Sensing? , 1996, AAAI/IAAI, Vol. 2.

[18]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[19]  Drew McDermott,et al.  A Heuristic Estimator for Means-Ends Analysis in Planning , 1996, AIPS.

[20]  Milos Hauskrecht,et al.  Planning and control in stochastic domains with imperfect information , 1997 .

[21]  Blai Bonet,et al.  A Robust and Fast Action Selection Mechanism for Planning , 1997, AAAI/IAAI.

[22]  Michael L. Littman,et al.  MAXPLAN: A New Approach to Probabilistic Planning , 1998, AIPS.

[23]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[24]  Shlomo Zilberstein,et al.  Heuristic Search in Cyclic AND/OR Graphs , 1998, AAAI/IAAI.

[25]  Ross D. Shachter,et al.  Decision-theoretic planning , 1998 .

[26]  Hector Geffner,et al.  Solving Large POMDPs using Real Time Dynamic Programming , 1998 .

[27]  Jacques Wainer,et al.  Modeling Action, Knowledge and Control , 1998, ECAI.

[28]  David E. Smith,et al.  Conformant Graphplan , 1998, AAAI/IAAI.

[29]  David E. Smith,et al.  Extending Graphplan to handle uncertainty and sensing actions , 1998, AAAI 1998.

[30]  Blai Bonet.  High-Level Planning and Control with Incomplete Information Using POMDP's , 1998 .

[31]  Craig A. Knoblock,et al.  PDDL-the planning domain definition language , 1998 .

[32]  Blai Bonet,et al.  Planning as Heuristic Search: New Results , 1999, ECP.

[33]  Jim Blythe,et al.  Decision-Theoretic Planning , 1999, AI Mag..

[34]  Peter van Beek,et al.  CPlan: A Constraint Programming Approach to Planning , 1999, AAAI/IAAI.

[35]  Fausto Giunchiglia,et al.  Planning as Model Checking , 1999, ECP.

[36]  Blai Bonet.  Functional Strips: a More General Language for Planning and Problem Solving (preliminary Version) , 1999 .

[37]  Ioannis P. Vlahavas,et al.  GRT: A Domain Independent Heuristic for STRIPS Worlds Based on Greedy Regression Tables , 1999, ECP.

[38]  Bart Selman,et al.  Unifying SAT-based and Graph-based Planning , 1999, IJCAI.

[39]  Dana S. Nau,et al.  On the Use of Integer Programming Models in AI Planning , 1999, IJCAI.

[40]  Jussi Rintanen,et al.  Constructing Conditional Plans by a Theorem-Prover , 1999, J. Artif. Intell. Res..

[41]  Marco Roveri,et al.  Recent Advances in AI Planning , 1999, Lecture Notes in Computer Science.

[42]  Blai Bonet,et al.  Planning as heuristic search , 2001, Artif. Intell..