High-Level Planning and Control with Incomplete Information Using POMDPs Hdctor Geffner and

We develop an approach to planning with incomplete information that is based on three elements: 1. a hlgh-level language for describing the effects of actions on both the world and the agent’s beliefs 2. a semantics that translates such descriptions into Partially Observable Markov Decision Processes or POMDPs, and 3. a real time dynamic programming algorithm that produces controllers for such POMDPs. We show that the resulting approach is not only clean and general but that may be practical as well. We have implemented a shell that accepts high-level descriptions of POMDPs and produces uitable controllers, and have tested it over a number of problems. In this paper we present the main elements of the approach and report empirical results for a challenging problem of planning with incomplete information.

[1]  Edward J. Sondik,et al.  The optimal control of par-tially observable Markov processes , 1971 .

[2]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[3]  Robert C. Moore A Formal Theory of Knowledge and Action , 1984 .

[4]  Y. Shoham What is the frame problem , 1987 .

[5]  Keiji Kanazawa,et al.  A model for reasoning about persistence and causation , 1989 .

[6]  Richard E. Korf,et al.  Real-Time Heuristic Search , 1990, Artif. Intell..

[7]  Raymond Reiter,et al.  The Frame Problem in the Situation Calculus: A Simple Solution (Sometimes) and a Completeness Result for Goal Regression , 1991, Artificial and Mathematical Theory of Computation.

[8]  Oren Etzioni,et al.  An Approach to Planning with Incomplete Information , 1992, KR.

[9]  Michael Gelfond,et al.  Representing Action and Change by Logic Programs , 1993, J. Log. Program..

[10]  Hector J. Levesque,et al.  The Frame Problem and Knowledge-Producing Actions , 1993, AAAI.

[11]  Leslie Pack Kaelbling,et al.  Acting Optimally in Partially Observable Stochastic Domains , 1994, AAAI.

[12]  Daniel S. Weld,et al.  Probabilistic Planning with Information Gathering and Contingent Execution , 1994, AIPS.

[13]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[14]  Gregg Collins,et al.  Planning Under Uncertainty: Some Key Issues , 1995, IJCAI.

[15]  Leslie Pack Kaelbling,et al.  Learning Policies for Partially Observable Environments: Scaling Up , 1997, ICML.

[16]  Craig Boutilier,et al.  Exploiting Structure in Policy Construction , 1995, IJCAI.

[17]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[18]  Hector J. Levesque,et al.  What Is Planning in the Presence of Sensing? , 1996, AAAI/IAAI, Vol. 2.

[19]  T. Dean,et al.  Planning under uncertainty: structural assumptions and computational leverage , 1996 .

[20]  Michael J. Metternich,et al.  Planning under Uncertainty , 2004 .

[21]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.