Classical, Probabilistic, and Contingent Planning: Three Models, One Algorithm

Various forms of planning in AI can be viewed as problems of sequential decision that diier only on whether the eeects of actions are predictable and/or observable. Three simple mathematical models, search, mdps and pomdps, provide the conceptual framework to understand such problems but suitable languages and algorithms are needed to model and solve them eeectively. In this paper, we analyze planning from this perspective and report the performance of a simple but general planning algorithm on various planning tasks. Actions We take an abstract view of planning as a problem of sequential decision in which actions have to be executed to achieve a goal. We distinguish two important aspects of actions: whether their eeects are predictable, and whether they are observable. We assume throughout that time is discrete. In deterministic action models, the eeect of actions is completely predictable and can be represented by transition function f such that s 0 = f(s; a) is the unique state that follows the state s after action a. Probabilistic action models, on the other hand, are represented by means of transition probabilities P a (s 0 js) whose sum over the possible successor states s 0 of s and a must add up to one. Observations provide feedback from the environment and determine the form of plans: in the absence of feedback, the plan is a sequence of actions; in the presence of feedback, the plan is a function of the observations. We will distinguish three cases; when the eeect of actions is fully observable, partially observable, or not observable at all. Plans We refer to planning in the presence of observations as closed-loop planning, and planning in the absence of observations as open-loop planning 23, 13, 1]. Classical planning is open-loop planning, while contingent and reactive planning are forms of closed-loop planning. The two forms of planning depend on assumptions about observability and not on whether actions are deterministic or probabilistic. Closed-loop planning is normally regarded as superior because it is more robust: it can recover from perturbations (e.g., a block falling oo the gripper) and errors in the initial conditions or action models (e.g., like assuming that actions are deterministic when they are not). Open-loop plans cannot recover. For this reason, the execution of open-loop plans is usually extended with mechanisms for plan switching, replanning, etc. Models Three standard mathematical models of sequential decisions allow us to make precise …