Incremental Contingency Planning

There has been considerable work in AI on planning under uncertainty. However, this work generally assumes an extremely simple model of action that does not consider continuous time and resources. These assumptions are not reasonable for a Mars rover, which must cope with uncertainty about the duration of tasks, the energy required, the data storage necessary, and its current position and orientation. In this paper, we outline an approach to generating contingency plans when the sources of uncertainty involve continuous quantities such as time and resources. The approach involves first constructing a "seed" plan, and then incrementally adding contingent branches to this plan in order to improve utility. The challenge is to figure out the best places to insert contingency branches. This requires an estimate of how much utility could be gained by building a contingent branch at any given place in the seed plan. Computing this utility exactly is intractable, but we outline an approximation method that back propagates utility distributions through a graph structure similar to that of a plan graph.

[1]  David H. D. Warren,et al.  Generating Conditional Plans and Programs , 1976, AISB.

[2]  Mark A. Peot,et al.  Conditional nonlinear planning , 1992 .

[3]  Oren Etzioni,et al.  An Approach to Planning with Incomplete Information , 1992, KR.

[4]  Robert P. Goldman,et al.  Conditional Linear Planning , 1994, AIPS.

[5]  Daniel S. Weld,et al.  Probabilistic Planning with Information Gathering and Contingent Execution , 1994, AIPS.

[6]  John L. Bresina,et al.  Just-In-Case Scheduling , 1994, AAAI.

[7]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[8]  Nicholas Kushmerick,et al.  An Algorithm for Probabilistic Planning , 1995, Artif. Intell..

[9]  Avrim Blum,et al.  Fast Planning Through Planning Graph Analysis , 1995, IJCAI.

[10]  Peter Haddawy,et al.  Efficient Decision-Theoretic Planning: Techniques and Empirical Analysis , 1995, UAI.

[11]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[12]  Gregg Collins,et al.  Planning for Contingencies: A Decision-based Approach , 1996, J. Artif. Intell. Res..

[13]  Richard Goodwin.  Using Loops in Decision-Theoretic Refinement Planners , 1996, AIPS.

[14]  Satinder P. Singh,et al.  How to Dynamically Merge Markov Decision Processes , 1997, NIPS.

[15]  Ross D. Shachter,et al.  Decision-theoretic planning , 1998 .

[16]  Kee-Eung Kim,et al.  Solving Very Large Weakly Coupled Markov Decision Processes , 1998, AAAI/IAAI.

[17]  David E. Smith,et al.  Conformant Graphplan , 1998, AAAI/IAAI.

[18]  David E. Smith,et al.  Extending Graphplan to handle uncertainty and sensing actions , 1998, AAAI 1998.

[19]  Keith Golden,et al.  Leap Before You Look: Information Gathering in the PUCCINI Planner , 1998, AIPS.

[20]  Michael L. Littman,et al.  The Computational Complexity of Probabilistic Planning , 1998, J. Artif. Intell. Res..

[21]  Jim Blythe,et al.  Planning Under Uncertainty in Dynamic Domains , 1998 .

[22]  Andrew W. Moore,et al.  Variable Resolution Discretization for High-Accuracy Solutions of Optimal Control Problems , 1999, IJCAI.

[23]  Blai Bonet,et al.  Planning as Heuristic Search: New Results , 1999, ECP.

[24]  Jim Blythe,et al.  Decision-Theoretic Planning , 1999, AI Mag..

[25]  Michael L. Littman,et al.  Contingent planning under uncertainty via stochastic satisfiability , 1999, Artif. Intell..

[26]  Craig Boutilier,et al.  Decision-Theoretic Planning: Structural Assumptions and Computational Leverage , 1999, J. Artif. Intell. Res..

[27]  John Langford,et al.  Probabilistic Planning in the Graphplan Framework , 1999, ECP.

[28]  Martha E. Pollack,et al.  Conditional, Probabilistic Planning: A Unifying Algorithm and Effective Search Control Mechanisms , 1999, AAAI/IAAI.

[29]  Jussi Rintanen,et al.  Constructing Conditional Plans by a Theorem-Prover , 1999, J. Artif. Intell. Res..

[30]  Sebastian Thrun,et al.  Monte Carlo POMDPs , 1999, NIPS.

[31]  Michael L. Littman,et al.  Exact Solutions to Time-Dependent MDPs , 2000, NIPS.

[32]  Enrico Giunchiglia,et al.  Planning as Satisfiability in Nondeterministic Domains , 2000, AAAI/IAAI.

[33]  Marco Roveri,et al.  Conformant Planning via Symbolic Model Checking , 2000, J. Artif. Intell. Res..

[34]  Patrik Haslum,et al.  Admissible Heuristics for Optimal Planning , 2000, AIPS.

[35]  Leslie Pack Kaelbling,et al.  Practical Reinforcement Learning in Continuous Spaces , 2000, ICML.

[36]  Subbarao Kambhampati,et al.  Distance-Based Goal-Ordering Heuristics for Graphplan , 2000, AIPS.

[37]  Blai Bonet,et al.  Planning with Incomplete Information as Heuristic Search in Belief Space , 2000, AIPS.

[38]  Piergiorgio Bertoli,et al.  Heuristic Search + Symbolic Model Checking = Efficient Conformant Planning , 2001, IJCAI.

[39]  Bernhard Nebel,et al.  The FF Planning System: Fast Plan Generation Through Heuristic Search , 2011, J. Artif. Intell. Res..

[40]  Piergiorgio Bertoli,et al.  Planning in Nondeterministic Domains under Partial Observability via Symbolic Model Checking , 2001, IJCAI.

[41]  P. Pandurang Nayak,et al.  Fragment-based Conformant Planning , 2002, AIPS.

[42]  Hudson Turner,et al.  Polynomial-Length Planning Spans the Polynomial Hierarchy , 2002, JELIA.

[43]  Håkan L. S. Younes,et al.  On the Role of Ground Actions in Refinement Planning , 2002, AIPS.

[44]  Subbarao Kambhampati,et al.  Planning graph as the basis for deriving heuristics for plan synthesis by state space and CSP search , 2002, Artif. Intell..

[45]  Subbarao Kambhampati,et al.  Planning Graph-based Heuristics for Cost-sensitive Temporal Planning , 2002, AIPS.

[46]  Rémi Munos,et al.  A Study of Reinforcement Learning in the Continuous Case by the Means of Viscosity Solutions , 2000, Machine Learning.

[47]  Amos Storkey,et al.  Advances in Neural Information Processing Systems 20 , 2007 .

[48]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.