A Concise Introduction to Models and Methods for Automated Planning

Planning is the model-based approach to autonomous behavior where the agent behavior is derived automatically from a model of the actions, sensors, and goals. The main challenges in planning are computational as all models, whether featuring uncertainty and feedback or not, are intractable in the worst case when represented in compact form. In this book, we look at a variety of models used in AI planning, and at the methods that have been developed for solving them. The goal is to provide a modern and coherent view of planning that is precise, concise, and mostly self-contained, without being shallow. For this, we make no attempt at covering the whole variety of planning approaches, ideas, and applications, and focus on the essentials. The target audience of the book are students and researchers interested in autonomous behavior and planning from an AI, engineering, or cognitive science perspective. Table of Contents: Preface / Planning and Autonomous Behavior / Classical Planning: Full Information and Deterministic Actions / Classical Planning: Variations and Extensions / Beyond Classical Planning: Transformations / Planning with Sensing: Logical Models / MDP Planning: Stochastic Actions and Full Feedback / POMDP Planning: Stochastic Actions and Partial Feedback / Discussion / Bibliography / Author's Biography

[1]  Marc Toussaint,et al.  Probabilistic inference for solving discrete and continuous state Markov Decision Processes , 2006, ICML.

[2]  Yngvi Björnsson,et al.  Simulation-Based Approach to General Game Playing , 2008, AAAI.

[3]  Blai Bonet,et al.  Faster Heuristic Search Algorithms for Planning with Uncertainty and Full Feedback , 2003, IJCAI.

[4]  Richard E. Korf,et al.  Depth-First Iterative-Deepening: An Optimal Admissible Tree Search , 1985, Artif. Intell..

[5]  Qiang Yang,et al.  Quantifying information and contradiction in propositional logic through test actions , 2009, IJCAI.

[6]  Christian J. Muise,et al.  Improved Non-Deterministic Planning by Exploiting State Relevance , 2012, ICAPS.

[7]  Scott Sanner,et al.  Symbolic Dynamic Programming for First-order POMDPs , 2010, AAAI.

[8]  Ronen I. Brafman,et al.  R-MAX - A General Polynomial Time Algorithm for Near-Optimal Reinforcement Learning , 2001, J. Mach. Learn. Res..

[9]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[10]  Giuseppe De Giacomo,et al.  Computing Infinite Plans for LTL Goals Using a Classical Planner , 2011, IJCAI.

[11]  Piergiorgio Bertoli,et al.  A Framework for Planning with Extended Goals under Partial Observability , 2003, ICAPS.

[12]  Avrim Blum,et al.  Fast Planning Through Planning Graph Analysis , 1995, IJCAI.

[13]  Mausam,et al.  LRTDP Versus UCT for Online Probabilistic Planning , 2012, AAAI.

[14]  Bart Selman,et al.  Pushing the Envelope: Planning, Propositional Logic and Stochastic Search , 1996, AAAI/IAAI, Vol. 2.

[15]  Scott Sanner,et al.  Practical solution techniques for first-order MDPs , 2009, Artif. Intell..

[16]  Fahiem Bacchus,et al.  Using temporal logics to express search control knowledge for planning , 2000, Artif. Intell..

[17]  Bart Selman,et al.  Planning as Satisfiability , 1992, ECAI.

[18]  Piergiorgio Bertoli,et al.  Improving Heuristics for Planning as Search in Belief Space , 2002, AIPS.

[19]  David Chapman,et al.  Pengi: An Implementation of a Theory of Activity , 1987, AAAI.

[20]  Earl D. Sacerdoti,et al.  Planning in a Hierarchy of Abstraction Spaces , 1974, IJCAI.

[21]  Patrik Haslum,et al.  Deterministic planning in the fifth international planning competition: PDDL3 and experimental evaluation of the planners , 2009, Artif. Intell..

[22]  Carmel Domshlak,et al.  Optimal admissible composition of abstraction heuristics , 2010, Artif. Intell..

[23]  Joelle Pineau,et al.  Anytime Point-Based Approximations for Large POMDPs , 2006, J. Artif. Intell. Res..

[24]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[25]  Ronen I. Brafman,et al.  Planning Games , 2009, IJCAI.

[26]  Joel Veness,et al.  Monte-Carlo Planning in Large POMDPs , 2010, NIPS.

[27]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[28]  Guy Shani,et al.  Noname manuscript No. (will be inserted by the editor) A Survey of Point-Based POMDP Solvers , 2022 .

[29]  Hector Geffner,et al.  Searching for Plans with Carefully Designed Probes , 2011, ICAPS.

[30]  Piergiorgio Bertoli,et al.  Planning in Nondeterministic Domains under Partial Observability via Symbolic Model Checking , 2001, IJCAI.

[31]  Robert Givan,et al.  FF-Replan: A Baseline for Probabilistic Planning , 2007, ICAPS.

[32]  H. Jaap van den Herik,et al.  Progressive Strategies for Monte-Carlo Tree Search , 2008 .

[33]  Patrik Haslum,et al.  Domain-Independent Construction of Pattern Database Heuristics for Cost-Optimal Planning , 2007, AAAI.

[34]  Tom Bylander,et al.  The Computational Complexity of Propositional STRIPS Planning , 1994, Artif. Intell..

[35]  Jussi Rintanen,et al.  Planning as satisfiability: Heuristics , 2012, Artif. Intell..

[36]  Daniel Bryce,et al.  Planning Graph Heuristics for Belief Space Search , 2006, J. Artif. Intell. Res..

[37]  Richard E. Korf,et al.  Real-Time Heuristic Search , 1990, Artif. Intell..

[38]  Robert P. Goldman,et al.  A probabilistic plan recognition algorithm based on plan tree grammars , 2009, Artif. Intell..

[39]  Bernhard Nebel,et al.  COMPLEXITY RESULTS FOR SAS+ PLANNING , 1995, Comput. Intell..

[40]  Daniel S. Weld.  An Introduction to Least Commitment Planning , 1994, AI Mag..

[41]  Sven Koenig,et al.  Comparing real-time and incremental heuristic search for real-time situated agents , 2009, Autonomous Agents and Multi-Agent Systems.

[42]  Allen Newell,et al.  Report on a general problem-solving program , 1959, IFIP Congress.

[43]  Manuela M. Veloso,et al.  A Formalization of Equilibria for Multiagent Planning , 2003, IJCAI.

[44]  Giuseppe De Giacomo,et al.  Automata-Theoretic Approach to Planning for Temporally Extended Goals , 1999, ECP.

[45]  Craig A. Knoblock,et al.  Combining the Expressivity of UCPOP with the Efficiency of Graphplan , 1997, ECP.

[46]  David Silver,et al.  Combining online and offline knowledge in UCT , 2007, ICML '07.

[47]  Farokh B. Bastani,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Simple and Fast Strong Cyclic Planning for Fully-Observable Nondeterministic Planning Problems �� , 2022 .

[48]  Hector Geffner,et al.  Probabilistic Plan Recognition Using Off-the-Shelf Classical Planners , 2010, AAAI.

[49]  Guy Shani,et al.  Replanning in Domains with Partial Information and Sensing Actions , 2012, J. Artif. Intell. Res..

[50]  Richard E. Korf,et al.  Planning as Search: A Quantitative Approach , 1987, Artif. Intell..

[51]  C. Striebel.  Sufficient statistics in the optimum control of stochastic systems , 1965 .

[52]  Hector Geffner,et al.  Goal Recognition over POMDPs: Inferring the Intention of a POMDP Agent , 2011, IJCAI.

[53]  David Chapman,et al.  Penguins Can Make Cake , 1989, AI Mag..

[54]  Carmel Domshlak,et al.  Structural Patterns Heuristics via Fork Decomposition , 2008, ICAPS.

[55]  Joel Veness,et al.  A Monte-Carlo AIXI Approximation , 2009, J. Artif. Intell. Res..

[56]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[57]  Hector Geffner,et al.  Branching and pruning: An optimal temporal POCL planner based on constraint programming , 2004, Artif. Intell..

[58]  Robert P. Goldman,et al.  Using Classical Planners to Solve Nondeterministic Planning Problems , 2008, ICAPS.

[59]  S. Kambhampati,et al.  Optiplan: Unifying IP-based and Graph-based Planning , 2005, J. Artif. Intell. Res..

[60]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[61]  Blai Bonet,et al.  Strengthening Landmark Heuristics via Hitting Sets , 2010, ECAI.

[62]  Malte Helmert,et al.  Sound and Complete Landmarks for And/Or Graphs , 2010, ECAI.

[63]  Jonathan Schaeffer,et al.  Sokoban: Enhancing general single-agent search methods using domain knowledge , 2001, Artif. Intell..

[64]  Andrew Coles,et al.  Temporal Planning in Domains with Linear Processes , 2009, IJCAI.

[65]  Neil Immerman,et al.  A new representation and associated algorithms for generalized planning , 2011, Artif. Intell..

[66]  Peng Dai,et al.  Reverse Iterative Deepening for Finite-Horizon MDPs with Large Branching Factors , 2012, ICAPS.

[67]  Craig Boutilier,et al.  Bounded Finite State Controllers , 2003, NIPS.

[68]  Malte Helmert,et al.  The More, the Merrier: Combining Heuristic Estimators for Satisficing Planning , 2010, ICAPS.

[69]  H. Simon,et al.  A Behavioral Model of Rational Choice , 1955 .

[70]  Blai Bonet,et al.  Planning with Incomplete Information as Heuristic Search in Belief Space , 2000, AIPS.

[71]  Blai Bonet,et al.  Width and Complexity of Belief Tracking in Non-Deterministic Conformant and Contingent Planning , 2012, AAAI.

[72]  Stuart J. Russell,et al.  Angelic Semantics for High-Level Actions , 2007, ICAPS.

[73]  Enrico Pontelli,et al.  On the Effectiveness of CNF and DNF Representations in Contingent Planning , 2011, IJCAI.

[74]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[75]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[76]  Patrik Haslum,et al.  Admissible Heuristics for Optimal Planning , 2000, AIPS.

[77]  Shlomo Zilberstein,et al.  LAO*: A heuristic search algorithm that finds solutions with loops , 2001, Artif. Intell..

[78]  Hector Geffner,et al.  Learning Generalized Policies in Planning Using Concept Languages , 2000, KR.

[79]  Erez Karpas,et al.  Sensible Agent Technology Improving Coordination and Communication in Biosurveillance Domains , 2009, IJCAI.

[80]  Blai Bonet,et al.  A Robust and Fast Action Selection Mechanism for Planning , 1997, AAAI/IAAI.

[81]  Craig Boutilier,et al.  Decision-Theoretic Planning: Structural Assumptions and Computational Leverage , 1999, J. Artif. Intell. Res..

[82]  Hector Geffner,et al.  Compiling Uncertainty Away in Conformant Planning Problems with Bounded Width , 2009, J. Artif. Intell. Res..

[83]  Blai Bonet,et al.  Conformant plans and beyond: Principles and complexity , 2010, Artif. Intell..

[84]  Zhengzhu Feng,et al.  Symbolic heuristic search for factored Markov decision processes , 2002, AAAI/IAAI.

[85]  Blai Bonet,et al.  Solving POMDPs: RTDP-Bel vs. Point-based Algorithms , 2009, IJCAI.

[86]  Bart Selman,et al.  Unifying SAT-based and Graph-based Planning , 1999, IJCAI.

[87]  Hector Geffner,et al.  A Translation-Based Approach to Contingent Planning , 2009, IJCAI.

[88]  Carmel Domshlak,et al.  Landmarks, Critical Paths and Abstractions: What's the Difference Anyway? , 2009, ICAPS.

[89]  Jorge A. Baier,et al.  A Heuristic Search Approach to Planning with Temporally Extended Preferences , 2007, IJCAI.

[90]  David E. Smith,et al.  Temporal Planning with Mutual Exclusion Reasoning , 1999, IJCAI.

[91]  David Furcy,et al.  Speeding up the calculation of heuristics for heuristic search-based planning , 2002, AAAI/IAAI.

[92]  Shlomo Zilberstein,et al.  Optimizing fixed-size stochastic controllers for POMDPs and decentralized POMDPs , 2010, Autonomous Agents and Multi-Agent Systems.

[93]  Eric A. Hansen,et al.  Solving POMDPs by Searching in Policy Space , 1998, UAI.

[94]  Craig Boutilier,et al.  Symbolic Dynamic Programming for First-Order MDPs , 2001, IJCAI.

[95]  Michael L. Littman,et al.  The Computational Complexity of Probabilistic Planning , 1998, J. Artif. Intell. Res..

[96]  Alan Fern,et al.  UCT for Tactical Assault Planning in Real-Time Strategy Games , 2009, IJCAI.

[97]  Blai Bonet,et al.  Planning as heuristic search , 2001, Artif. Intell..

[98]  Marvin Minsky,et al.  Steps toward Artificial Intelligence , 1995, Proceedings of the IRE.

[99]  Ivan Serina,et al.  An approach to efficient planning with numerical fluents and multi-criteria plan quality , 2008, Artif. Intell..

[100]  Pierre Wolper,et al.  Reasoning About Infinite Computations , 1994, Inf. Comput..

[101]  Jussi Rintanen,et al.  Distance Estimates for Planning in the Discrete Belief Space , 2004, AAAI.

[102]  Blai Bonet,et al.  Automatic Derivation of Memoryless Policies and Finite-State Controllers Using Classical Planners , 2009, ICAPS.

[103]  Hector Geffner,et al.  Computational models of planning. , 2013, Wiley interdisciplinary reviews. Cognitive science.

[104]  Hector Geffner,et al.  Heuristics for Planning with Action Costs Revisited , 2008, ECAI.

[105]  Neil Immerman,et al.  Qualitative Numeric Planning , 2011, AAAI.

[106]  Rajesh P. N. Rao,et al.  Embodiment is the foundation, not a level , 1996, Behavioral and Brain Sciences.

[107]  Jussi Rintanen,et al.  Complexity of Planning with Partial Observability , 2004, ICAPS.

[108]  Hector Geffner,et al.  Fair LTL Synthesis for Non-Deterministic Systems using Strong Cyclic Planners , 2013, IJCAI.

[109]  Silvia Richter,et al.  The LAMA Planner: Guiding Cost-Based Anytime Planning with Landmarks , 2010, J. Artif. Intell. Res..

[110]  Hagai Attias,et al.  Planning by Probabilistic Inference , 2003, AISTATS.

[111]  Blai Bonet,et al.  Planning as Heuristic Search: New Results , 1999, ECP.

[112]  Hector Geffner,et al.  Artificial Intelligence: From programs to solvers , 2014, AI Commun..

[113]  Scott Sanner,et al.  A Survey of the Seventh International Planning Competition , 2012, AI Mag..

[114]  Patrik Haslum,et al.  Some Results on the Complexity of Planning with Incomplete Information , 1999, ECP.

[115]  Shlomo Zilberstein,et al.  Symbolic Generalization for On-line Planning , 2002, UAI.

[116]  Blai Bonet,et al.  mGPT: A Probabilistic Planner Based on Heuristic Search , 2005, J. Artif. Intell. Res..

[117]  Marco Pistore,et al.  Weak, strong, and strong cyclic planning via symbolic model checking , 2003, Artif. Intell..

[118]  Jörg Hoffmann.  Analyzing Search Topology Without Running Any Search: On the Connection Between Causal Graphs and h+ , 2011, J. Artif. Intell. Res..

[119]  Eric A. Hansen,et al.  Anytime Heuristic Search , 2011, J. Artif. Intell. Res..

[120]  Piergiorgio Bertoli,et al.  Conformant planning via symbolic model checking and heuristic search , 2004, Artif. Intell..

[121]  Subbarao Kambhampati,et al.  When is Temporal Planning Really Temporal? , 2007, IJCAI.

[122]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[123]  Christer Bäckström,et al.  Tractable Planning with State Variables by Exploiting Structural Restrictions , 1994, AAAI.

[124]  Qiang Yang,et al.  Downward Refinement and the Efficiency of Hierarchical Problem Solving , 1994, Artif. Intell..

[125]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[126]  Anders Jonsson.  The Role of Macros in Tractable Planning over Causal Graphs , 2007, IJCAI.

[127]  Amir Pnueli,et al.  The temporal logic of programs , 1977, 18th Annual Symposium on Foundations of Computer Science (sfcs 1977).

[128]  Hector Geffner,et al.  Soft Goals Can Be Compiled Away , 2009, J. Artif. Intell. Res..

[129]  Robert P. Goldman,et al.  Expressive Planning and Explicit Knowledge , 1996, AIPS.

[130]  Hector Geffner,et al.  Effective Heuristics and Belief Tracking for Planning with Incomplete Information , 2011, ICAPS.

[131]  Marcel Schoppers,et al.  Universal Plans for Reactive Robots in Unpredictable Environments , 1987, IJCAI.

[132]  Patrik Haslum,et al.  LTL Goal Specifications Revisited , 2010, ECAI.

[133]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[134]  Subbarao Kambhampati,et al.  Reviving Partial Order Planning , 2001, IJCAI.

[135]  David A. McAllester,et al.  Systematic Nonlinear Planning , 1991, AAAI.

[136]  Risto Miikkulainen,et al.  Real-time neuroevolution in the NERO video game , 2005, IEEE Transactions on Evolutionary Computation.

[137]  Hubie Chen,et al.  Act Local, Think Global: Width Notions for Tractable Planning , 2007, ICAPS.

[138]  Eugene C. Freuder.  A Sufficient Condition for Backtrack-Free Search , 1982, JACM.

[139]  Austin Tate,et al.  Generating Project Networks , 1977, IJCAI.

[140]  Sheila A. McIlraith,et al.  Planning with complex actions , 2002, NMR.

[141]  Craig A. Knoblock.  Learning Abstraction Hierarchies for Problem Solving , 1990, AAAI.

[142]  Robert Givan,et al.  Approximate Policy Iteration with a Policy Language Bias , 2003, NIPS.

[143]  Karl Johan Åström,et al.  Optimal control of Markov processes with incomplete state information , 1965 .

[144]  Wheeler Ruml,et al.  The Joy of Forgetting: Faster Anytime Search via Restarting , 2010, ICAPS.

[145]  Hector Geffner,et al.  Width and Serialization of Classical Planning Problems , 2012, ECAI.

[146]  Blai Bonet,et al.  Labeled RTDP: Improving the Convergence of Real-Time Dynamic Programming , 2003, ICAPS.

[147]  Guy Shani,et al.  A Multi-Path Compilation Approach to Contingent Planning , 2012, AAAI.

[148]  Subbarao Kambhampati,et al.  Planning as Refinement Search: A Unified Framework for Evaluating Design Tradeoffs in Partial-Order Planning , 1995, Artif. Intell..

[149]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[150]  Eyal Amir,et al.  Factored planning , 2003, IJCAI 2003.

[151]  David M. Sobel,et al.  A theory of causal learning in children: causal maps and Bayes nets. , 2004, Psychological review.

[152]  Ronen I. Brafman,et al.  Factored Planning: How, When, and When Not , 2006, AAAI.

[153]  Carmel Domshlak,et al.  Optimal Additive Composition of Abstraction-based Admissible Heuristics , 2008, ICAPS.

[154]  Stefan Edelkamp,et al.  On the Compilation of Plan Constraints and Preferences , 2006, ICAPS.

[155]  Pierre Wolper,et al.  Simple on-the-fly automatic verification of linear temporal logic , 1995, PSTV.

[156]  P. P. Chakrabarti,et al.  Best first search in and/or graphs , 1988, CSC '88.

[157]  Edwin P. D. Pednault,et al.  ADL: Exploring the Middle Ground Between STRIPS and the Situation Calculus , 1989, KR.

[158]  Sylvie Thiébaux,et al.  Search Control in Planning for Temporally Extended Goals , 2005, ICAPS.

[159]  Yuxiao Hu,et al.  Generalized Planning: Synthesizing Plans that Work for Multiple Environments , 2011, IJCAI.

[160]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[161]  Malte Helmert,et al.  High-Quality Policies for the Canadian Traveler's Problem , 2010, SOCS.

[162]  S. Edelkamp.  Planning with Pattern Databases , 2014 .

[163]  Bernhard Nebel,et al.  On the Compilability and Expressive Power of Propositional Planning Formalisms , 2000, J. Artif. Intell. Res..

[164]  Paolo Traverso,et al.  Strong Cyclic Planning Revisited , 1999, ECP.

[165]  David E. Smith.  Choosing Objectives in Over-Subscription Planning , 2004, ICAPS.

[166]  Matthew L. Ginsberg,et al.  Universal Planning: An (Almost) Universally Bad Idea , 1989, AI Mag..

[167]  Henry A. Kautz,et al.  Generalized Plan Recognition , 1986, AAAI.