General Policies, Serializations, and Planning Width

It has been observed that in many of the benchmark planning domains, atomic goals can be reached with a simple polynomial exploration procedure, called IW, that runs in time exponential in the problem width. Such problems have indeed a bounded width: a width that does not grow with the number of problem variables and is often no greater than two. Yet, while the notion of width has become part of the state-of-the-art planning algorithms like BFWS, there is still no good explanation for why so many benchmark domains have bounded width. In this work, we address this question by relating bounded width and serialized width to ideas of generalized planning, where general policies aim to solve multiple instances of a planning problem all at once. We show that bounded width is a property of planning domains that admit optimal general policies in terms of features that are explicitly or implicitly represented in the domain encoding. The results are extended to much larger class of domains with bounded serialized width where the general policies do not have to be optimal. The study leads also to a new simple, meaningful, and expressive language for specifying domain serializations in the form of policy sketches which can be used for encoding domain control knowledge by hand or for learning it from traces. The use of sketches and the meaning of the theoretical results are all illustrated through a number of examples.

[1]  Hector Geffner,et al.  Best-First Width Search: Exploration and Exploitation in Classical Planning , 2017, AAAI.

[2]  Neil Immerman,et al.  Learning Generalized Plans Using Abstract Counting , 2008, AAAI.

[3]  Jörg Hoffmann,et al.  Ordered Landmarks in Planning , 2004, J. Artif. Intell. Res..

[4]  Rina Dechter Reasoning with Probabilistic and Deterministic Graphical Models: Exact Algorithms , 2013, Reasoning with Probabilistic and Deterministic Graphical Models: Exact Algorithms.

[5]  Blai Bonet,et al.  A Concise Introduction to Models and Methods for Automated Planning , 2013, A Concise Introduction to Models and Methods for Automated Planning.

[6]  Blai Bonet,et al.  Automatic Derivation of Memoryless Policies and Finite-State Controllers Using Classical Planners , 2009, ICAPS.

[7]  Marc G. Bellemare,et al.  Count-Based Exploration with Neural Density Models , 2017, ICML.

[8]  Bernhard Nebel,et al.  In Defense of PDDL Axioms , 2003, IJCAI.

[9]  Blai Bonet,et al.  Learning Features and Abstract Actions for Computing Generalized Plans , 2018, AAAI.

[10]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents , 2012, J. Artif. Intell. Res..

[11]  Blai Bonet,et al.  Planning with Pixels in (Almost) Real Time , 2018, AAAI.

[12]  Hector J. Levesque,et al.  Foundations for Generalized Planning in Unbounded Stochastic Domains , 2016, KR.

[13]  Blai Bonet,et al.  Qualitative Numeric Planning: Reductions and Complexity , 2019, J. Artif. Intell. Res..

[14]  Neil Immerman,et al.  Qualitative Numeric Planning , 2011, AAAI.

[15]  Hector Geffner,et al.  A Polynomial Planning Algorithm that Beats LAMA and FF , 2017, ICAPS.

[16]  Javier Segovia Aguas,et al.  Generalized Planning with Procedural Domain Control Knowledge , 2016, ICAPS.

[17]  Alexei A. Efros,et al.  Curiosity-Driven Exploration by Self-Supervised Prediction , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[18]  Blai Bonet,et al.  Features, Projections, and Representation Change for Generalized Planning , 2018, IJCAI.

[19]  Hector Geffner,et al.  Purely Declarative Action Descriptions are Overrated: Classical Planning with Simulators , 2017, IJCAI.

[20]  Silvia Richter,et al.  The LAMA Planner: Guiding Cost-Based Anytime Planning with Landmarks , 2010, J. Artif. Intell. Res..

[21]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[22]  Kenneth O. Stanley,et al.  Evolving a diversity of virtual creatures through novelty search and local competition , 2011, GECCO '11.

[23]  Yuxiao Hu,et al.  Generalized Planning: Synthesizing Plans that Work for Multiple Environments , 2011, IJCAI.

[24]  Paolo Traverso,et al.  Automated Planning and Acting , 2016 .

[25]  Kenneth O. Stanley,et al.  Abandoning Objectives: Evolution Through the Search for Novelty Alone , 2011, Evolutionary Computation.

[26]  Hector Geffner,et al.  Classical Planning with Simulators: Results on the Atari Video Games , 2015, IJCAI.

[27]  Hector Geffner,et al.  Width and Serialization of Classical Planning Problems , 2012, ECAI.

[28]  Hector Geffner,et al.  Purely Declarative Action Representations are Overrated : Classical Planning with Simulators , 2017 .

[29]  Filip De Turck,et al.  #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning , 2016, NIPS.