Options in Readylog Reloaded - Generating Decision-Theoretic Plan Libraries in Golog

Readylog is a logic-based agent programming language and combines many important features from other Golog dialects. One of the features of Readylog is to make use of decision-theoretic planning for specifying the behavior of an agent or robot. In this paper we show a method to reduce the planning time for decision-theoretic planning in the Readylog framework. Instead of planning policies on the fly over and over again, we calculate an abstract policy once and store it in a plan library. This policy can later be re-instantiated. With this plan library the on-line planning time can be significantly reduced. We compare computing policies on the fly with those stored in our plan library with examples from the robotic soccer domain. In the 2D soccer simulation league we show the significant speed-up when using our plan library approach. Moreover, the use of the plan library together with a suitable state space abstraction for the soccer domain makes it possible to apply macro-actions in an otherwise continuous domain.

[1]  Alexander Ferrein,et al.  Football is coming home , 2006, PCAR '06.

[2]  Michael Thielscher,et al.  Under Consideration for Publication in Theory and Practice of Logic Programming Flux: a Logic Programming Method for Reasoning Agents , 2003 .

[3]  Mikhail Soutchanski,et al.  High-level robot programming in dynamic and incompletely known environments , 2005 .

[4]  Gerhard Lakemeyer,et al.  ccGolog -- A Logical Language Dealing with Continuous Change , 2003, Log. J. IGPL.

[5]  Alexander Ferrein,et al.  Using Golog for Deliberation and Team Coordination in Robotic Soccer , 2005, Künstliche Intell..

[6]  Raymond Reiter,et al.  Knowledge in Action: Logical Foundations for Specifying and Implementing Dynamical Systems , 2001 .

[7]  J. McCarthy.  Situations, Actions, and Causal Laws , 1963 .

[8]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[9]  Ian Frank,et al.  Soccer Server: A Tool for Research on Multiagent Systems , 1998, Appl. Artif. Intell..

[10]  Henrik Grosskreutz,et al.  Probabilistic Projection and Belief Update in the pGOLOG Framework , 2000, GI Jahrestagung.

[11]  Craig Boutilier,et al.  Decision-Theoretic, High-Level Agent Programming in the Situation Calculus , 2000, AAAI/IAAI.

[12]  Alexander Ferrein,et al.  Qualitative World Models for Soccer Robots , 2006 .

[13]  Alexander Ferrein,et al.  Extending DTGOLOG with Options , 2003, IJCAI.

[14]  Raymond Reiter,et al.  Logical Foundations for Cognitive Agents: Contributions in Honor of Ray Reiter , 2001 .

[15]  John G. Gibbons.  Knowledge in Action , 2001 .

[16]  Hector J. Levesque,et al.  An Incremental Interpreter for High-Level Programs with Sensing , 1999 .

[17]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[18]  Milos Hauskrecht,et al.  Hierarchical Solution of Markov Decision Processes using Macro-actions , 1998, UAI.

[19]  Alexander Ferrein,et al.  On-Line Decision-Theoretic Golog for Unpredictable Domains , 2004, KI.

[20]  Hector J. Levesque,et al.  ConGolog, a concurrent programming language based on the situation calculus , 2000, Artif. Intell..

[21]  Hector J. Levesque,et al.  GOLOG: A Logic Programming Language for Dynamic Domains , 1997, J. Log. Program..