Relevance Grounding for Planning in Relational Domains

Probabilistic relational models are an efficient way to learn and represent the dynamics in realistic environments consisting of many objects. Autonomous intelligent agents that ground this representation for all objects need to plan in exponentially large state spaces and large sets of stochastic actions. A key insight for computational efficiency is that successful planning typically involves only a small subset of relevant objects. In this paper, we introduce a probabilistic model to represent planning with subsets of objects and provide a definition of object relevance. Our definition is sufficient to prove consistency between repeated planning in partially grounded models restricted to relevant objects and planning in the fully grounded model. We propose an algorithm that exploits object relevance to plan efficiently in complex domains. Empirical results in a simulated 3D blocksworld with an articulated manipulator and realistic physics prove the effectiveness of our approach.

[1]  L. P. Kaelbling,et al.  Learning Symbolic Models of Stochastic Domains , 2007, J. Artif. Intell. Res..

[2]  Marc Toussaint,et al.  Approximate inference for planning in stochastic relational worlds , 2009, ICML '09.

[3]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[4]  Leslie Pack Kaelbling,et al.  Envelope-based Planning in Relational MDPs , 2003, NIPS.

[5]  R. Berndt,et al.  Working memory retention systems: a state of activated long-term memory. , 2003, The Behavioral and brain sciences.

[6]  Craig Boutilier,et al.  Decision-Theoretic Planning: Structural Assumptions and Computational Leverage , 1999, J. Artif. Intell. Res..

[7]  Craig Boutilier,et al.  Symbolic Dynamic Programming for First-Order MDPs , 2001, IJCAI.

[8]  Matthew Botvinick,et al.  Goal-directed decision making in prefrontal cortex: a computational framework , 2008, NIPS.

[9]  Scott Sanner,et al.  Approximate Solution Techniques for Factored First-Order MDPs , 2007, ICAPS.

[10]  Maurice Bruynooghe,et al.  Online Learning and Exploiting Relational Models in Reinforcement Learning , 2007, IJCAI.

[11]  Peter Geibel,et al.  Learning Models of Relational MDPs Using Graph Kernels , 2007, MICAI.

[12]  Daniel S. Weld Recent Advances in AI Planning , 1999, AI Mag..

[13]  Luc De Raedt,et al.  Bellman goes relational , 2004, ICML.

[14]  A. Baddeley The episodic buffer: a new component of working memory? , 2000, Trends in Cognitive Sciences.

[15]  Leslie Pack Kaelbling,et al.  Action-Space Partitioning for Planning , 2007, AAAI.

[16]  Marc Toussaint,et al.  Probabilistic inference for solving discrete and continuous state Markov Decision Processes , 2006, ICML.

[17]  Leslie Pack Kaelbling,et al.  Adaptive Envelope MDPs for Relational Equivalence-based Planning , 2008 .

[18]  Martijn van Otterlo,et al.  The Logic of Adaptive Behavior - Knowledge Representation and Algorithms for Adaptive Sequential Decision Making under Uncertainty in First-Order and Relational Domains , 2009, Frontiers in Artificial Intelligence and Applications.

[19]  Kurt Driessens,et al.  Relational Reinforcement Learning , 1998, Machine-mediated learning.

[20]  Thomas Gärtner,et al.  Graph kernels and Gaussian processes for relational reinforcement learning , 2006, Machine Learning.