Learning to Plan with Portable Symbols

We present a framework for autonomously learning a portable symbolic representation that describes a collection of low-level continuous environments. We show that abstract representations can be learned in a task-independent space specific to the agent that, when combined with problem-specific information, can be used for planning. We demonstrate knowledge transfer in a video game domain where an agent learns portable, task-independent symbolic rules, and then learns instantiations of these rules on a pertask basis, reducing the number of samples required to learn a representation of a new task.

[1]  Rob Fergus,et al.  Composable Planning with Attributes , 2018, ICML.

[2]  Leslie Pack Kaelbling,et al.  From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning , 2018, J. Artif. Intell. Res..

[3]  George Konidaris,et al.  Active Exploration for Learning Symbolic Representations , 2017, NIPS.

[4]  Leslie Pack Kaelbling,et al.  Learning composable models of parameterized skills , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Leslie Pack Kaelbling,et al.  Symbol Acquisition for Probabilistic High-Level Planning , 2015, IJCAI.

[6]  Justus H. Piater,et al.  Bottom-up learning of object categories, action effects and logical rules: From continuous manipulative exploration to symbolic planning , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Marc Toussaint,et al.  Learning Grounded Relational Symbols from Continuous Data for Abstract Reasoning , 2013 .

[8]  Andrew G. Barto,et al.  Transfer in Reinforcement Learning via Shared Features , 2012, J. Mach. Learn. Res..

[9]  Michael L. Littman,et al.  Efficient Reinforcement Learning with Relocatable Action Models , 2007, AAAI.

[10]  Stefan Edelkamp,et al.  Automated Planning: Theory and Practice , 2007, Künstliche Intell..

[11]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[12]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[13]  Leslie Pack Kaelbling,et al.  Learning Planning Rules in Noisy Stochastic Worlds , 2005, AAAI.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  Leslie Pack Kaelbling,et al.  Learning Probabilistic Relational Planning Rules , 2004, ICAPS.

[16]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[17]  Håkan L. S. Younes,et al.  PPDDL 1 . 0 : An Extension to PDDL for Expressing Planning Domains with Probabilistic Effects , 2004 .

[18]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[19]  Doina Precup,et al.  Temporal abstraction in reinforcement learning , 2000, ICML 2000.

[20]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[21]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[22]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[23]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .