Reinforcement Learning and Function Approximation

Relational reinforcement learning combines traditional reinforcement learning with a strong emphasis on a relational (rather than attribute-value) representation. Earlier work used relational reinforcement learning on a learning version of the classic Blocks World planning problem (a version where the learner does not know what the result of taking an action will be). “Structural” learning results have been obtained, such as learning in a mixed 3–5 block environment and being able to perform in a 3 or 10 block environment. Here we instead take a function approximation approach to reinforcement learning for this same problem. We obtain similar learning accuracies, with much better running times, allowing us to consider much larger problem sizes. For instance, we can train on a mix of 3–7 blocks and then perform well on worlds with 100–800 blocks—using less running time than the relational method required for 3–10 blocks.