Reinforcement Learning with Heterogeneous Policy Representations

In Reinforcement Learning (RL) the goal is to find a policy π that maximizes the expected future return, calculated based on a scalar reward function R(·) ∈ R. The policy π determines what actions will be performed by the RL agent. Traditionally, the RL problem is formulated in terms of a Markov Decision Process (MDP) or a Partially Observable MDP (POMDP). In this formulation, the policy π is viewed as a mapping function (π : s 7−→ a) from state s ∈ S to action a ∈ A. This approach, however, suffers severely from the curse of dimensionality.