Hierarchical Reinforcement Learning and Parallel Computing Applied to the k-server Problem

In this paper was proposed an algorithm based on Hierarchical Reinforcement Learning (HRL) and Parallel Computing to solve an online computing problem, the K-Server Problem (KSP). The size of the storage structure used for reinforcement learning to obtain the optimal policy grows exponentially with the number of states and actions, limiting its use to smaller problems due to the curse of dimensionality. The problem is modeled as a multiple steps decision process computed in parallel by applying the Q-learning algorithm to obtain optimal policies in a reduced number of nodes obtained from an clustering process. The results show the applicability of the proposed method to real problems of large size.