Glue: Enhancing Compatibility and Flexibility of Reinforcement Learning Platforms by Decoupling Algorithms and Environments

Reinforcement Learning (RL) platforms play an important role in translating the rapid advances of RL algorithms into the successes of real-world tasks. These platforms integrate multiple simulation environments, allowing testing, evaluating and finally applying RL algorithms in different scenarios. However, the algorithm code is required to execute in the same runtime system with the underlying environments, which limits platforms’ compatibility when adapting an algorithm and flexibility when switching between different algorithms. We propose GLUE to resolve this issue, by decoupling the executions of algorithms and environments first, then leveraging the RPC protocol to orchestrate a seamless workflow between them. GLUE is further implemented as a library, which hides the handling of language-specific RPCs from users. We evaluate GLUE by adapting 6 RL algorithm implementations to a representative RL platform. Compared with the baseline approach, GLUE enables algorithms to achieve competitive performance, but reduces lines of algorithm code to be changed in adaption by 27.77% , at the cost of 5.40% longer training time, on average.