Macro and Micro Reinforcement Learning for Playing Nine-ball Pool

We present a method of training a reinforcement learning agent to play nine-ball pool. The training process uses a combination of reinforcement learning, deep neural networks and search trees. These technologies have achieved tremendous results in discrete strategy board games, and we extend their applications to pool games, which is a complicated continuous case. Pool types of games have a huge action space, to improve the efficiency of exploration, we use a macro and micro action framework to combine reinforcement learning and the search tree. The agent learns skills such as choosing pockets and control the post-collision position. Our method shows the potential to solve billiards planning problems through AI.