Training a robust reinforcement learning controller for the uncertain system based on policy gradient method

Abstract The target of this paper is to design a model-free robust controller for uncertain systems. The uncertainties of the control system mainly consists of model uncertainty and external disturbance, which widely exist in the practical utilization. These uncertainties will negatively influence the system performance and this motivates us to train a model-free controller to solve this problem. Reinforcement learning is an important branch of machine learning and is able to achieve well performed control results by optimizing a policy without the knowledge of mathematical plant model. In this paper, we construct a reward function module to describe the specific environment of the concerned system, taking uncertainties into account. Then we utilize a new policy gradient method to optimize the policy and implement this algorithm with the actor-critic structure neuro networks. These two networks are our reinforcement learning controllers. Finally, we illustrate the applicability and efficiency of the proposed method by applying it on an experimental helicopter platform model, which includes model uncertainties and external disturbances.