Generation of locally optimal trajectories against moving obstacles using Gaussian sampling

Differential Dynamic Programming (DDP) can effectively solve an optimal control problem; however, it cannot deal with temporally changing environments such as an appearance of a moving obstacle. In this paper, we present the segmentation of locally optimal trajectories under an environment with a moving obstacle. The agent finds locally optimal trajectories by sampling Gaussian samples according to its existing incomplete policy. After one episode of the agent movement is over and if this episode performs better than the previous one, the policy is reinforced by learning the trajectories of the episode. We show that the algorithm successfully generates the locally optimal trajectories to avoid moving obstacles, and the performance of the resulting policy is improved as the episode progresses. These results would help apply reinforcement learning to robotics in two respects: learning the policy with a small number of iteration by reusing DDP policy, and taking action against changing environments. Because a real-world robot has to deal with a variant environment and has a limit of iterations for policy learning, these two results would help to settle the reinforcement learning issues for robotics.