Preparation behaviour synthesis with reinforcement learning

When humans perform a series of motions, they prepare for the next motion in advance so as to enhance the response time of their movements. This kind of preparation behaviour results in a natural and smooth transition of the overall movement. In this paper, we propose a new method to synthesize the behaviour using reinforcement learning. To create preparation movements, we propose a customized motion blending algorithm that is governed by a single numerical value, which we called the level of preparation. During the offline process, the system learns the optimal way to approach a target, as well as the realistic behaviour to prepare for interaction considering the level of preparation. At run-time, the trained controller indicates the character to move to a target with the appropriate level of preparation, resulting in human-like movements. We synthesized scenes in which the character has to move in a complex environment and interact with objects, such as a character crawling under and jumping over obstacles while walking. The method is useful not only for computer animation, but also for real-time applications such as computer games, in which the characters need to accomplish a series of tasks in a given environment.