Layered Approach to Learning Client Behaviors in the Robocup Soccer Server

In the past few years, multiagent systems (MAS) have emerged as an active subfield of artificial intelligence (AI). Because of the inherent complexity of MAS, there is much interest in using machine learning (ML) techniques to help build multiagent systems. Robotic soccer is a particularly good domain for studying MAS and multiagent learning. Our approach to using ML as a tool for building Soccer Server clients involves layering increasingly complex learned behaviors. In this article, we describe two levels of learned behaviors. First, the clients learn a low-level individual skill that allows them to control the ball effectively. Then, using this learned skill, they learn a higher level skill that involves multiple players. For both skills, we describe the learning method in detail and report on our extensive empirical testing. We also verify empirically that the learned skills are applicable to game situations.