Adversarial Reinforcement Learning

Reinforcement Learning has been used for a number of years in single agent environments. This article reports on our investigation of Reinfor cement Learning techniques in a multi-agent and adversarial environment with continuous o b ervable state information. We introduce a new framework, two-player hexagonal grid soccer, in which to evaluate algorithms. We then compare the performance of several single-agent Rei forcement Learning techniques in that environment. These are further compared to a previou sly developed adversarial Reinforcement Learning algorithm designed for Markov games. Bu ilding upon these efforts, we introduce new algorithms to handle the multi-agent, the adv ersarial, and the continuous-valued aspects of the domain. We introduce a technique for modellin g the opponent in an adversarial game. We introduce an extension to Prioritized Sweeping tha t allows generalization of learnt knowledge over neighboring states in the domain; and we intr oduce an extension to the U Tree generalizing algorithm that allows the handling of continu ous state spaces. Extensive empirical evaluation is conducted in the grid soccer domain.