Co-Operative Strategy for an Interactive Robot Soccer System by Reinforcement Learning Method

This paper presents a cooperation strategy between a human operator and autono- mous robots for an interactive robot soccer game. The interactive robot soccer game has been developed to allow humans to join into the game dynamically and reinforce entertainment char- acteristics. In order to make these games more interesting, a cooperation strategy between hu- mans and autonomous robots on a team is very important. Strategies can be pre-programmed or learned by robots themselves with learning or evolving algorithms. Since the robot soccer sys- tem is hard to model and its environment changes dynamically, it is very difficult to pre-program cooperation strategies between robot agents. Q-learning - one of the most representative rein- forcement learning methods - is shown to be effective for solving problems dynamically without explicit knowledge of the system. Therefore, in our research, a Q-learning based learning method has been utilized. Prior to utilizing Q-learning, state variables describing the game situa- tion and actions' sets of robots have been defined. After the learning process, the human opera- tor could play the game more easily. To evaluate the usefulness of the proposed strategy, some simulations and games have been carried out.