Reinforcement Learning of Competitive Skills with Soccer Agents

Reinforcement learning plays an important role in Multi-Agent Systems. The reasoning and learning ability of agents is the key for autonomous agents. Autonomous agents are required to be able to adapt and learn in uncertain environments via communication and collaboration (in both competitive and cooperative situations). For real-time, non-deterministic and dynamic systems, it is often extremely complex and difficult to formally verify their properties a priori. In this paper, we adopt the reinforcement learning algorithms to verify goal-oriented agents' competitive and cooperative learning abilities for decision making. In doing so, a simulation testbed is applied to test the learning algorithms in the specified scenarios. In addition, the function approximation technique known as tile coding (TC), is used to generate value functions, which can avoid the value function growing exponentially with the number of the state values.

[1]  Jean-Daniel Zucker,et al.  Abstraction, Reformulation and Approximation, 6th International Symposium, SARA 2005, Airth Castle, Scotland, UK, July 26-29, 2005, Proceedings , 2005, SARA.

[2]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Sergey Serebryakov,et al.  Data Mining Techniques for RoboCup Soccer Agents , 2005, AIS-ADM.

[5]  Nicholas R. Jennings,et al.  Intelligent agents: theory and practice , 1995, The Knowledge Engineering Review.

[6]  Martin A. Riedmiller,et al.  Karlsruhe Brainstormers - A Reinforcement Learning Approach to Robotic Soccer , 2000, RoboCup.

[7]  Peter Stone,et al.  Keepaway Soccer: From Machine Learning Testbed to Benchmark , 2005, RoboCup.

[8]  Peter Stone,et al.  Function Approximation via Tile Coding: Automating Parameter Choice , 2005, SARA.

[9]  Lakhmi C. Jain,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2004, Lecture Notes in Computer Science.

[10]  A. Roadmapof A Roadmap of Agent Research and Development , 1995 .

[11]  Gregory Kuhlmann and Peter Stone and Raymond J. Mooney and Shavlik Guiding a Reinforcement Learner with Natural Language Advice: Initial Results in RoboCup Soccer , 2004, AAAI 2004.

[12]  Richard S. Sutton,et al.  Reinforcement learning with replacing eligibility traces , 2004, Machine Learning.

[13]  P. Dayan,et al.  TD(λ) converges with probability 1 , 2004, Machine Learning.

[14]  Michael Wooldridge,et al.  Applications of intelligent agents , 1998 .

[15]  Jiming Liu,et al.  Autonomous Intelligent Systems: Agents and Data Mining: International Workshop, AIS-ADM 2005, St. Petersburg, Russia, June 6-8, 2005. Proceedings , 2005, AIS-ADM.

[16]  Peter Stone,et al.  Layered learning in multiagent systems - a winning approach to robotic soccer , 2000, Intelligent robotics and autonomous agents.

[17]  Manuela M. Veloso,et al.  Team-partitioned, opaque-transition reinforcement learning , 1999, AGENTS '99.

[18]  Peter Stone,et al.  Keepaway Soccer: A Machine Learning Testbed , 2001, RoboCup.

[19]  John N. Tsitsiklis,et al.  Asynchronous stochastic approximation and Q-learning , 1994, Mach. Learn..

[20]  Lakhmi C. Jain,et al.  Teamwork and Simulation in Hybrid Cognitive Architecture , 2006, KES.