Multiagent Reinforcement Learning Algorithm Research Based on Non Markov Environment

In this paper several multiagent reinforcement learning algorithms are investigated, compared and analyzed. An effective reinforcement learning algorithm based on non Markov environment is proposed. This algorithm uses linear programming to find the best-response policy, and avoids solving multiple Nash equilibria problem. The algorithm involves simple procedures and easy computations, and can guarantee good learning convergence in some situations. Experiment results show that this algorithm is effective. Keyword: multiagent; reinforcement learning; markov environment; nash equilibria

[1]  Michael L. Littman,et al.  Friend-or-Foe Q-learning in General-Sum Games , 2001, ICML.

[2]  Keith B. Hall,et al.  Correlated Q-Learning , 2003, ICML.

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Reda Alhajj,et al.  Modular fuzzy-reinforcement learning approach with internal model capabilities for multiagent systems , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[6]  Kazushi Murakoshi,et al.  A parameter control method in reinforcement learning to rapidly follow unexpected environmental changes. , 2004, Bio Systems.

[7]  Yutaka Sakaguchi,et al.  Reliability of internal prediction/estimation and its application. I. Adaptive action selection reflecting reliability of value function , 2004, Neural Networks.

[8]  Nikos Vlassis,et al.  A Concise Introduction to Multiagent Systems and Distributed AI , 2003 .

[9]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[10]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[11]  Manuela M. Veloso,et al.  Multiagent learning using a variable learning rate , 2002, Artif. Intell..

[12]  Manuela M. Veloso,et al.  Rational and Convergent Learning in Stochastic Games , 2001, IJCAI.

[13]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[14]  Yoav Shoham,et al.  Multi-Agent Reinforcement Learning:a critical survey , 2003 .