Learning Sequences of Compatible Actions Among Agents

Action coordination in multiagent systemsis a difficult task especially in dynamicenvironments. If the environment possessescooperation, least communication,incompatibility and local informationconstraints, the task becomes even moredifficult. Learning compatible action sequencesto achieve a designated goal under theseconstraints is studied in this work. Two newmultiagent learning algorithms called QACE andNoCommQACE are developed. To improve theperformance of the QACE and NoCommQACEalgorithms four heuristics, stateiteration, means-ends analysis, decreasing reward and do-nothing, aredeveloped. The proposed algorithms are testedon the blocks world domain and the performanceresults are reported.

[1]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[2]  Sandip Sen,et al.  Individual learning of coordination knowledge , 1998, J. Exp. Theor. Artif. Intell..

[3]  Ville Könönen.  Multiagent reinforcement learning in Markov games : asymmetric and symmetric approaches , 2004 .

[4]  Reda Alhajj,et al.  Multiagent reinforcement learning using function approximation , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[5]  Gerhard Weiss,et al.  Multiagent systems: a modern approach to distributed artificial intelligence , 1999 .

[6]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[7]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[8]  Shashi Shekhar,et al.  A Negotiation Platform for Cooperating Multi-agent Systems , 1993 .

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Satinder Singh,et al.  Learning to Solve Markovian Decision Processes , 1993 .

[11]  David J. C. Mackay,et al.  Introduction to Monte Carlo Methods , 1998, Learning in Graphical Models.

[12]  Gerhard Weiss,et al.  Learning to Coordinate Actions in Multi-Agent-Systems , 1993, IJCAI.

[13]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[14]  Sandip Sen,et al.  Multiagent Coordination with Learning Classifier Systems , 1995, Adaption and Learning in Multi-Agent Systems.

[15]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[16]  Faruk Polat,et al.  A Conflict Resolution-Based Decentralized Multi-Agent Problem Solving Model , 1992, MAAMAW.

[17]  Richard S. Sutton,et al.  Generalization in ReinforcementLearning : Successful Examples UsingSparse Coarse , 1996 .

[18]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[19]  Sandip Sen,et al.  Learning to Coordinate without Sharing Information , 1994, AAAI.

[20]  Munindar P. Singh,et al.  Readings in agents , 1997 .

[21]  Gavin Adrian Rummery.  Problem solving with reinforcement learning , 1995 .

[22]  Reda Alhajj,et al.  Function approximation based multi-agent reinforcement learning , 2000, Proceedings 12th IEEE Internationals Conference on Tools with Artificial Intelligence. ICTAI 2000.