Solving Multi-agent Control Problems Using Particle Swarm Optimization

This paper outlines an approximate algorithm for finding an optimal decentralized control in multi-agent systems. Decentralized partially observable Markov decision processes and their extension to infinite state, observation and action spaces are utilized as a theoretical framework. In the presented algorithm, policies of each agent are represented by a feedforward neural network. Then, a search is performed in a joint weight space of all networks. Particle swarm optimization is applied as a search algorithm. Experimental results are provided showing that the algorithm finds good solutions for the classical Tiger problem extended to multi-agent systems, as well as for a multi-agent navigation task involving large state and action spaces

[1]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization approaches to coevolve strategies for the iterated prisoner's dilemma , 2005, IEEE Transactions on Evolutionary Computation.

[2]  S. Zilberstein,et al.  Formal Models and Algorithms for Decentralized Control of Multiple Agents Technical Report UM-CS-2005-068 , 2005 .

[3]  A.P. Engelbrecht,et al.  Learning to play games using a PSO-based competitive learning approach , 2004, IEEE Transactions on Evolutionary Computation.

[4]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[5]  Shlomo Zilberstein,et al.  Dynamic Programming for Partially Observable Stochastic Games , 2004, AAAI.

[6]  Makoto Yokoo,et al.  Taming Decentralized POMDPs: Towards Efficient Policy Computation for Multiagent Settings , 2003, IJCAI.

[7]  R. Braun,et al.  A nature inspired multi-agent framework for autonomic service management in ubiquitous computing environments , 2005, 2005 ICSC Congress on Computational Intelligence Methods and Applications.

[8]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[9]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[10]  Sean Luke,et al.  Cooperative Multi-Agent Learning: The State of the Art , 2005, Autonomous Agents and Multi-Agent Systems.

[11]  Paulo Cortez,et al.  Particle swarms for feedforward neural network training , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[12]  Hyeong Soo Chang An adaptation of particle swarm optimization for Markov decision processes , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[13]  Craig Boutilier,et al.  Sequential Optimality and Coordination in Multiagent Systems , 1999, IJCAI.

[14]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[15]  Kee-Eung Kim,et al.  Learning to Cooperate via Policy Search , 2000, UAI.

[16]  Claudia V. Goldman,et al.  Decentralized Control of Cooperative Systems: Categorization and Complexity Analysis , 2004, J. Artif. Intell. Res..

[17]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..