COOPERATIVE COEVOLUTION OF MULTI-AGENT SYSTEMS

In certain tasks such as pursuit and evasion, multiple agent s need to coordinate their behavior to achieve a common goal. An interesting question is, how can such behav ior best be evolved? When the agents are controlled with neural networks, a powerful method is to coevolve them in separate subpopulations, and test together in the common task. In this paper, such a met hod, called Multi-Agent ESP (Enforced Subpopulations) is presented, and demonstrated in a prey-c apture task. The approach is shown more efficient and robust than evolving a single central controller for all agents. The role of communication in such domains is also studied, and shown to be unnecessary and even trimental if effective behavior in the task can be expressed as role-based cooperation rather than sync hronization.

[1]  Zbigniew Michalewicz,et al.  Evolutionary Computation 1 , 2018 .

[2]  L. D. Whitley,et al.  Genetic Reinforcement Learning for Neurocontrol Problems , 2004, Machine Learning.

[3]  Lashon B. Booker,et al.  Proceedings of the fourth international conference on Genetic algorithms , 1991 .

[4]  Alan S. Perelson,et al.  Searching for Diverse, Cooperative Populations with Genetic Algorithms , 1993, Evolutionary Computation.

[5]  Tucker Balch,et al.  Learning Roles: Behavioral Diversity in Robot Teams , 1997 .

[6]  G. M. Werner.  Evolution of Communication in Artificial Organisms, Artifial Life II , 1991 .

[7]  Sandip Sen,et al.  Crossover Operators for Evolving A Team , 1997 .

[8]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[9]  Kyle Wagner,et al.  Cooperative Strategies and the Evolution of Communication , 2000, Artificial Life.

[10]  L. Darrell Whitley,et al.  Genetic Reinforcement Learning with Multilayer Neural Networks , 1991, ICGA.

[11]  M. Benda,et al.  On Optimal Cooperation of Knowledge Sources , 1985 .

[12]  L. Darrell Whitley,et al.  Cellular Encoding Applied to Neurocontrol , 1995, ICGA.

[13]  Heinz Mühlenbein,et al.  The parallel genetic algorithm as function optimizer , 1991, Parallel Comput..

[14]  Mark D. Pendrith.  On Reinforcement Learning of Control Actions in Noisy and Non-Markovian Domains , 1994 .

[15]  A. Rosenfeld,et al.  IEEE TRANSACTIONS ON SYSTEMS , MAN , AND CYBERNETICS , 2022 .

[16]  Larry D. Pyeatt,et al.  A comparison between cellular encoding and direct encoding for genetic neural networks , 1996 .

[17]  David E. Moriarty,et al.  Symbiotic Evolution of Neural Networks in Sequential Decision Tasks , 1997 .

[18]  Samir W. Mahfoud.  Niching methods for genetic algorithms , 1996 .

[19]  Risto Miikkulainen,et al.  Forming Neural Networks Through Efficient and Adaptive Coevolution , 1997, Evolutionary Computation.

[20]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[21]  Tony Savage,et al.  Shaping: The Link Between Rats and Robots , 1998, Connect. Sci..

[22]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[23]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning through Symbiotic Evolution , 1996, Machine Learning.

[24]  Kee-Eung Kim,et al.  Learning Finite-State Controllers for Partially Observable Environments , 1999, UAI.

[25]  Sandip Sen,et al.  Evolving Beharioral Strategies in Predators and Prey , 1995, Adaption and Learning in Multi-Agent Systems.

[26]  J. Batali,et al.  Innate biases and critical periods: Combining evolution and learning in the acquisition of syntax , 1994 .

[27]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[28]  C.W. Anderson,et al.  Learning to control an inverted pendulum using neural networks , 1989, IEEE Control Systems Magazine.

[29]  Sandip Sen,et al.  Co-adaptation in a Team , 1997 .