Multiagent Learning through Neuroevolution

Neuroevolution is a promising approach for constructing intelligent agents in many complex tasks such as games, robotics, and decision making. It is also well suited for evolving team behavior for many multiagent tasks. However, new challenges and opportunities emerge in such tasks, including facilitating cooperation through reward sharing and communication, accelerating evolution through social learning, and measuring how good the resulting solutions are. This paper reviews recent progress in these three areas, and suggests avenues for future work.

[1]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[2]  Dario Floreano,et al.  Evolutionary robots with on-line self-organization and behavioral fitness , 2000, Neural Networks.

[3]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[4]  Risto Miikkulainen,et al.  Evolving Symmetry for Modular System Design , 2011, IEEE Trans. Evol. Comput..

[5]  M. Tomasello.  The Cultural Origins of Human Cognition , 2000 .

[6]  Risto Miikkulainen,et al.  OpenNERO: A Game Platform for AI Research and Education , 2008, AIIDE.

[7]  G. Simpson.  THE BALDWIN EFFECT , 1953 .

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  M. Nowak.  Five Rules for the Evolution of Cooperation , 2006, Science.

[10]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1998 .

[11]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[12]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[13]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[14]  N. Humphrey.  The Social Function of Intellect , 1976 .

[15]  Douglas Thain,et al.  Distributed computing in practice: the Condor experience , 2005, Concurr. Pract. Exp..

[16]  Wolfgang Banzhaf,et al.  Advances in Artificial Life , 2003, Lecture Notes in Computer Science.

[17]  Risto Miikkulainen,et al.  Evolving a real-world vehicle warning system , 2006, GECCO.

[18]  Risto Miikkulainen,et al.  Coevolution of Role-Based Cooperation in Multiagent Systems , 2009, IEEE Transactions on Autonomous Mental Development.

[19]  Thomas Stützle,et al.  Incremental Social Learning in Particle Swarms , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  Risto Miikkulainen,et al.  Forming Neural Networks Through Efficient and Adaptive Coevolution , 1997, Evolutionary Computation.

[21]  Dario Floreano,et al.  Effects of Group Composition and Level of Selection in the Evolution of Cooperation in Artificial Ants , 2003, ECAL.

[22]  R. Byrne,et al.  Machiavellian intelligence : social expertise and the evolution of intellect in monkeys, apes, and humans , 1990 .

[23]  C. Ember.  Hierarchy in the Forest: The Evolution of Egalitarian Behavior , 2001 .

[24]  Toshio Fukuda,et al.  Virus-evolutionary genetic algorithm-coevolution of planar grid model , 1996, Proceedings of IEEE 5th International Fuzzy Systems.

[25]  Risto Miikkulainen,et al.  Evolving agent behavior in multiobjective domains using fitness-based shaping , 2010, GECCO '10.

[26]  A. E. Eiben,et al.  Social learning in Population-based Adaptive Systems , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[27]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[28]  R. Reynolds.  AN INTRODUCTION TO CULTURAL ALGORITHMS , 2008 .

[29]  M. Tomasello,et al.  Humans Have Evolved Specialized Skills of Social Cognition: The Cultural Intelligence Hypothesis , 2007, Science.

[30]  D. Floreano,et al.  Division of labour and colony efficiency in social insects: effects of interactions between genetic architecture, colony kin structure and rate of perturbations , 2006, Proceedings of the Royal Society B: Biological Sciences.

[31]  Risto Miikkulainen,et al.  Active Guidance for a Finless Rocket Using Neuroevolution , 2003, GECCO.

[32]  Domenico Parisi,et al.  Cultural Evolution in a Population of Neural Networks , 1997 .

[33]  Domenico Parisi,et al.  Cultural evolution in neural networks , 1997 .

[34]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[35]  Evert Haasdijk,et al.  Modeling Social Learning of Language and Skills , 2010, Artificial Life.

[36]  Risto Miikkulainen,et al.  Real-time neuroevolution in the NERO video game , 2005, IEEE Transactions on Evolutionary Computation.

[37]  J. Stevenson.  The cultural origins of human cognition , 2001 .

[38]  Hiromi Seno,et al.  A density-dependent diffusion model of shoaling of nesting fish , 1990 .

[39]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[40]  L. Buşoniu.  Evolutionary function approximation for reinforcement learning , 2006 .

[41]  W. Oechel,et al.  Automatic design and manufacture of robotic lifeforms , 2022 .

[42]  Stefano Nolfi,et al.  Social learning and cultural evolution in embodied and situated agents , 2007, 2007 IEEE Symposium on Artificial Life.

[43]  David B. Fogel,et al.  Blondie24: Playing at the Edge of AI , 2001 .

[44]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[45]  Geoffrey I. Webb,et al.  Encyclopedia of Machine Learning , 2011, Encyclopedia of Machine Learning.

[46]  Domenico Parisi,et al.  Cultural Transmission Between and Within Generations , 2006, J. Artif. Soc. Soc. Simul..

[47]  Michael D. Thomure,et al.  The Role of Space in the Success of Coevolutionary Learning , 2006 .

[48]  Risto Miikkulainen,et al.  Evolving Multimodal Networks for Multitask Games , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[49]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[50]  Olympia Roeva,et al.  Multiple model approach to modelling of Escherichia coli fed-batch cultivation extracellular production of bacterial phytase , 2007 .

[51]  T. Vicsek,et al.  Collective behavior of interacting self-propelled particles , 2000, cond-mat/0611742.

[52]  Risto Miikkulainen,et al.  Constructing competitive and cooperative agent behavior using coevolution , 2010, CIG.

[53]  P. Kleingeld,et al.  The Stanford Encyclopedia of Philosophy , 2013 .