Genetic Team Composition and Level of Selection in the Evolution of Cooperation

In cooperative multiagent systems, agents interact to solve tasks. Global dynamics of multiagent teams result from local agent interactions, and are complex and difficult to predict. Evolutionary computation has proven a promising approach to the design of such teams. The majority of current studies use teams composed of agents with identical control rules (ldquogenetically homogeneous teamsrdquo) and select behavior at the team level (ldquoteam-level selectionrdquo). Here we extend current approaches to include four combinations of genetic team composition and level of selection. We compare the performance of genetically homogeneous teams evolved with individual-level selection, genetically homogeneous teams evolved with team-level selection, genetically heterogeneous teams evolved with individual-level selection, and genetically heterogeneous teams evolved with team-level selection. We use a simulated foraging task to show that the optimal combination depends on the amount of cooperation required by the task. Accordingly, we distinguish between three types of cooperative tasks and suggest guidelines for the optimal choice of genetic team composition and level of selection.

[1]  Lincoln Smith,et al.  Evolving teamwork and role-allocation with real robots , 2002 .

[2]  Kagan Tumer,et al.  Efficient Evaluation Functions for Multi-rover Systems , 2004, GECCO.

[3]  Risto Miikkulainen,et al.  Neuroevolution for adaptive teams , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[4]  Sean Luke,et al.  Genetic Programming Produced Competitive Soccer Softbot Teams for RoboCup97 , 1998 .

[5]  D. Floreano,et al.  Evolutionary Conditions for the Emergence of Communication in Robots , 2007, Current Biology.

[6]  Vidroha Debroy,et al.  Genetic Programming , 1998, Lecture Notes in Computer Science.

[7]  Kenneth A. De Jong,et al.  Evolving Behaviors for Cooperating Agents , 2000, ISMIS.

[8]  Alan C. Schultz,et al.  Heterogeneity in the Coevolved Behaviors of Mobile Robots: The Emergence of Specialists , 2001, IJCAI.

[9]  Sean Luke,et al.  Cooperative Multi-Agent Learning: The State of the Art , 2005, Autonomous Agents and Multi-Agent Systems.

[10]  Dante Augusto Couto Barone,et al.  Predation: an approach to improving the evolution of real robots with a distributed evolutionary controller , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[11]  Domenico Parisi,et al.  How can we explain the emergence of a language that benefits the hearer but not the speaker? , 2005, Connect. Sci..

[12]  Marco Dorigo,et al.  Cooperative Transport of Objects of Different Shapes and Sizes , 2004, ANTS Workshop.

[13]  James A. Reggia,et al.  Discovery of self-replicating structures using a genetic algorithm , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[14]  Sandip Sen,et al.  Strongly Typed Genetic Programming in Evolving Cooperation Strategies , 1995, ICGA.

[15]  Andrew B. Kahng,et al.  Cooperative Mobile Robotics: Antecedents and Directions , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[16]  Stefano Nolfi,et al.  Evolution of Collective Behavior in a Team of Physically Linked Robots , 2003, EvoWorkshops.

[17]  Robert J. Collins,et al.  AntFarm: Towards Simulated Evolution , 2007 .

[18]  Akira Hara,et al.  Emergence of the cooperative behavior using ADG; Automatically Defined Groups , 1999, GECCO.

[19]  Stewart W. Wilson,et al.  Not) Evolving Collective Behaviours in Synthetic Fish , 1996 .

[20]  C. Baray,et al.  Evolving cooperation via communication in homogeneous multi-agent systems , 1997, Proceedings Intelligent Information Systems. IIS'97.

[21]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[22]  Alex Fukunaga,et al.  Cooperative mobile robotics: antecedents and directions , 1995 .

[23]  J. Tukey,et al.  Variations of Box Plots , 1978 .

[24]  C. Lee Giles,et al.  Talking Helps: Evolving Communicating Agents for the Predator-Prey Pursuit Problem , 2000, Artificial Life.

[25]  Lee Spector,et al.  Emergence of Collective Behavior in Evolving Populations of Flying Agents , 2003, Genetic Programming and Evolvable Machines.

[26]  Eric Bonabeau,et al.  Evolving Ant Colony Optimization , 1998, Adv. Complex Syst..

[27]  Lee Spector,et al.  Evolving teamwork and coordination with genetic programming , 1996 .

[28]  Sandip Sen,et al.  Crossover Operators for Evolving A Team , 1997 .

[29]  W. Hamilton The genetical evolution of social behaviour. II. , 1964, Journal of theoretical biology.

[30]  R. Arkin,et al.  Behavioral diversity in learning robot teams , 1998 .

[31]  Laurent Keller,et al.  Ant-like task allocation and recruitment in cooperative robots , 2000, Nature.

[32]  John S. McCaskill,et al.  Reducing Collective Behavioural Complexity through Heterogeneity , 2000 .

[33]  Michael G. Dyer,et al.  Evolution of herding behavior in artificial animals , 1993 .

[34]  W. Hamilton The genetical evolution of social behaviour. I. , 1964, Journal of theoretical biology.

[35]  C. Hauert,et al.  Replicator dynamics for optional public good games. , 2002, Journal of theoretical biology.

[36]  L. Keller,et al.  The evolution of cooperation and altruism – a general framework and a classification of models , 2006, Journal of evolutionary biology.

[37]  A. Griffin,et al.  Kin selection: fact and fiction , 2002 .

[38]  W. Hamilton,et al.  Altruism and Related Phenomena, Mainly in Social Insects , 1972 .

[39]  R. Paul Wiegand,et al.  An empirical analysis of collaboration methods in cooperative coevolutionary algorithms , 2001 .

[40]  Pattie Maes,et al.  The Evolution of Communication Schemes Over Continuous Channels , 1996 .

[41]  Arvin Agah,et al.  Phylogenetic and Ontogenetic Learning in a Colony of Interacting Robots , 1997, Auton. Robots.

[42]  Jordan B. Pollack,et al.  Embodied Evolution: Distributing an evolutionary algorithm in a population of robots , 2002, Robotics Auton. Syst..

[43]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[44]  Marco Dorigo,et al.  Cooperative hole avoidance in a swarm-bot , 2006, Robotics Auton. Syst..

[45]  Josh C. Bongard,et al.  The Legion System: A Novel Approach to Evolving Hetrogeneity for Collective Problem Solving , 2000, EuroGP.

[46]  Angelo Cangelosi,et al.  The Emergence of a 'Language' in an Evolving Population of Neural Networks , 1998, Connect. Sci..

[47]  Stewart W. Wilson,et al.  Not) Evolving Collective Behaviours in Synthetic Fish , 1996 .

[48]  A. Griffin,et al.  Testing Hamilton's rule with competition between relatives , 2001, Nature.

[49]  J. Traniello Foraging Strategies of Ants , 1989 .

[50]  D. Queller,et al.  Genetic relatedness in viscous populations , 2005, Evolutionary Ecology.

[51]  Tucker R. Balch The impact of diversity on performance in multi-robot foraging , 1999, AGENTS '99.

[52]  Hiroaki Kitano,et al.  RoboCup-98: Robot Soccer World Cup II , 2001, Lecture Notes in Computer Science.

[53]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[54]  David Andre The Automatic Programming of Agents that Learn Mental Models and Create Simple Plans of Action , 1995, IJCAI.

[55]  Gary B. Parker,et al.  Co-Evolving Team Capture Strategies for Dissimilar Robots , 2004, AAAI Technical Report.

[56]  Alcherio Martinoli,et al.  Swarm intelligence in autonomous collective robotics , 1999 .

[57]  Kazuo Tanie,et al.  Robots playing to win: evolutionary soccer strategies , 1997, Proceedings of International Conference on Robotics and Automation.

[58]  Matt Quinn,et al.  A comparison of approaches to the evolution of homogeneous multi-robot teams , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[59]  Lee Spector,et al.  Using Genetic Programming with Multiple Data Types and Automatic Modularization to Evolve Decentralized and Coordinated Navigation in Multi-Agent Systems , 2002, GECCO Late Breaking Papers.

[60]  Annie S. Wu,et al.  Evolving control for distributed micro air vehicles , 1999, Proceedings 1999 IEEE International Symposium on Computational Intelligence in Robotics and Automation. CIRA'99 (Cat. No.99EX375).

[61]  G. M. Werner Evolution of Communication in Artificial Organisms, Artifial Life II , 1991 .

[62]  Stefano Nolfi,et al.  Evolving Mobile Robots Able to Display Collective Behaviors , 2003, Artificial Life.

[63]  Marvin Minsky,et al.  Steps toward Artificial Intelligence , 1995, Proceedings of the IRE.

[64]  James A. Hendler,et al.  Co-evolving Soccer Softbot Team Coordination with Genetic Programming , 1997, RoboCup.

[65]  Joaquin L Reyes Lopez Optimal Foraging in Seed-Harvester Ants: Computer-Aided Simulation. , 1987, Ecology.

[66]  Risto Miikkulainen,et al.  Online Interactive Neuro-evolution , 2000, Neural Processing Letters.

[67]  Craig W. Reynolds An evolved, vision-based behavioral model of coordinated group motion , 1993 .

[68]  G. Caprari Autonomous Micro-Robots: Applications and Limitations , 2003 .

[69]  D. Floreano,et al.  Division of labour and colony efficiency in social insects: effects of interactions between genetic architecture, colony kin structure and rate of perturbations , 2006, Proceedings of the Royal Society B: Biological Sciences.

[70]  Belle M. Nixon Altruism , 1938, Encyclopedia of Social Insects.

[71]  Joaquin L. Reyes OPTIMAL FORAGING IN SEED-HARVESTER ANTS: COMPUTER-AIDED SIMULATION1 , 1987 .

[72]  Stefano Nolfi,et al.  Toward open-ended evolutionary robotics: evolving elementary robotic units able to self-assemble and self-reproduce , 2004, Connect. Sci..

[73]  A. Griffin,et al.  Cooperation and competition in pathogenic bacteria , 2004, Nature.

[74]  Stefano Nolfi,et al.  Coordination and Behaviour Integration in Cooperating Simulated Robots , 2004 .

[75]  Graham Kendall,et al.  Evolving Collective Behavior in an Artificial Ecology , 2001, Artificial Life.

[76]  Lincoln Smith,et al.  Evolving controllers for a homogeneous system of physical robots: structured cooperation with minimal sensors , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[77]  Marco Dorigo,et al.  Evolving Aggregation Behaviors in a Swarm of Robots , 2003, ECAL.

[78]  Dario Floreano,et al.  Influence of the level of polyandry and genetic architecture on division of labour , 2006 .

[79]  TrianniVito,et al.  Evolving Self-Organizing Behaviors for a Swarm-Bot , 2004 .

[80]  John J. Grefenstette,et al.  Credit assignment in rule discovery systems based on genetic algorithms , 1988, Machine Learning.

[81]  Astro Teller,et al.  Evolving Team Darwin United , 1998, RoboCup.

[82]  Marco Dorigo,et al.  Evolving a cooperative transport behavior for two simple robots , 2004 .

[83]  L. Darrell Whitley,et al.  Evolving cooperative strategies for UAV teams , 2005, GECCO '05.

[84]  L. Keller Levels of selection in evolution , 1999 .

[85]  Stefano Nolfi,et al.  Self-Organization of Communication in Evolving Robots , 2006 .

[86]  Luca Maria Gambardella,et al.  Evolving Self-Organizing Behaviors for a Swarm-Bot , 2004, Auton. Robots.

[87]  Risto Miikkulainen,et al.  Real-time neuroevolution in the NERO video game , 2005, IEEE Transactions on Evolutionary Computation.