Evolving goal-driven multi-agent communication: what, when, and to whom

This paper presents an evolutionary approach that, given a performance goal, produces a communication strategy that can improve a multi-agent system’s performance with respect to the desired goal. The evolved strategy determines what, when, and to whom agents communicate. The proposed approach further enables tuning the trade-off between the performance goal and communication cost, to produce a strategy that achieves a good balance between the two objectives, according the system’s designer needs. Experiments are designed to evaluate the approach using the Wumpus World application domain, with variations of three factors: fitness parameters (including objectives’ weights and action and communication costs), fitness goal, and simulation environment. Results show that the system’s performance can be highly tuned by controlling communication, and that the presented approach has significant utilization in improving the performance with respect to the goal.

[1]  Dirkjan Bussink,et al.  A Comparison of Language Evolution and Communication Protocols in Multi-agent Systems , 2004 .

[2]  Shlomo Zilberstein,et al.  Myopic and Non-myopic Communication under Partial Observability , 2009, 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology.

[3]  Yu Zhang Observant and Proactive Communication in Multi-agent Teamwork , 2006, 2006 IEEE/WIC/ACM International Conference on Intelligent Agent Technology.

[4]  Luc Steels,et al.  The Talking Heads experiment: Origins of words and meanings , 2015 .

[5]  Yan Meng,et al.  Communication Scheme Comparison for a Distributed Multi-Agent System , 2008, International Conference on Artificial Intelligence.

[6]  Dirk Thierens,et al.  Selection Schemes, Elitist Recombination, and Selection Intensity , 1997, ICGA.

[7]  Manuela Veloso,et al.  Execution-time communication decisions for coordination of multi-agent teams , 2007 .

[8]  Ronald A. Howard,et al.  Information Value Theory , 1966, IEEE Trans. Syst. Sci. Cybern..

[9]  Stefano Nolfi,et al.  The emergence of communication in evolutionary robots , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[10]  Hitoshi Iba,et al.  Evolving communicating agents based on genetic programming , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[11]  Luc Steels,et al.  The Evolution of Communication Systems by Adaptive Agents , 2002, Adaptive Agents and Multi-Agents Systems.

[12]  Stefano Nolfi,et al.  Origins of Communication in Evolving Robots , 2006, SAB.

[13]  Angelo Cangelosi,et al.  Evolution of communication and language using signals, symbols, and words , 2001, IEEE Trans. Evol. Comput..

[14]  G. M. Werner Evolution of Communication in Artificial Organisms, Artifial Life II , 1991 .

[15]  Risto Miikkulainen,et al.  Evolution of a Communication Code in Cooperative Tasks , 2012, ALIFE.

[16]  Luc Steels,et al.  The Emergence of Embodied Communication in Artificial Agents and Humans , 2008 .

[17]  Koen V. Hindriks,et al.  The Role of Communication in Coordination Protocols for Cooperative Robot Teams , 2014, ICAART.

[18]  Stefano Nolfi,et al.  Emergence of communication and language in evolving robots , 2013 .

[19]  Victor R. Lesser,et al.  Analyzing myopic approaches for multi-agent communication , 2005, IEEE/WIC/ACM International Conference on Intelligent Agent Technology.

[20]  Francisco S. Melo,et al.  A POMDP-based Model for Optimizing Communication in Multiagent Systems , 2011 .

[21]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[22]  Bruce J. MacLennan,et al.  Evolution of Communication in a Population of Simple Machines , 1990 .

[23]  Angelo Cangelosi,et al.  Modeling the Evolution of Communication: From Stimulus Associations to Grounded Symbolic Associations , 1999, ECAL.

[24]  Claudia V. Goldman,et al.  Communicating Effectively in Resource-Constrained Multi-Agent Systems , 2007, IJCAI.

[25]  Sandip Sen,et al.  Computing effective communication policies in multiagent systems , 2007, AAMAS '07.

[26]  C. Lee Giles,et al.  Talking Helps: Evolving Communicating Agents for the Predator-Prey Pursuit Problem , 2000, Artificial Life.

[27]  Matthew Quinn,et al.  Evolving Communication without Dedicated Communication Channels , 2001, ECAL.

[28]  Tucker R. Balch,et al.  Communication in reactive multiagent robotic systems , 1995, Auton. Robots.

[29]  Luc Steels,et al.  A (very) Brief Introduction to Fluid Construction Grammar , 2006 .

[30]  Stefano Nolfi,et al.  Evolution of Communication and Language in Embodied Agents , 2009 .

[31]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[32]  Communication Strategies for Cooperating Behavior-Based Robots , 2001 .

[33]  L. Steels,et al.  PLANNING WHAT TO SAY : Second Order Semantics for Fluid Construction Grammars , 2005 .

[34]  D. Floreano,et al.  Historical contingency affects signaling strategies and competitive abilities in evolving populations of simulated robots , 2012, Proceedings of the National Academy of Sciences.

[35]  D. Floreano,et al.  Evolutionary Conditions for the Emergence of Communication in Robots , 2007, Current Biology.

[36]  Sridhar Mahadevan,et al.  Learning to communicate and act using hierarchical reinforcement learning , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[37]  Marco Dorigo,et al.  Evolution of Direct Communication for a Swarm-bot Performing Hole Avoidance , 2004, ANTS Workshop.

[38]  Makoto Yokoo,et al.  Communications for improving policy computation in distributed POMDPs , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[39]  Arvin Agah,et al.  Evolutionary Learning of Goal-Oriented Communication Strategies in Multi-Agent Systems , 2015, J. Autom. Mob. Robotics Intell. Syst..

[40]  Siani Pearson,et al.  An Adaptive Choice of Messaging Protocol in Multi Agent Systems , 1998, Foundations and Applications of Multi-Agent Systems.

[41]  Kenneth J. Mackin,et al.  Unsupervised training of multiobjective agent communication using genetic programming , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[42]  Stefano Nolfi,et al.  Emergence of communication in embodied agents evolved for the ability to solve a collective navigation problem , 2007, Connect. Sci..

[43]  Hitoshi Iba,et al.  Evolutionary Learning of Communicating Agents , 1998, Inf. Sci..

[44]  Paul Tarau,et al.  An empirical evaluation of communication effectiveness in autonomous reactive multiagent systems , 2005, SAC '05.

[45]  Nicholas R. Jennings,et al.  A principled information valuation for communications during multi-agent coordination , 2008 .

[46]  Stefano Nolfi,et al.  Self-Organization of Communication in Evolving Robots , 2006 .

[47]  L. Steels Evolving grounded communication for robots , 2003, Trends in Cognitive Sciences.

[48]  Jian Luo,et al.  Communication based on Interactive Dynamic Influence Diagrams in cooperative multi-agent systems , 2013, 2013 8th International Conference on Computer Science & Education.

[49]  Francisco S. Melo,et al.  QueryPOMDP: POMDP-Based Communication in Multiagent Systems , 2011, EUMAS.

[50]  Stefano Nolfi,et al.  Emergence of communication in embodied agents: co-adapting communicative and non-communicative behaviours , 2005, Connect. Sci..

[51]  Claudia V. Goldman,et al.  Optimizing information exchange in cooperative multi-agent systems , 2003, AAMAS '03.

[52]  Stefano Nolfi,et al.  Evolution of Implicit and Explicit Communication in Mobile Robots , 2010, Evolution of Communication and Language in Embodied Agents.

[53]  Eiichiro Tazaki,et al.  Multiagent communication combining genetic programming and pheromone communication , 2002 .

[54]  Kuo-Chi Lin,et al.  An incremental genetic algorithm approach to multiprocessor scheduling , 2004, IEEE Transactions on Parallel and Distributed Systems.

[55]  Feng Wu,et al.  Online planning for multi-agent systems with bounded communication , 2011, Artif. Intell..

[56]  Costas Tsatsoulis,et al.  Learning Communication Strategies in Multiagent Systems , 1998, Applied Intelligence.