A Distributed Learning Algorithm for Communication Development

We study the question of how a local learning algorithm, executed by multiple distributed agents, can lead to a global system of communication. First, the notion of a perfect communication system is defined. Next, two measures of communication system quality are specified. It is shown that maximization of these measures leads to perfect communication production. Based on this principle, local adaptation rules for communication development are constructed. The resulting stochastic algorithm is validated in computational experiments. Empirical analysis indicates that a mild degree of stochasticity is instrumental in reaching states that correspond to accurate communication.

[1]  T. Schelling,et al.  The Strategy of Conflict. , 1961 .

[2]  David Lewis.  Convention: A Philosophical Study , 1986 .

[3]  M. Hauser.  The Evolution of Communication , 1996 .

[4]  B. MacLennan.  Synthetic Ethology An Approach to the Study of Communication , 2004 .

[5]  Ron Sun,et al.  Multi-agent reinforcement learning: weighting and partitioning , 1999, Neural Networks.

[6]  Luc Steels.  A self-organizing spatial vocabulary , 1995 .

[7]  Luc Steels,et al.  The synthetic modeling of language origins , 1997 .

[8]  Kagan Tumer,et al.  Collective Intelligence and Braess' Paradox , 2000, AAAI/IAAI.

[9]  Edwin D. de Jong,et al.  Autonomous Formation of Concepts and Communication , 2000 .

[10]  Simon Kirby,et al.  Spontaneous evolution of linguistic structure-an iterated learning model of the emergence of regularity and irregularity , 2001, IEEE Trans. Evol. Comput..

[11]  John W. Sheppard,et al.  Colearning in Differential Games , 1998, Machine Learning.

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  Mike Mannion,et al.  Complex systems , 1997, Proceedings International Conference and Workshop on Engineering of Computer-Based Systems.

[14]  M A Nowak,et al.  The evolution of language. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[16]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[17]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[18]  Seth Bullock.  A continuous evolutionary simulation model of the attainability of honest signalling equilibria , 1998 .

[19]  T. Schelling.  The Strategy of Conflict , 1963 .

[20]  Manuela M. Veloso,et al.  Rational and Convergent Learning in Stochastic Games , 2001, IJCAI.

[21]  Craig Boutilier,et al.  Learning Conventions in Multiagent Stochastic Domains using Likelihood Estimates , 1996, UAI.

[22]  Ezequiel A. Di Paolo,et al.  Social coordination and spatial organization: Steps towards the evolution of communication , 1997 .

[23]  Holly A. Yanco,et al.  An adaptive communication protocol for cooperating mobile robots , 1993 .

[24]  Aude Billard,et al.  Learning to Communicate Through Imitation in Autonomous Robots , 1997, ICANN.

[25]  Luc Steels,et al.  The Origins of Ontologies and Communication Conventions in Multi-Agent Systems , 2004, Autonomous Agents and Multi-Agent Systems.

[26]  Michael Oliphant,et al.  Formal approaches to innate and learned communication : laying the foundation for language , 1998 .

[27]  Bart de Boer,et al.  The Origins of Vowel Systems , 2001 .

[28]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[29]  Aude Billard,et al.  DRAMA, a Connectionist Architecture for Control and Learning in Autonomous Robots , 1999, Adapt. Behav..

[30]  T. Ikegami,et al.  Emergence of net-grammar in communicating agents. , 1996, Bio Systems.

[31]  Simon Kirby,et al.  Function, Selection, and Innateness: The Emergence of Language Universals , 1999 .

[32]  Luc Steels,et al.  The Origins of Syntax in Visually Grounded Robotic Agents , 1997, IJCAI.

[33]  Moshe Tennenholtz,et al.  On the Emergence of Social Conventions: Modeling, Analysis, and Simulations , 1997, Artif. Intell..

[34]  G. M. Werner.  Evolution of Communication in Artificial Organisms, Artifial Life II , 1991 .

[35]  Masanao Koeda,et al.  Proto-symbol emergence , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[36]  Ariel Rubinstein,et al.  WHY ARE CERTAIN PROPERTIES OF BINARY RELATIONS RELATIVELY MORE COMMON IN NATURAL LANGUAGE , 1996 .

[37]  Jason Noble,et al.  The Evolution of Animal Communication Systems: Questions of Function Examined through Simulation , 1998 .

[38]  James R. Hurford,et al.  Biological evolution of the Saussurean sign as a component of the language acquisition device , 1989 .