Emergence of Grounded Compositional Language in Multi-Agent Populations

By capturing statistical patterns in large corpora, machine learning has enabled significant advances in natural language processing, including in machine translation, question answering, and sentiment analysis. However, for agents to intelligently interact with humans, simply capturing the statistical patterns is insufficient. In this paper we investigate if, and how, grounded compositional language can emerge as a means to achieve goals in multi-agent populations. Towards this end, we propose a multi-agent learning environment and learning methods that bring about emergence of a basic compositional language. This language is represented as streams of abstract discrete symbols uttered by agents over time, but nonetheless has a coherent structure that possesses a defined vocabulary and syntax. We also observe emergence of non-verbal communication such as pointing and guiding when language communication is unavailable.

[1]  Gebräuchliche Fertigarzneimittel,et al.  V , 1893, Therapielexikon Neurologie.

[2]  Terry Winograd,et al.  A procedural model of language understanding , 1986 .

[3]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[4]  Luc Steels,et al.  A Self-Organizing Spatial Vocabulary , 1995, Artificial Life.

[5]  Simon Kirby,et al.  Syntax out of Learning: The Cultural Evolution of Structured Communication in a Population of Induction Algorithms , 1999, ECAL.

[6]  Martin A. Nowak,et al.  The evolution of syntactic communication , 2000, Nature.

[7]  Simon Kirby,et al.  Spontaneous evolution of linguistic structure-an iterated learning model of the emergence of regularity and irregularity , 2001, IEEE Trans. Evol. Comput..

[8]  장윤희,et al.  Y. , 2003, Industrial and Labor Relations Terms.

[9]  Siobhan Chapman.  Logic and Conversation , 2005 .

[10]  Luc Steels,et al.  What triggers the emergence of grammar , 2005 .

[11]  Csr Young,et al.  How to Do Things With Words , 2009 .

[12]  Richard L. Lewis,et al.  A new approach to exploring language emergence as boundedly optimal control in the face of environmental and cognitive constraints , 2010 .

[13]  Dan Klein,et al.  A Game-Theoretic Approach to Generating Spatial Descriptions , 2010, EMNLP.

[14]  Yee Whye Teh,et al.  Dirichlet Process , 2017, Encyclopedia of Machine Learning and Data Mining.

[15]  Michael C. Frank,et al.  Predicting Pragmatic Reasoning in Language Games , 2012, Science.

[16]  L. Steels,et al.  Agent-Based Models of Strategies for the Emergence and Evolution of Grammatical Agreement , 2013, PloS one.

[17]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[18]  J. Koenderink.  Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[19]  Michael C. Frank,et al.  Learning to Reason Pragmatically with Cognitive Limitations , 2014, CogSci.

[20]  S. Kirby,et al.  Iterated learning and the evolution of language , 2014, Current Opinion in Neurobiology.

[21]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[22]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[23]  Shimon Whiteson,et al.  Learning to Communicate with Deep Multi-Agent Reinforcement Learning , 2016, NIPS.

[24]  Christopher D. Manning,et al.  Learning Language Games through Interaction , 2016, ACL.

[25]  Rob Fergus,et al.  Learning Multiagent Communication with Backpropagation , 2016, NIPS.

[26]  Dan Klein,et al.  Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints , 2016, ACL.

[27]  Igor Mordatch,et al.  A Paradigm for Situated and Goal-Driven Language Learning , 2016, ArXiv.

[28]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[29]  Dan Klein,et al.  Reasoning about Pragmatics with Neural Listeners and Speakers , 2016, EMNLP.

[30]  Yang Xu,et al.  The Pragmatics of Spatial Language , 2016, CogSci.

[31]  Angeliki Lazaridou,et al.  Towards Multi-Agent Communication-Based Language Learning , 2016, ArXiv.

[32]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[33]  Tom Schaul,et al.  The Predictron: End-To-End Learning and Planning , 2016, ICML.

[34]  Vladlen Koltun,et al.  Learning to Act by Predicting the Future , 2016, ICLR.

[35]  Yee Whye Teh,et al.  The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables , 2016, ICLR.

[36]  Anca D. Dragan,et al.  Translating Neuralese , 2017, ACL.

[37]  Alexander Peysakhovich,et al.  Multi-Agent Cooperation and the Emergence of (Natural) Language , 2016, ICLR.

[38]  Jianfeng Gao,et al.  Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access , 2016, ACL.

[39]  Tsuyoshi Murata,et al.  {m , 1934, ACML.