Emergence of Compositional Language with Deep Generational Transmission

Consider a collaborative task that requires communication. Two agents are placed in an environment and must create a language from scratch in order to coordinate. Recent work has been interested in what kinds of languages emerge when deep reinforcement learning agents are put in such a situation, and in particular in the factors that cause language to be compositional-i.e. meaning is expressed by combining words which themselves have meaning. Evolutionary linguists have also studied the emergence of compositional language for decades, and they find that in addition to structural priors like those already studied in deep learning, the dynamics of transmitting language from generation to generation contribute significantly to the emergence of compositionality. In this paper, we introduce these cultural evolutionary dynamics into language emergence by periodically replacing agents in a population to create a knowledge gap, implicitly inducing cultural transmission of language. We show that this implicit cultural transmission encourages the resulting languages to exhibit better compositional generalization and suggest how elements of cultural dynamics can be further integrated into populations of deep agents.

[1]  Yoshua Bengio,et al.  Knowledge Matters: Importance of Prior Information for Optimization , 2013, J. Mach. Learn. Res..

[2]  Alexander Peysakhovich,et al.  Multi-Agent Cooperation and the Emergence of (Natural) Language , 2016, ICLR.

[3]  Stefan Lee,et al.  Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Pieter Abbeel,et al.  Emergence of Grounded Compositional Language in Multi-Agent Populations , 2017, AAAI.

[5]  Yoshua Bengio,et al.  Evolving Culture vs Local Minima , 2012, ArXiv.

[6]  Dhruv Batra,et al.  Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  S. Kirby,et al.  Compression and communication in the cultural evolution of linguistic structure , 2015, Cognition.

[8]  Shimon Whiteson,et al.  Learning to Communicate with Deep Multi-Agent Reinforcement Learning , 2016, NIPS.

[9]  Stephen Clark,et al.  Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input , 2018, ICLR.

[10]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[11]  S. Kirby,et al.  Language evolution in the laboratory , 2010, Trends in Cognitive Sciences.

[12]  Ivan Titov,et al.  Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols , 2017, NIPS.

[13]  S. Pinker,et al.  Natural language and natural selection , 1990, Behavioral and Brain Sciences.

[14]  Marco Baroni,et al.  Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks , 2017, ICML.

[15]  Shiri Lev-Ari,et al.  Compositional structure can emerge without generational transmission , 2019, Cognition.

[16]  Tomas Mikolov,et al.  A Roadmap Towards Machine Intelligence , 2015, CICLing.

[17]  Quoc V. Le,et al.  Large-Scale Evolution of Image Classifiers , 2017, ICML.

[18]  S. Kirby,et al.  Complex Adaptive Systems and the Origins of Adaptive Structure: What Experiments Can Tell Us , 2009 .

[19]  W. Strange.  Evolution of language. , 1984, JAMA.

[20]  Amy Perfors,et al.  Simulated Evolution of Language: a Review of the Field , 2002, J. Artif. Soc. Soc. Simul..

[21]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[22]  Anca D. Dragan,et al.  Translating Neuralese , 2017, ACL.

[23]  Simon Kirby,et al.  Minimal Requirements for the Emergence of Learned Signaling , 2014, Cogn. Sci..

[24]  Henry Brighton,et al.  Compositional Syntax From Cultural Transmission , 2002, Artificial Life.

[25]  Kevin Murphy,et al.  Generative Models of Visually Grounded Imagination , 2017, ICLR.

[26]  Elia Bruni,et al.  Co-evolution of language and agents in referential games , 2020, EACL.

[27]  Simon Kirby,et al.  Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language , 2008, Proceedings of the National Academy of Sciences.

[28]  Simon Kirby,et al.  Spontaneous evolution of linguistic structure-an iterated learning model of the emergence of regularity and irregularity , 2001, IEEE Trans. Evol. Comput..

[29]  Ted Briscoe.  Grammatical acquisition: Inductive bias and coevolution of language and the language acquisition device , 2000 .

[30]  Margaret Mitchell,et al.  VQA: Visual Question Answering , 2015, International Journal of Computer Vision.

[31]  Morten H. Christiansen,et al.  Language evolution: consensus and controversies , 2003, Trends in Cognitive Sciences.

[32]  Kenny Smith,et al.  Cultural Evolution of Language , 2006 .

[33]  José M. F. Moura,et al.  Natural Language Does Not Emerge ‘Naturally’ in Multi-Agent Dialog , 2017, EMNLP.

[34]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[35]  Nando de Freitas,et al.  Compositional Obverter Communication Learning From Raw Visual Input , 2018, ICLR.

[36]  Kenneth O. Stanley,et al.  A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks , 2009, Artificial Life.

[37]  Jason Lee,et al.  Emergent Translation in Multi-Agent Communication , 2017, ICLR.

[38]  Martin A. Nowak,et al.  The evolution of syntactic communication , 2000, Nature.

[39]  Paul Vogt,et al.  The emergence of compositional structures in perceptually grounded language games , 2005, Artif. Intell..

[40]  Rob Fergus,et al.  Learning Multiagent Communication with Backpropagation , 2016, NIPS.

[41]  Simon Kirby,et al.  Iterated Learning: A Framework for the Emergence of Language , 2003, Artificial Life.

[42]  S. Kirby,et al.  Iterated learning and the evolution of language , 2014, Current Opinion in Neurobiology.

[43]  Simon Kirby,et al.  Natural Language From Artificial Life , 2002, Artificial Life.

[44]  J. Stevenson.  The cultural origins of human cognition , 2001 .

[45]  Joelle Pineau,et al.  How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation , 2016, EMNLP.

[46]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[47]  Igor Mordatch,et al.  A Paradigm for Situated and Goal-Driven Language Learning , 2016, ArXiv.

[48]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[49]  Stephen Clark,et al.  Virtual Embodiment: A Scalable Long-Term Strategy for Artificial Intelligence Research , 2016, ArXiv.