Emergent Communication: Generalization and Overfitting in Lewis Games

Lewis signaling games are a class of simple communication games for simulating the emergence of language. In these games, two agents must agree on a communication protocol in order to solve a cooperative task. Previous work has shown that agents trained to play this game with reinforcement learning tend to develop languages that display undesirable properties from a linguistic point of view (lack of generalization, lack of compositionality, etc). In this paper, we aim to provide better understanding of this phenomenon by analytically studying the learning problem in Lewis games. As a core contribution, we demonstrate that the standard objective in Lewis games can be decomposed in two components: a co-adaptation loss and an information loss. This decomposition enables us to surface two potential sources of overfitting, which we show may undermine the emergence of a structured communication protocol. In particular, when we control for overfitting on the co-adaptation loss, we recover desired properties in the emergent languages: they are more compositional and generalize better.

[1]  O. Pietquin,et al.  On the role of population heterogeneity in emergent communication , 2022, ICLR.

[2]  Lukas Galke,et al.  Emergent Communication for Understanding Human Language Evolution: What's Missing? , 2022, ArXiv.

[3]  Ashley D. Edwards,et al.  Learning Robust Real-Time Cultural Transmission without Human Data , 2022, ArXiv.

[4]  Stefano V. Albrecht,et al.  Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictability , 2021, ICLR.

[5]  Bilal Piot,et al.  Emergent Communication at Scale , 2022, ICLR.

[6]  A. Kalinowska S ITUATED COMMUNICATION : A SOLUTION TO OVER COMMUNICATION BETWEEN ARTIFICIAL AGENTS , 2022 .

[7]  Tomasz Korbak,et al.  Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication , 2021, NeurIPS.

[8]  Eugene Kharitonov,et al.  Interpretable agent communication from scratch(with a generic visual processor emerging on the side) , 2021, NeurIPS.

[9]  Noah D. Goodman,et al.  Emergent Communication of Generalizations , 2021, NeurIPS.

[10]  Marco Baroni,et al.  Communicating artificial neural networks develop efficient color-naming systems , 2021, Proceedings of the National Academy of Sciences.

[11]  Aaron C. Courville,et al.  Emergent Communication under Competition , 2021, AAMAS.

[12]  Jooyeon Kim,et al.  Emergent Communication under Varying Sizes and Connectivities , 2021, NeurIPS.

[13]  Rahma Chaabouni,et al.  “LazImpa”: Lazy and Impatient neural agents learn to communicate efficiently , 2020, CONLL.

[14]  Pierre H. Richemond,et al.  Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning , 2020, NeurIPS.

[15]  Angeliki Lazaridou,et al.  Emergent Multi-Agent Communication in the Deep Learning Era , 2020, ArXiv.

[16]  Eugene Kharitonov,et al.  Compositionality and Generalization In Emergent Languages , 2020, ACL.

[17]  Eugene Kharitonov,et al.  Emergent Language Generalization and Acquisition Speed are not tied to Compositionality , 2020, BLACKBOXNLP.

[18]  Simon Kirby,et al.  Compositional Languages Emerge in a Neural Iterated Learning Model , 2020, ICLR.

[19]  Abhinav Gupta,et al.  Exploring Structural Inductive Biases in Emergent Communication , 2020, ArXiv.

[20]  Joelle Pineau,et al.  On the interaction between supervision and self-play in emergent communication , 2020, ICLR.

[21]  Andrew M. Dai,et al.  Capacity, Bandwidth, and Compositionality in Emergent Language Learning , 2019, AAMAS.

[22]  Joel Nothman,et al.  SciPy 1.0-Fundamental Algorithms for Scientific Computing in Python , 2019, ArXiv.

[23]  Marco Baroni,et al.  Entropy Minimization In Emergent Languages , 2019, ICML.

[24]  Piotr Milos,et al.  Emergence of compositional language in communication through noisy channel , 2020 .

[25]  Eugene Kharitonov,et al.  EGG: a toolkit for research on Emergence of lanGuage in Games , 2019, EMNLP.

[26]  Michael Bowling,et al.  Ease-of-Teaching and Language Structure from Emergent Communication , 2019, NeurIPS.

[27]  Eugene Kharitonov,et al.  Anti-efficient encoding in emergent communication , 2019, NeurIPS.

[28]  Marco Baroni,et al.  CNNs found to jump around more skillfully than RNNs: Compositional Generalization in Seq2seq Convolutional Networks , 2019, ACL.

[29]  Michael Cogswell,et al.  Emergence of Compositional Language with Deep Generational Transmission , 2019, ArXiv.

[30]  Marco Baroni,et al.  Linguistic generalization and compositionality in modern artificial neural networks , 2019, Philosophical Transactions of the Royal Society B.

[31]  Jacob Andreas,et al.  Measuring Compositionality in Representation Learning , 2019, ICLR.

[32]  Laura Graesser,et al.  Emergent Linguistic Phenomena in Multi-Agent Communication Games , 2019, EMNLP.

[33]  Shiri Lev-Ari,et al.  Compositional structure can emerge without generational transmission , 2019, Cognition.

[34]  K. Zuberbühler,et al.  Compositionality in animals and humans , 2018, PLoS biology.

[35]  Charles Kemp,et al.  Efficient compression in color naming and its evolution , 2018, Proceedings of the National Academy of Sciences.

[36]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[37]  Stephen Clark,et al.  Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input , 2018, ICLR.

[38]  Kyunghyun Cho,et al.  Emergent Communication in a Multi-Modal, Multi-Step Referential Game , 2017, ICLR.

[39]  José M. F. Moura,et al.  Natural Language Does Not Emerge ‘Naturally’ in Multi-Agent Dialog , 2017, EMNLP.

[40]  Ivan Titov,et al.  Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols , 2017, NIPS.

[41]  Simon Kirby,et al.  Sequence Memory Constraints Give Rise to Language-Like Structure through Iterated Learning , 2017, PloS one.

[42]  Alexander Peysakhovich,et al.  Multi-Agent Cooperation and the Emergence of (Natural) Language , 2016, ICLR.

[43]  Simon Kirby,et al.  Minimal Requirements for the Emergence of Learned Signaling , 2014, Cogn. Sci..

[44]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[45]  S. Kirby,et al.  Compression and communication in the cultural evolution of linguistic structure , 2015, Cognition.

[46]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[47]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[48]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[49]  Simon Kirby,et al.  Culture: Copying, Compression, and Conventionality , 2014, Cogn. Sci..

[50]  Simon M. Huttegger,et al.  Some dynamics of signaling games , 2014, Proceedings of the National Academy of Sciences.

[51]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[52]  T. Mark Ellison,et al.  The Cultural Evolution of Human Communication Systems in Different Sized Populations: Usability Trumps Learnability , 2013, PloS one.

[53]  Armin W. Schulz Signals: evolution, learning, and information , 2012 .

[54]  Simon Garrod,et al.  Experimental Semiotics: A Review , 2010, Front. Hum. Neurosci..

[55]  G. Lupyan,et al.  Language Structure Is Partly Determined by Social Structure , 2010, PloS one.

[56]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[57]  Nick C. Ellis,et al.  The Dynamics of Second Language Emergence: Cycles of Language Use, Language Change, and Language Acquisition , 2008 .

[58]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[59]  A. Wray,et al.  The consequences of talking to strangers: Evolutionary corollaries of socio-cultural influences on linguistic form , 2007 .

[60]  D. Bickerton Language evolution: A brief guide for linguists , 2007 .

[61]  S. Harnad Symbol grounding problem , 1990, Scholarpedia.

[62]  Simon Kirby,et al.  Understanding Linguistic Evolution by Visualizing the Emergence of Topographic Mappings , 2006, Artificial Life.

[63]  Philipp Koehn,et al.  Statistical Significance Tests for Machine Translation Evaluation , 2004, EMNLP.

[64]  Simon Kirby,et al.  Iterated Learning: A Framework for the Emergence of Language , 2003, Artificial Life.

[65]  Morten H. Christiansen,et al.  Language evolution: consensus and controversies , 2003, Trends in Cognitive Sciences.

[66]  James A. Reggia,et al.  Progress in the Simulation of Emergent Communication and Language , 2003, Adapt. Behav..

[67]  S. Kirby,et al.  The emergence of linguistic structure: an overview of the iterated learning model , 2002 .

[68]  Simon Kirby,et al.  Spontaneous evolution of linguistic structure-an iterated learning model of the emergence of regularity and irregularity , 2001, IEEE Trans. Evol. Comput..

[69]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[70]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[71]  Luc Steels,et al.  The synthetic modeling of language origins , 1997 .

[72]  H. H. Clark,et al.  Conceptual pacts and lexical choice in conversation. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[73]  M. Clyne Linguistic and Sociolinguistic Aspects of Language Contact, Maintenance and Loss: Towards a Multifacet Theory , 1992 .

[74]  S. Kroon,et al.  Maintenance and loss of minority languages , 1992 .

[75]  Anders Krogh,et al.  A Simple Weight Decay Can Improve Generalization , 1991, NIPS.

[76]  Jing Peng,et al.  Function Optimization using Connectionist Reinforcement Learning Algorithms , 1991 .

[77]  W. Beyer CRC Standard Probability And Statistics Tables and Formulae , 1990 .

[78]  Geoffrey E. Hinton Learning Translation Invariant Recognition in Massively Parallel Networks , 1987, PARLE.

[79]  J. Sobel,et al.  STRATEGIC INFORMATION TRANSMISSION , 1982 .

[80]  S. Harnad,et al.  Origins and Evolution of Language and Speech , 1976 .

[81]  R. Kirk CONVENTION: A PHILOSOPHICAL STUDY , 1970 .

[82]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .