Which one is the dax? Achieving mutual exclusivity with neural networks

Learning words is a challenge for children and neural networks alike. However, what they struggle with can differ. When prompted by novel words, children have been shown to tend to associate them with unfamiliar referents. This has been taken to reflect a propensity toward mutual exclusivity. In this study, we investigate whether and under which circumstances neural models can exhibit analogous behavior. To this end, we evaluate cross-situational neural models on novel items with distractors, contrasting the interaction between different word learning and referent selection strategies. We find that, as long as they bring about competition between words, constraints in both learning and referent selection can improve success in tasks with novel words and referents. For neural network research, our findings clarify the role of available options for enhanced performance in tasks where mutual exclusivity is advantageous. For cognitive research, they highlight latent interactions between word learning, referent selection mechanisms, and the structure of stimuli.

[1]  Afsaneh Fazly,et al.  A Probabilistic Computational Model of Cross-Situational Word Learning , 2010, Cogn. Sci..

[2]  Suzanne Stevenson,et al.  Calculating Probabilities Simplifies Word Learning , 2017, CogSci.

[3]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[4]  Grzegorz Chrupala,et al.  Learning language through pictures , 2015, ACL.

[5]  Christopher Potts,et al.  Learning in the Rational Speech Acts Model , 2015, ArXiv.

[6]  Michael C. Frank,et al.  Cross-situational Word Learning Respects Mutual Exclusivity , 2009 .

[7]  Victor S. Ferreira,et al.  How do speakers avoid ambiguous linguistic expressions? , 2005, Cognition.

[8]  P. Bloom.  How children learn the meanings of words , 2000 .

[9]  L. Gleitman,et al.  Language and Experience: Evidence from the Blind Child , 1988 .

[10]  Michael C. Frank,et al.  PSYCHOLOGICAL SCIENCE Research Article Using Speakers ’ Referential Intentions to Model Early Cross-Situational Word Learning , 2022 .

[11]  Larissa K. Samuelson,et al.  Word learning emerges from the interaction of online referent selection and slow associative learning. , 2012, Psychological review.

[12]  Brenden M. Lake,et al.  Mutual exclusivity as a challenge for deep neural networks , 2019, NeurIPS.

[13]  A. Avramides.  Studies in the Way of Words , 1992 .

[14]  Emmanuel Dupoux,et al.  Learning Words from Images and Speech , 2014 .

[15]  Tiago Tresoldi,et al.  The Database of Cross-Linguistic Colexifications, reproducible analysis of cross-linguistic polysemies , 2020, Scientific Data.

[16]  James R. Glass,et al.  Unsupervised Learning of Spoken Language with Visual Context , 2016, NIPS.

[17]  Susan Carey,et al.  Acquiring a Single New Word , 1978 .

[18]  Willard Van Orman Quine,et al.  Word and Object , 1960 .

[19]  David Schlangen,et al.  Know What You Don’t Know: Modeling a Pragmatic Speaker that Refers to Objects of Unknown Categories , 2019, ACL.

[20]  Noah D. Goodman,et al.  Animal, dog, or dalmatian? Level of abstraction in nominal referring expressions , 2016, CogSci.

[21]  J. Halberda,et al.  Is this a dax which I see before me? Use of the logical argument disjunctive syllogism supports word-learning in children and adults , 2006, Cognitive Psychology.

[22]  Charles Yang,et al.  How to Make the Most out of Very Little , 2019, Top. Cogn. Sci..

[23]  Peter Young,et al.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions , 2014, TACL.

[24]  Michael Franke,et al.  Pragmatic Back-and-Forth Reasoning , 2014 .

[25]  Herbert H. Clark,et al.  When Nouns Surface as Verbs , 1979 .

[26]  N. Goodman.  On Likeness of Meaning , 1949 .

[27]  Larissa K. Samuelson,et al.  Fast Mapping but Poor Retention by 24-Month-Old Infants. , 2008, Infancy : the official journal of the International Society on Infant Studies.

[28]  Michael C. Frank,et al.  Predicting Pragmatic Reasoning in Language Games , 2012, Science.

[29]  Michael C. Frank,et al.  Review Pragmatic Language Interpretation as Probabilistic Inference , 2022 .

[30]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[31]  B. MacWhinney.  The CHILDES project: tools for analyzing talk , 1992 .

[32]  Christopher Potts,et al.  Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding , 2017, TACL.

[33]  Grzegorz Chrupala,et al.  Learning word meanings from images of natural scenes , 2014, Trait. Autom. des Langues.

[34]  James R. Glass,et al.  Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input , 2018, ECCV.

[35]  Carlos Santana,et al.  Ambiguity in Cooperative Signaling , 2014, Philosophy of Science.

[36]  E. Clark.  On the logic of contrast. , 1988, Journal of child language.

[37]  Afsaneh Fazly,et al.  Fast Mapping in Word Learning: What Probabilities Tell Us , 2008, CoNLL.

[38]  Dan Klein,et al.  Reasoning about Pragmatics with Neural Listeners and Speakers , 2016, EMNLP.

[39]  E. Markman,et al.  Children's use of mutual exclusivity to constrain the meanings of words , 1988, Cognitive Psychology.

[40]  W. Quine.  Main trends in recent philosophy: two dogmas of empiricism. , 1951 .

[41]  Brenden M. Lake,et al.  Mutual exclusivity as a challenge for neural networks , 2019, ArXiv.

[42]  Siobhan Chapman.  Logic and Conversation , 2005 .

[43]  Svetlana Lazebnik,et al.  Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[44]  Marco Baroni,et al.  Multimodal Semantic Learning from Child-Directed Input , 2016, NAACL.

[45]  E. Markman,et al.  Use of the mutual exclusivity assumption by young word learners , 2003, Cognitive Psychology.