A computational cognitive neuroscience perspective on word meaning in context

In this section, we consider the nature of human word knowledge and the relationship between such knowledge and the concept of a word embedding. We formulate our conception of human word knowledge in terms of two challenges: inference and integration. By inference we mean the process of determining the contextually appropriate meaning of the word, and by integration, we mean the process of combining the results of this inference process with prior knowledge for subsequent use. We frame our proposed approaches to these challenges within an interactive, distributed processing framework for inference and a complementary learning systems framework for integration. We relate these ideas to the traditional concept of a word embeddings (Embeddings 1.0) and to our proposed updated conception of word embeddings (Embeddings 2.0). There are two key points:

[1]  James L. McClelland,et al.  I like coffee with cream and dog? Change in an implicit probabilistic representation captures meaning processing in the brain , 2017, bioRxiv.

[2]  Kenneth A. Norman,et al.  Discovering Event Structure in Continuous Narrative Perception and Memory , 2016, Neuron.

[3]  Quoc V. Le,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[4]  Daan Wierstra,et al.  One-shot Learning with Memory-Augmented Neural Networks , 2016, ArXiv.

[5]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[6]  James L. McClelland,et al.  Parallel Distributed Processing at 25: Further Explorations in the Microstructure of Cognition , 2014, Cogn. Sci..

[7]  Matthew H. Davis,et al.  Predictive Top-Down Integration of Prior Knowledge during Speech Perception , 2012, The Journal of Neuroscience.

[8]  Peter Hagoort,et al.  When Elephants Fly: Differential Sensitivity of Right and Left Inferior Frontal Gyri to Discourse and World Knowledge , 2009, Journal of Cognitive Neuroscience.

[9]  G. Altmann,et al.  Discourse-mediation of the mapping between language and the visual world: Eye movements and mental representation , 2009, Cognition.

[10]  Mante S. Nieuwland,et al.  When Peanuts Fall in Love: N400 Evidence for the Power of Discourse , 2005, Journal of Cognitive Neuroscience.

[11]  James L. McClelland,et al.  No Right to Speak? The Relationship between Object Naming and Semantic Impairment:Neuropsychological Evidence and a Computational Model , 2001, Journal of Cognitive Neuroscience.

[12]  David C. Plaut,et al.  Are non-semantic morphological effects incompatible with a distributed connectionist approach to lexical processing? , 2000 .

[13]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[14]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[15]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[16]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[17]  James L. McClelland,et al.  Distributed memory and the representation of general and specific information. , 1985, Journal of experimental psychology. General.

[18]  G. Simpson.  Lexical ambiguity and its role in models of word recognition. , 1984, Psychological bulletin.

[19]  T. H. Brown,et al.  Associative long-term potentiation in hippocampal slices. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[20]  L. Squire,et al.  Retrograde amnesia and remote memory impairment , 1981, Neuropsychologia.

[21]  B. McNaughton,et al.  Synaptic enhancement in fascia dentata: Cooperativity among coactive afferents , 1978, Brain Research.

[22]  D Marr,et al.  Simple memory: a theory for archicortex. , 1971, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[23]  R. M. Warren.  Perceptual Restoration of Missing Speech Sounds , 1970, Science.

[24]  B. Milner,et al.  Further analysis of the hippocampal amnesic syndrome: 14-year follow-up study of H.M.☆ , 1968 .

[25]  W. Scoville,et al.  LOSS OF RECENT MEMORY AFTER BILATERAL HIPPOCAMPAL LESIONS , 1957, Journal of neurology, neurosurgery, and psychiatry.

[26]  F. Heider,et al.  An experimental study of apparent behavior , 1944 .

[27]  Surya Ganguli,et al.  Learning hierarchical categories in deep neural networks , 2013, CogSci.

[28]  Neal J Cohen,et al.  Development of shared information in communication despite hippocampal amnesia , 2006, Nature Neuroscience.

[29]  James L. McClelland,et al.  Structure and deterioration of semantic memory: a neuropsychological and computational investigation. , 2004, Psychological review.

[30]  Geoffrey E. Hinton.  Using fast weights to deblur old memories , 1987 .

[31]  H. Marchand.  The categories and types of present-day English word-formation : a synchronic-diachronic approach , 1960 .