Modeling the Structure and Dynamics of Semantic Processing

Abstract The contents and structure of semantic memory have been the focus of much recent research, with major advances in the development of distributional models, which use word co‐occurrence information as a window into the semantics of language. In parallel, connectionist modeling has extended our knowledge of the processes engaged in semantic activation. However, these two lines of investigation have rarely been brought together. Here, we describe a processing model based on distributional semantics in which activation spreads throughout a semantic network, as dictated by the patterns of semantic similarity between words. We show that the activation profile of the network, measured at various time points, can successfully account for response times in lexical and semantic decision tasks, as well as for subjective concreteness and imageability ratings. We also show that the dynamics of the network is predictive of performance in relational semantic tasks, such as similarity/relatedness rating. Our results indicate that bringing together distributional semantic networks and spreading of activation provides a good fit to both automatic lexical processing (as indexed by lexical and semantic decisions) as well as more deliberate processing (as indexed by ratings), above and beyond what has been reported for previous models that take into account only similarity resulting from network structure.

[1]  Melvin J Yap,et al.  The Calgary semantic decision project: concrete/abstract decision data for 10,000 English words , 2016, Behavior Research Methods.

[2]  M. Brysbaert,et al.  Explaining human performance in psycholinguistic tasks with models of semantic similarity based on prediction and counting : A review and empirical validation , 2017 .

[3]  Daiker An Abundance of Riches , 2017 .

[4]  Felix Hill,et al.  SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity , 2016, EMNLP.

[5]  Michael N. Jones,et al.  Graph-Theoretic Properties of Networks Based on Word Association Norms: Implications for Models of Lexical Semantic Memory , 2016, Cogn. Sci..

[6]  Ashley N. Danguecan,et al.  Semantic Neighborhood Effects for Abstract versus Concrete Words , 2016, Front. Psychol..

[7]  Francisco Pereira,et al.  A comparative evaluation of off-the-shelf distributed semantic representations for modelling behavioural data , 2016, Cognitive neuropsychology.

[8]  Gerrit Storms,et al.  Structure and organization of the mental lexicon: A network approach derived from syntactic dependency relations and word associations , 2016 .

[9]  Alexander Panchenko,et al.  Noun Sense Induction and Disambiguation using Graph-Based Distributional Semantics , 2016, KONVENS.

[10]  Akira Utsumi,et al.  A Complex Network Approach to Distributional Semantic Models , 2015, PloS one.

[11]  Alexander Mehler,et al.  Towards a Theoretical Framework for Analyzing Complex Linguistic Networks , 2015 .

[12]  Michael N. Jones,et al.  Foraging in Semantic Fields: How We Search Through Memory , 2015, Top. Cogn. Sci..

[13]  Paul D. Siakaluk,et al.  Situated conceptualization and semantic processing: effects of emotional experience and context availability in semantic categorization and naming tasks , 2015, Psychonomic bulletin & review.

[14]  P. Hoffman,et al.  Opposing Effects of Semantic Diversity in Lexical and Semantic Relatedness Decisions , 2015, Journal of experimental psychology. Human perception and performance.

[15]  Felix Hill,et al.  SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation , 2014, CL.

[16]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[17]  Amy Beth Warriner,et al.  Concreteness ratings for 40 thousand generally known English word lemmas , 2014, Behavior research methods.

[18]  Georgiana Dinu,et al.  Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors , 2014, ACL.

[19]  Carina Silberer,et al.  Learning Grounded Meaning Representations with Autoencoders , 2014, ACL.

[20]  Ian S. Hargreaves,et al.  Get rich quick: The signal to respond procedure reveals the time course of semantic richness effects during visual word recognition , 2014, Cognition.

[21]  Marc Brysbaert,et al.  Subtlex-UK: A New and Improved Word Frequency Database for British English , 2014, Quarterly journal of experimental psychology.

[22]  Elia Bruni,et al.  Multimodal Distributional Semantics , 2014, J. Artif. Intell. Res..

[23]  T. Rogers,et al.  Semantic diversity: A measure of semantic ambiguity based on variability in the contextual usage of words , 2012, Behavior Research Methods.

[24]  Alberto Testolin,et al.  Modeling language and cognition with deep unsupervised learning: a tutorial overview , 2013, Front. Psychol..

[25]  Lenka Zdrazilova,et al.  Grasping the invisible: Semantic processing of abstract words , 2013, Psychonomic bulletin & review.

[26]  Kirsty Kitto,et al.  How activation, entanglement, and searching a semantic network contribute to event memory , 2013, Memory & cognition.

[27]  Nick Chater,et al.  Networks in Cognitive Science , 2013, Trends in Cognitive Sciences.

[28]  Amy Beth Warriner,et al.  Norms of valence, arousal, and dominance for 13,915 English lemmas , 2013, Behavior Research Methods.

[29]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[30]  Danielle J. Navarro,et al.  Better explanations of lexical and semantic cognition using networks derived from continued rather than single-word associations , 2012, Behavior Research Methods.

[31]  Michael N. Jones,et al.  The semantic richness of abstract concepts , 2012, Front. Hum. Neurosci..

[32]  Paul D. Siakaluk,et al.  Effects of Emotional and Sensorimotor Knowledge in Semantic Processing of Concrete and Abstract Nouns , 2012, Front. Hum. Neurosci..

[33]  Gemma Boleda,et al.  Distributional Semantics in Technicolor , 2012, ACL.

[34]  Brendan T. Johns,et al.  The role of semantic diversity in lexical organization. , 2012, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[35]  M. Brysbaert,et al.  Age-of-acquisition ratings for 30,000 English words , 2012, Behavior research methods.

[36]  Mark J. Huff,et al.  An Abundance of Riches: Cross-Task Comparisons of Semantic Richness Effects in Visual Word Recognition , 2012, Front. Hum. Neurosci..

[37]  D. Mirman,et al.  Competition and cooperation among similar representations: toward a unified account of facilitative and inhibitory effects of lexical neighbors. , 2012, Psychological review.

[38]  Thomas T. Hills,et al.  Optimal foraging in semantic memory. , 2012, Psychological review.

[39]  John A Bullinaria,et al.  Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD , 2012, Behavior Research Methods.

[40]  Marc Brysbaert,et al.  The British Lexicon Project: Lexical decision data for 28,730 monosyllabic and disyllabic English words , 2011, Behavior Research Methods.

[41]  Chris Biemann Word Sense Induction and Disambiguation , 2012 .

[42]  Amy Perfors,et al.  Strong structure in weak semantic similarity: A graph based account , 2012, CogSci.

[43]  Ian S. Hargreaves,et al.  Is more always better? Effects of semantic richness on lexical decision, speeded pronunciation, and semantic classification , 2011, Psychonomic bulletin & review.

[44]  Lidia Suárez,et al.  Observing neighborhood effects without neighbors , 2011, Psychonomic bulletin & review.

[45]  Michael N. Jones,et al.  Redundancy in Perceptual and Linguistic Experience: Comparing Feature-Based and Distributional Models of Semantic Representation , 2010, Top. Cogn. Sci..

[46]  Marc W. Howard,et al.  Constructing Semantic Representations From a Gradually Changing Representation of Temporal Context , 2011, Top. Cogn. Sci..

[47]  James L. McClelland,et al.  Letting structure emerge: connectionist and dynamical systems approaches to cognition , 2010, Trends in Cognitive Sciences.

[48]  J. Tenenbaum,et al.  Probabilistic models of cognition: exploring representations and inductive biases , 2010, Trends in Cognitive Sciences.

[49]  Petr Sojka,et al.  Software Framework for Topic Modelling with Large Corpora , 2010 .

[50]  Alexandre Arenas,et al.  Semantic Networks: Structure and Dynamics , 2010, Entropy.

[51]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[52]  Massimo Poesio,et al.  Strudel: A Corpus-Based Semantic Model Based on Properties and Types , 2010, Cogn. Sci..

[53]  Ricard V. Solé,et al.  Language networks: Their structure, function, and evolution , 2007, Complex..

[54]  Gabriella Vigliocco,et al.  The Hidden Markov Topic Model: A Probabilistic Model of Semantic Representation , 2010, Top. Cogn. Sci..

[55]  Peter Bruza,et al.  Is There Something Quantum-Like about the Human Mental Lexicon? , 2009, INEX.

[56]  Gabriella Vigliocco,et al.  Integrating experiential and distributional data to learn semantic representations. , 2009, Psychological review.

[57]  D. Balota,et al.  Moving beyond Coltheart’s N: A new measure of orthographic similarity , 2008, Psychonomic bulletin & review.

[58]  Gert Storms,et al.  Word associations: Network and semantic properties , 2008, Behavior research methods.

[59]  Gert Storms,et al.  Word associations: Norms for 1,424 Dutch words in a continuous task , 2008, Behavior research methods.

[60]  Ian S. Hargreaves,et al.  There are many ways to be rich: Effects of three measures of semantic richness on visual word recognition , 2008, Psychonomic bulletin & review.

[61]  Daniel Mirman,et al.  Attractor Dynamics and Semantic Neighborhood Density: Processing Is Slowed by near Neighbors and Speeded by Distant Neighbors We Thank Ann Kulikowski for Her Help with Data Collection And , 2022 .

[62]  T. Griffiths,et al.  Google and the Mind , 2007, Psychological science.

[63]  Rebecca Treiman,et al.  The English Lexicon Project , 2007, Behavior research methods.

[64]  J. Bullinaria,et al.  Extracting semantic representations from word co-occurrence statistics: A computational study , 2007, Behavior research methods.

[65]  Mirella Lapata,et al.  Dependency-Based Construction of Semantic Space Models , 2007, CL.

[66]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[67]  Michael N Jones,et al.  Representing word meaning and order information in a composite holographic lexicon. , 2007, Psychological review.

[68]  H. Stadthagen-González,et al.  The Bristol norms for age of acquisition, imageability, and familiarity , 2006, Behavior research methods.

[69]  D. Nelson,et al.  What is preexisting strength? Predicting free association probabilities, similarity ratings, and cued recall probabilities , 2005, Psychonomic bulletin & review.

[70]  Richard M. Shiffrin,et al.  Word Association Spaces for Predicting Semantic Similarity Effects in Episodic Memory. , 2005 .

[71]  Thomas A. Schreiber,et al.  The University of South Florida free association, rhyme, and word fragment norms , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[72]  R. Clark,et al.  Competition and Cooperation , 2004 .

[73]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[74]  Stefan Richter,et al.  Centrality Indices , 2004, Network Analysis.

[75]  M. Kahana Associative symmetry and memory theory , 2002, Memory & cognition.

[76]  R. Shillcock,et al.  Rethinking the Word Frequency Effect: The Neglected Role of Distributional Information in Lexical Processing , 2001, Language and speech.

[77]  Curt Burgess,et al.  Characterizing semantic space: Neighborhood effects in word recognition , 2001, Psychonomic bulletin & review.

[78]  N. Gee,et al.  Interpreting the influence of implicitly activated memories on recall and recognition. , 1998, Psychological review.

[79]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[80]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[81]  Geoffrey Leech,et al.  CLAWS4: The Tagging of the British National Corpus , 1994, COLING.

[82]  A. Roelofs,et al.  A spreading-activation theory of lemma retrieval in speaking , 1992, Cognition.

[83]  G S Dell,et al.  A spreading-activation theory of retrieval in sentence production. , 1986, Psychological review.

[84]  John R. Anderson A spreading activation theory of memory. , 1983 .

[85]  R. Logie,et al.  Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1,944 words , 1980 .

[86]  A. Tversky Features of Similarity , 1977 .

[87]  Max Coltheart,et al.  Access to the internal lexicon , 1977 .

[88]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[89]  Zellig S. Harris,et al.  Distributional Structure , 1954 .