Predicting Lexical Priming Effects from Distributional Semantic Similarities: A Replication with Extension

In two experiments, we attempted to replicate and extend findings by Günther et al. (2016) that word similarity measures obtained from distributional semantics models—Latent Semantic Analysis (LSA) and Hyperspace Analog to Language (HAL)—predict lexical priming effects. To this end, we used the pseudo-random method to generate item material while systematically controlling for word similarities introduced by Günther et al. (2016) which was based on LSA cosine similarities (Experiment 1) and HAL cosine similarities (Experiment 2). Extending the original study, we used semantic spaces created from far larger corpora, and implemented several additional methodological improvements. In Experiment 1, we only found a significant effect of HAL cosines on lexical decision times, while we found significant effects for both LSA and HAL cosines in Experiment 2. As further supported by an analysis of the pooled data from both experiments, this indicates that HAL cosines are a better predictor of priming effects than LSA cosines. Taken together, the results replicate the finding that priming effects can be predicted from distributional semantic similarity measures.

[1]  R. Schvaneveldt,et al.  Facilitation in recognizing pairs of words: evidence of a dependence between retrieval operations. , 1971, Journal of experimental psychology.

[2]  Eric Brill Processing Natural Language without Natural Language Processing , 2003, CICLing.

[3]  Lorraine K. Tyler,et al.  A Distributed Memory Model of the Associative Boost in Semantic Priming , 1994, Connect. Sci..

[4]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[5]  Jason Wittenberg,et al.  Clarify: Software for Interpreting and Presenting Statistical Results , 2003 .

[6]  P. Robinson,et al.  Efficient Estimation of the , 2007 .

[7]  Fritz Günther,et al.  LSAfun - An R package for computations based on Latent Semantic Analysis , 2014, Behavior Research Methods.

[8]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[9]  Barbara Kaup,et al.  Is there a difference between stripy journeys and stripy ladybirds? The N400 response to semantic and world-knowledge violations during sentence processing , 2016, Brain and Cognition.

[10]  Massimo Poesio,et al.  Strudel: A Corpus-Based Semantic Model Based on Properties and Types , 2010, Cogn. Sci..

[11]  Ping Li,et al.  Contextual self-organizing map: software for constructing semantic representations , 2011, Behavior research methods.

[12]  Alessandro Lenci,et al.  Distributional semantics in linguistic and cognitive research , 2008 .

[13]  Edwin Mims,et al.  The University in the South , 1903 .

[14]  C. Burgess,et al.  Semantic and associative priming in the cerebral hemispheres: Some words do, some words don't … sometimes, some places , 1990, Brain and Language.

[15]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[16]  Mirella Lapata,et al.  Dependency-Based Construction of Semantic Space Models , 2007, CL.

[17]  J. Bullinaria,et al.  Extracting semantic representations from word co-occurrence statistics: A computational study , 2007, Behavior research methods.

[18]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[19]  John A Bullinaria,et al.  Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD , 2012, Behavior Research Methods.

[20]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[21]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[22]  Michael N Jones,et al.  Representing word meaning and order information in a composite holographic lexicon. , 2007, Psychological review.

[23]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[24]  R. Baayen,et al.  Analyzing Reaction Times , 2010 .

[25]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[26]  David A. Balota,et al.  The semantic priming project , 2013, Behavior Research Methods.

[27]  Gabriella Vigliocco,et al.  Integrating experiential and distributional data to learn semantic representations. , 2009, Psychological review.

[28]  Michael J Cortese,et al.  Predicting semantic priming at the item level , 2008 .

[29]  Curt Burgess,et al.  Explorations in context space: Words, sentences, discourse , 1998 .

[30]  John B. Goodenough,et al.  Contextual correlates of synonymy , 1965, CACM.

[31]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[32]  Elizabeth Gilbert,et al.  Reproducibility Project: Results (Part of symposium called "The Reproducibility Project: Estimating the Reproducibility of Psychological Science") , 2014 .

[33]  C. A. Becker Semantic context effects in visual word recognition: An analysis of semantic strategies , 1980, Memory & cognition.

[34]  Zellig S. Harris,et al.  Distributional Structure , 1954 .

[35]  Mirella Lapata,et al.  Measuring Distributional Similarity in Context , 2010, EMNLP.

[36]  Fritz Günther,et al.  Latent semantic analysis cosines as a cognitive similarity measure: Evidence from priming studies , 2016, Quarterly journal of experimental psychology.

[37]  M. Abernethy,et al.  Semantic category priming in the left cerebral hemisphere , 1996, Neuropsychologia.

[38]  J. Fox Effect Displays in R for Generalised Linear Models , 2003 .

[39]  Gertrud Faaß,et al.  SdeWaC - A Corpus of Parsable Sentences from the Web , 2013, GSCL.

[40]  James M. Hodgson Informational constraints on pre-lexical priming , 1991 .

[41]  Rolf Ulrich,et al.  Inflation von falsch-positiven Befunden in der psychologischen Forschung : mögliche Ursachen und Gegenmaßnahmen , 2016 .

[42]  Walter Kintsch,et al.  Predication , 2001, Cogn. Sci..

[43]  M. Marelli,et al.  Affixation in semantic space: Modeling morpheme meanings with compositional distributional semantics. , 2015, Psychological review.

[44]  Patrick Bonin,et al.  Mental lexicon : "some words to talk about words" , 2004 .

[45]  M. Lucas,et al.  Semantic priming without association: A meta-analytic review , 2000, Psychonomic bulletin & review.

[46]  Keith A Hutchison,et al.  Is semantic priming due to association strength or feature overlap? A microanalytic review , 2003, Psychonomic bulletin & review.

[47]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[48]  J. H. Neely Semantic priming effects in visual word recognition: A selective review of current findings and theories. , 1991 .

[49]  C. Van Petten,et al.  Examining the N400 semantic context effect item-by-item: relationship to corpus-based measures of word co-occurrence. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[50]  G. Miller,et al.  Contextual correlates of semantic similarity , 1991 .

[51]  C. Petten Examining the N400 semantic context effect item-by-item: relationship to corpus-based measures of word co-occurrence. , 2014 .

[52]  Curt Burgess,et al.  The Dynamics of Meaning in Memory , 1998 .

[53]  Magnus Sahlgren,et al.  The Distributional Hypothesis , 2008 .

[54]  J. H. Neely,et al.  Semantic priming in the lexical decision task: roles of prospective prime-generated expectancies and retrospective semantic matching. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[55]  Marc Brysbaert,et al.  Wuggy: A multilingual pseudoword generator , 2010, Behavior research methods.

[56]  Curt Burgess,et al.  Modelling Parsing Constraints with High-dimensional Context Space , 1997 .

[57]  Rolf Ulrich,et al.  Inflation von falsch-positiven Befunden in der psychologischen Forschung : mögliche Ursachen und Gegenmaßnahmen , 2016 .

[58]  C. Chiarello,et al.  Another look at categorical priming in the cerebral hemispheres , 1992, Neuropsychologia.

[59]  M. Brysbaert,et al.  Explaining human performance in psycholinguistic tasks with models of semantic similarity based on prediction and counting : A review and empirical validation , 2017 .

[60]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[61]  A. D. Groot,et al.  Primed Lexical Decision: Combined Effects of the Proportion of Related Prime-Target Pairs and the Stimulus-Onset Asynchrony of Prime and Target , 1984 .

[62]  Thomas A. Schreiber,et al.  The University of South Florida free association, rhyme, and word fragment norms , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[63]  Peter Wiemer-Hastings,et al.  Latent semantic analysis , 2004, Annu. Rev. Inf. Sci. Technol..

[64]  Georgiana Dinu,et al.  DISSECT - DIStributional SEmantics Composition Toolkit , 2013, ACL.

[65]  J. H. Neely,et al.  Semantic Context Effects on Visual Word Processing: A Hybrid Prospective-Retrospective Processing Theory , 1989 .

[66]  Mante S. Nieuwland,et al.  Quantification, Prediction, and the Online Impact of Sentence Truth-Value: Evidence From Event-Related Potentials , 2015, Journal of experimental psychology. Learning, memory, and cognition.

[67]  W. Kintsch The role of knowledge in discourse comprehension: a construction-integration model. , 1988, Psychological review.

[68]  K I Forster,et al.  The potential for experimenter bias effects in word recognition experiments , 2000, Memory & cognition.

[69]  Carsten Eulitz,et al.  'Verstehen ' ('understand') primes ' stehen ' ('stand'): Morphological structure overrides semantic compositionality in the lexical representation of German complex verbs , 2014 .

[70]  M R Quillian,et al.  Word concepts: a theory and simulation of some basic semantic capabilities. , 1967, Behavioral science.

[71]  Timothy P. McNamara,et al.  Theories of priming. I : associative distance and lag , 1992 .

[72]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[73]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[74]  Michael W. Berry,et al.  Mathematical Foundations Behind Latent Semantic Analysis , 2007 .

[75]  Ehud Rivlin,et al.  Placing search in context: the concept revisited , 2002, TOIS.

[76]  W. Kintsch,et al.  High-Dimensional Semantic Space Accounts of Priming. , 2006 .