Word-embeddings Italian semantic spaces: A semantic model for psycholinguistic research

Distributional semantics has been for long a source of successful models in psycholinguistics, permitting to obtain semantic estimates for a large number of words in an automatic and fast way. However, resources in this respect remain scarce or limitedly accessible for languages different from English. The present paper describes WEISS (Word-Embeddings Italian Semantic Space), a distributional semantic model based on Italian. WEISS includes models of semantic representations that are trained adopting state-of-the-art word-embeddings methods, applying neural networks to induce distributed representations for lexical meanings. The resource is evaluated against two test sets, demonstrating that WEISS obtains a better performance with respect to a baseline encoding word associations. Moreover, an extensive qualitative analysis of the WEISS output provides examples of the model potentialities in capturing several semantic phenomena. Two variants of WEISS are released and made easily accessible via web through the SNAUT graphic interface.

[1]  Angeliki Lazaridou,et al.  Multimodal Word Meaning Induction From Minimal Exposure to Natural Text. , 2017, Cognitive science.

[2]  Georgiana Dinu,et al.  DISSECT - DIStributional SEmantics Composition Toolkit , 2013, ACL.

[3]  R. Holloway.  The broth in my brother ’ s brothel : Morpho-orthographic segmentation in visual word recognition , 2005 .

[4]  M. Marelli,et al.  Affixation in semantic space: Modeling morpheme meanings with compositional distributional semantics. , 2015, Psychological review.

[5]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[6]  Peter Pirolli,et al.  Modeling Information Scent: A Comparison of LSA, PMI and GLSA Similarity Measures on Common Tests and Corpora , 2007, RIAO.

[7]  Omer Levy,et al.  Neural Word Embedding as Implicit Matrix Factorization , 2014, NIPS.

[8]  Trevor Bekolay,et al.  A Large-Scale Model of the Functioning Brain , 2012, Science.

[9]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[10]  Magnus Sahlgren,et al.  The Distributional Hypothesis , 2008 .

[11]  Geoff Hollis,et al.  The principals of meaning: Extracting semantic dimensions from co-occurrence models of semantics , 2016, Psychonomic Bulletin & Review.

[12]  Zellig S. Harris,et al.  Distributional Structure , 1954 .

[13]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[14]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[15]  Magnus Sahlgren.  Towards a Flexible Model of Word Meaning , 2002, AAAI 2002.

[16]  Curt Burgess,et al.  Characterizing semantic space: Neighborhood effects in word recognition , 2001, Psychonomic bulletin & review.

[17]  Sudeep Bhatia,et al.  The semantic representation of prejudice and stereotypes , 2017, Cognition.

[18]  J. Bullinaria,et al.  Extracting semantic representations from word co-occurrence statistics: A computational study , 2007, Behavior research methods.

[19]  Adam Tauman Kalai,et al.  Quantifying and Reducing Stereotypes in Word Embeddings , 2016, ArXiv.

[20]  Michael Ramscar,et al.  Testing the Distributional Hypothesis 3 Testing the Distributional Hypothesis : The Influence of Context on Judgments of Semantic Similarity The Distributional Hypothesis , 2009 .

[21]  W. Kintsch,et al.  High-Dimensional Semantic Space Accounts of Priming. , 2006 .

[22]  Sterling Hutchinson,et al.  Language statistics explain the spatial–numerical association of response codes , 2014, Psychonomic bulletin & review.

[23]  Max M. Louwerse,et al.  Symbol Interdependency in Symbolic and Embodied Cognition , 2011, Top. Cogn. Sci..

[24]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[25]  Georgiana Dinu,et al.  Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors , 2014, ACL.

[26]  M. Marelli,et al.  Corpus-based estimates of word association predict biases in judgment of word co-occurrence likelihood , 2014, Cognitive Psychology.

[27]  Quoc V. Le,et al.  Exploiting Similarities among Languages for Machine Translation , 2013, ArXiv.

[28]  Adam Kilgarriff,et al.  Large Linguistically-Processed Web Corpora for Multiple Languages , 2006, EACL.

[29]  Arvind Narayanan,et al.  Semantics derived automatically from language corpora contain human-like biases , 2016, Science.

[30]  L. L. Jones,et al.  Pure mediated priming: a retrospective semantic matching model. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[31]  Marco Marelli,et al.  Semantic Transparency in Free Stems: The Effect of Orthography-Semantics Consistency on Word Recognition , 2015, Quarterly journal of experimental psychology.

[32]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[33]  Max M. Louwerse,et al.  Representing Spatial Structure Through Maps and Language: Lord of the Rings Encodes the Spatial Structure of Middle Earth , 2012, Cogn. Sci..

[34]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[35]  Gina R Kuperberg,et al.  An electrophysiological investigation of indirect semantic priming. , 2006, Psychophysiology.

[36]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[37]  Massimo Poesio,et al.  EEG responds to conceptual stimuli and corpus semantics , 2009, EMNLP.

[38]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[39]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[40]  Remo Job,et al.  Le associazioni verbali PD-DPSS: norme per 294 parole , 2002 .

[41]  M. Marelli,et al.  Picking buttercups and eating butter cups: Spelling alternations, semantic relatedness, and their consequences for compound processing , 2014, Applied Psycholinguistics.

[42]  M. Brysbaert,et al.  Explaining human performance in psycholinguistic tasks with models of semantic similarity based on prediction and counting : A review and empirical validation , 2017 .