A Structured Vector Space Model for Word Meaning in Context

We address the task of computing vector space representations for the meaning of word occurrences, which can vary widely according to context. This task is a crucial step towards a robust, vector-based compositional account of sentence meaning. We argue that existing models for this task do not take syntactic structure sufficiently into account. We present a novel structured vector space model that addresses these issues by incorporating the selectional preferences for words' argument positions. This makes it possible to integrate syntax into the computation of word meaning in context. In addition, the model performs at and above the state of the art for modeling the contextual adequacy of paraphrases.

[1]  P. Resnik.  Selectional constraints: an information-theoretic model and its computational realization , 1996, Cognition.

[2]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[3]  Diana McCarthy,et al.  Disambiguating Nouns, Verbs, and Adjectives Using Automatically Acquired Selectional Preferences , 2003, CL.

[4]  Mirella Lapata,et al.  Evaluating and Combining Approaches to Selectional Preference Acquisition , 2003, EACL.

[5]  Christina L. Gagné,et al.  Thematic role focusing by participle inflections: evidence from conceptual combination. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[6]  Frank Keller,et al.  Combining Syntax and Thematic Fit in a Probabilistic Model of Sentence Processing , 2006 .

[7]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[8]  Alessandro Moschitti,et al.  Kernels on Linguistic Structures for Answer Extraction , 2008, ACL.

[9]  Serge Sharoff,et al.  Open-source Corpora: Using the net to fish for linguistic data , 2006 .

[10]  J. Elman,et al.  A basis for generating expectancies for verbs from nouns , 2005, Memory & cognition.

[11]  Daniel Jurafsky,et al.  A Bayesian Model Predicts Human Parse Preference and Reading Times in Sentence Processing , 2001, NIPS.

[12]  Geoffrey E. Hinton.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[13]  Adam Kilgarriff,et al.  "I Don’t Believe in Word Senses" , 1997, Comput. Humanit..

[14]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[15]  Mirella Lapata,et al.  Dependency-Based Construction of Semantic Space Models , 2007, CL.

[16]  Katrin Erk,et al.  A Simple, Similarity-based Model for Selectional Preferences , 2007, ACL.

[17]  Walter Kintsch,et al.  Predication , 2001, Cogn. Sci..

[18]  Haim Levkowitz,et al.  Introduction to information retrieval (IR) , 2008 .

[19]  Ido Dagan,et al.  Contextual Preferences , 2008, ACL.

[20]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[21]  Alexander S. Yeh,et al.  More accurate tests for the statistical significance of result differences , 2000, COLING.

[22]  Dekang Lin,et al.  Principle-Based Parsing Without Overgeneration , 1993, ACL.

[23]  Michael Ramscar,et al.  Testing the Distributioanl Hypothesis: The influence of Context on Judgements of Semantic Similarity , 2001 .

[24]  Mirella Lapata,et al.  Vector-based Models of Semantic Composition , 2008, ACL.

[25]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[26]  Dekang Lin,et al.  Automatic Retrieval and Clustering of Similar Words , 1998, ACL.

[27]  Roberto Navigli,et al.  SemEval-2007 Task 10: English Lexical Substitution Task , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[28]  Scott McDonald,et al.  A Distributional Model of Semantic Context Effects in Lexical Processing , 2004, ACL.

[29]  Katrin Erk,et al.  Flexible, Corpus-Based Modelling of Human Plausibility Judgements , 2007, EMNLP.

[30]  Mats Rooth,et al.  Structural Ambiguity and Lexical Relations , 1991, ACL.

[31]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[32]  Michael N Jones,et al.  Representing word meaning and order information in a composite holographic lexicon. , 2007, Psychological review.

[33]  Daniel Gildea,et al.  Automatic Labeling of Semantic Roles , 2000, ACL.

[34]  M. Tanenhaus,et al.  Modeling the Influence of Thematic Fit (and Other Constraints) in On-line Sentence Comprehension , 1998 .

[35]  J. Fodor,et al.  The structure of a semantic theory , 1963 .