Domain and Function: A Dual-Space Model of Semantic Relations and Compositions

Given appropriate representations of the semantic relations between carpenter and wood and between mason and stone (for example, vectors in a vector space model), a suitable algorithm should be able to recognize that these relations are highly similar (carpenter is to wood as mason is to stone; the relations are analogous). Likewise, with representations of dog, house, and kennel, an algorithm should be able to recognize that the semantic composition of dog and house, dog house, is highly similar to kennel (dog house and kennel are synonymous). It seems that these two tasks, recognizing relations and compositions, are closely connected. However, up to now, the best models for relations are significantly different from the best models for compositions. In this paper, we introduce a dual-space model that unifies these two tasks. This model matches the performance of the best previous models for relations and compositions. The dual-space model consists of a space for measuring domain similarity and a space for measuring function similarity. Carpenter and wood share the same domain, the domain of carpentry. Mason and stone share the same domain, the domain of masonry. Carpenter and mason share the same function, the function of artisans. Wood and stone share the same function, the function of materials. In the composition dog house, kennel has some domain overlap with both dog and house (the domains of pets and buildings). The function of kennel is similar to the function of house (the function of shelters). By combining domain and function similarities in various ways, we can model relations, compositions, and other aspects of semantics.

[1]  J. R. Firth,et al.  A Synopsis of Linguistic Theory, 1930-1955 , 1957 .

[2]  Noam Chomsky,et al.  The Logical Structure of Linguistic Theory , 1975 .

[3]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[4]  Zellig S. Harris,et al.  Distributional Structure , 1954 .

[5]  Dedre Gentner,et al.  Structure-Mapping: A Theoretical Framework for Analogy , 1983, Cogn. Sci..

[6]  N. J. A. Sloane,et al.  Sphere Packings, Lattices and Groups , 1987, Grundlehren der mathematischen Wissenschaften.

[7]  D. Over,et al.  Studies in the Way of Words. , 1989 .

[8]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[9]  C. Burgess,et al.  Semantic and associative priming in the cerebral hemispheres: Some words do, some words don't … sometimes, some places , 1990, Brain and Language.

[10]  Beatrice Santorini,et al.  Part-of-Speech Tagging Guidelines for the Penn Treebank Project (3rd Revision) , 1990 .

[11]  Beatrice Santorini.  Part-of-speech tagging guidelines for the penn treebank project , 1990 .

[12]  D. Gentner,et al.  Language and the career of similarity. , 1991 .

[13]  Geoffrey E. Hinton.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[14]  James Pustejovsky,et al.  The Generative Lexicon , 1995, CL.

[15]  A. Avramides.  Studies in the Way of Words , 1992 .

[16]  Marti A. Hearst.  Automatic Acquisition of Hyponyms from Large Text Corpora , 1992, COLING.

[17]  C. Daganzo.  THE CELL TRANSMISSION MODEL.. , 1994 .

[18]  Yoshihiko Nitta,et al.  Co-Occurrence Vectors From Corpora vs. Distance Vectors From Dictionaries , 1994, COLING.

[19]  Philip Resnik,et al.  Using Information Content to Evaluate Semantic Similarity in a Taxonomy , 1995, IJCAI.

[20]  Tony A. Plate,et al.  Holographic reduced representations , 1995, IEEE Trans. Neural Networks.

[21]  Graeme Hirst,et al.  Lexical chains as representations of context for the detection and correction of malapropisms , 1995 .

[22]  Gene H. Golub,et al.  Matrix Computations, Third Edition , 1996 .

[23]  Yves Lepage,et al.  Saussurian analogy: a theoretical account and its application , 1996, COLING.

[24]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[25]  David W. Conrath,et al.  Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy , 1997, ROCLING/IJCLCLP.

[26]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[27]  Martin Chodorow,et al.  Combining local context and wordnet similarity for word sense identification , 1998 .

[28]  Christiane Fellbaum,et al.  Combining Local Context and Wordnet Similarity for Word Sense Identification , 1998 .

[29]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[30]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[31]  W. Kintsch.  Metaphor comprehension: A computational theory , 2000, Psychonomic bulletin & review.

[32]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[33]  Walter Kintsch,et al.  Predication , 2001, Cogn. Sci..

[34]  Barbara Rosario,et al.  Classifying the Semantic Relations in Noun Compounds via a Domain-Specific Lexical Hierarchy , 2001, EMNLP.

[35]  John Caron,et al.  Experiments with LSA scoring: optimal rank and basis , 2001 .

[36]  Peter D. Turney.  Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL , 2001, ECML.

[37]  Patrick Pantel,et al.  Discovering word senses from text , 2002, KDD.

[38]  Ernest Lepore,et al.  The compositionality papers , 2002 .

[39]  Barbara Rosario,et al.  The Descent of Hierarchy, and Selection in Relational Semantics , 2002, ACL.

[40]  Thomas K. Landauer,et al.  On the computational basis of learning and cognition: Arguments from LSA , 2002 .

[41]  R. Rapp.  Word sense discovery based on sense descriptor dissimilarity , 2003, MTSUMMIT.

[42]  Diederik Aerts,et al.  LETTER TO THE EDITOR: Quantum aspects of semantic analysis and symbolic artificial intelligence , 2003, quant-ph/0309022.

[43]  Jeffrey P. Bigham,et al.  Combining Independent Modules to Solve Multiple-choice Synonym and Analogy Problems , 2003, ArXiv.

[44]  J. Quesada,et al.  Analogy-making as Predication Using Relational Information and LSA Vectors , 2004 .

[45]  Tony Veale,et al.  WordNet Sits the S.A.T. - A Knowledge-Based Approach to Lexical Analogy , 2004, ECAI.

[46]  Charles L. A. Clarke,et al.  Efficiency vs. Effectiveness in Terabyte-Scale Information Retrieval , 2005, TREC.

[47]  Michael L. Littman,et al.  Corpus-based Learning of Analogies and Semantic Relations , 2005, Machine Learning.

[48]  Preslav Nakov,et al.  Using Verbs to Characterize Noun-Noun Relations , 2006, AIMSA.

[49]  Magnus Sahlgren,et al.  The Word-Space Model: using distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector spaces , 2006 .

[50]  Deniz Yuret,et al.  Clustering Word Pairs to Answer Analogy Questions , 2006 .

[51]  Peter D. Turney.  Similarity of Semantic Relations , 2006, CL.

[52]  Stan Szpakowicz,et al.  Learning Noun-Modifier Semantic Relations with Corpus-based and WordNet-based Features , 2006, AAAI.

[53]  Peter D. Turney.  Expressing Implicit Semantic Relations without Supervision , 2006, ACL.

[54]  Ergun Biçici.  Clustering Word Pairs to Answer Analogy Questions , 2006 .

[55]  Preslav Nakov,et al.  UCB: System Description for SemEval Task #4 , 2007, SemEval@ACL.

[56]  Stephen Clark,et al.  Combining Symbolic and Distributional Models of Meaning , 2007, AAAI Spring Symposium: Quantum Interaction.

[57]  Michael N Jones,et al.  Representing word meaning and order information in a composite holographic lexicon. , 2007, Psychological review.

[58]  Danielle S. McNamara,et al.  Handbook of latent semantic analysis , 2007 .

[59]  J. Bullinaria,et al.  Extracting semantic representations from word co-occurrence statistics: A computational study , 2007, Behavior research methods.

[60]  Ari Rappoport,et al.  Unsupervised Discovery of Generic Relationships Using Pattern Clusters and its Evaluation by Automatically Generated SAT Analogy Questions , 2008, ACL.

[61]  Peter D. Turney.  A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations , 2008, COLING.

[62]  Dominic Widdows,et al.  Semantic Vector Products: Some Initial Investigations , 2008 .

[63]  Alessandro Moschitti,et al.  Kernels on Linguistic Structures for Answer Extraction , 2008, ACL.

[64]  Mirella Lapata,et al.  Vector-based Models of Semantic Composition , 2008, ACL.

[65]  Katrin Erk,et al.  A Structured Vector Space Model for Word Meaning in Context , 2008, EMNLP.

[66]  Peter D. Turney.  The Latent Relation Mapping Engine: Algorithm and Experiments , 2008, J. Artif. Intell. Res..

[67]  S. Clark,et al.  A Compositional Distributional Model of Meaning , 2008 .

[68]  Akira Utsumi.  Computational Semantics of Noun Compounds in a Semantic Space Model , 2009, IJCAI.

[69]  Marco Baroni,et al.  BagPack: A General Framework to Represent Semantic Relations , 2009, ArXiv.

[70]  Danushka Bollegala,et al.  Measuring the similarity between implicit semantic relations from the web , 2009, WWW '09.

[71]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[72]  Diarmuid Ó Séaghdha,et al.  Using Lexical and Relational Similarity to Classify Semantic Relations , 2009, EACL.

[73]  Marco Baroni,et al.  Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space , 2010, EMNLP.

[74]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[75]  Stefan Thater,et al.  Contextualizing Semantic Representations Using Syntactically Enriched Vector Models , 2010, ACL.

[76]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[77]  Christopher D. Manning,et al.  Learning Continuous Phrase Representations and Syntactic Parsing with Recursive Neural Networks , 2010 .

[78]  E. Guevara.  A Regression Model of Adjective-Noun Compositionality in Distributional Semantics , 2010 .

[79]  Anders Søgaard,et al.  Shared Task System Description: Frustratingly Hard Compositionality Prediction , 2011 .

[80]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[81]  Christian Biemann,et al.  Distributional Semantics and Compositionality 2011: Shared Task Description and Results , 2011 .

[82]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[83]  Mehrnoosh Sadrzadeh,et al.  Experimenting with transitive verbs in a DisCoCat , 2011, GEMS.

[84]  Hui Lin,et al.  A Class of Submodular Functions for Document Summarization , 2011, ACL.

[85]  Saif Mohammad,et al.  SemEval-2012 Task 2: Measuring Degrees of Relational Similarity , 2012, *SEMEVAL.

[86]  K. McRae,et al.  Semantic and associative relations in adolescents and young adults: Examining a tenuous dichotomy. , 2012 .

[87]  N. Foo.  Conceptual Spaces—The Geometry of Thought , 2022 .