Distributed Prediction of Relations for Entities: The Easy, The Difficult, and The Impossible

Word embeddings are supposed to provide easy access to semantic relations such as “male of” (man–woman). While this claim has been investigated for concepts, little is known about the distributional behavior of relations of (Named) Entities. We describe two word embedding-based models that predict values for relational attributes of entities, and analyse them. The task is challenging, with major performance differences between relations. Contrary to many NLP tasks, high difficulty for a relation does not result from low frequency, but from (a) one-to-many mappings; and (b) lack of context patterns expressing the relation that are easy to pick up by word embeddings.

[1]  Sabine Schulte im Walde,et al.  Multilingual Reliability and “Semantic” Structure of Continuous Word Spaces , 2015, IWCS.

[2]  Gemma Boleda,et al.  Distributional vectors encode referential attributes , 2015, EMNLP.

[3]  Deniz Yuret,et al.  Probabilistic Modeling of Joint-context in Distributional Similarity , 2014, CoNLL.

[4]  Aurélie Herbelot,et al.  Mr Darcy and Mr Toad, gentlemen: distributional names and their kinds , 2015, IWCS.

[5]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[6]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[7]  John Miller,et al.  Traversing Knowledge Graphs in Vector Space , 2015, EMNLP.

[8]  J. Fodor,et al.  All at sea in semantic space : Churchland on meaning similarity , 1999 .

[9]  Michael Gamon,et al.  Representing Text for Joint Embedding of Text and Knowledge Bases , 2015, EMNLP.

[10]  Paul Buitelaar,et al.  A Distributional Semantics Approach for Selective Reasoning on Commonsense Graph Knowledge Bases , 2014, NLDB.

[11]  Ted Briscoe,et al.  Looking for Hyponyms in Vector Space , 2014, CoNLL.

[12]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[13]  Tom M. Mitchell,et al.  Learning a Compositional Semantics for Freebase with an Open Predicate Vocabulary , 2015, TACL.

[14]  Omer Levy,et al.  Dependency-Based Word Embeddings , 2014, ACL.

[15]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[16]  Omer Levy,et al.  Linguistic Regularities in Sparse and Explicit Word Representations , 2014, CoNLL.

[17]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[18]  Geoffrey E. Hinton.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[19]  Ming-Wei Chang,et al.  Inferring Missing Entity Type Instances for Knowledge Base Completion: New Dataset and Methods , 2015, NAACL.