JECI++: A Modified Joint Knowledge Graph Embedding Model for Concepts and Instances

Abstract Concepts and instances are important parts in knowledge graphs, but most knowledge graph embedding models treat them as entities equally, that leads to inaccurate embeddings of concepts and instances. Aiming to solve this problem, we propose a novel knowledge graph embedding model called JECI++ to jointly embed concepts and instances. First, JECI++ simplifies hierarchical concepts based on subClassOf relation and instanceOf relation, then re-links instances to the simplified concepts as new instanceOf triples. Consequently, an instance can be obtained by its neighbor instances and its belonging simplified concepts. Second, circular convolution is utilized to locate an instance in the embedding space, based on neighbor instances and simplified concepts. Finally, simplified concepts and instances are jointly embedded by the embeddings learner with CBOW (Continuous Bag-of-Words) or Skip-Gram strategies. Especially, JECI++ can alleviate the problem of complex relations by incorporating neighbor information of instances. JECI++ is evaluated by link prediction and triple classification on real world datasets. Experimental results demonstrate that it outperforms state-of-the-art models in most cases.

[1]  Paolo Ciancarini,et al.  Empirical Analysis of Foundational Distinctions in Linked Open Data , 2018, IJCAI.

[2]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[3]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[4]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[5]  Yoshua Bengio,et al.  Word Representations: A Simple and General Method for Semi-Supervised Learning , 2010, ACL.

[6]  Jens Lehmann,et al.  DBpedia - A crystallization point for the Web of Data , 2009, J. Web Semant..

[7]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[8]  Li Guo,et al.  Improving Knowledge Graph Embedding Using Simple Constraints , 2018, ACL.

[9]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[10]  Jun Zhao,et al.  Knowledge Graph Embedding via Dynamic Mapping Matrix , 2015, ACL.

[11]  Peng Wang,et al.  JECI: A Joint Knowledge Graph Embedding Model for Concepts and Instances , 2019, JIST.

[12]  Gerhard Weikum,et al.  YAGO: A Large Ontology from Wikipedia and WordNet , 2008, J. Web Semant..

[13]  Markus Krötzsch,et al.  Wikidata , 2014 .

[14]  Huanbo Luan,et al.  Modeling Relation Paths for Representation Learning of Knowledge Bases , 2015, EMNLP.

[15]  Li Guo,et al.  Jointly Embedding Knowledge Graphs and Logical Rules , 2016, EMNLP.

[16]  Li Guo,et al.  Semantically Smooth Knowledge Graph Embedding , 2015, ACL.

[17]  Zhiyuan Liu,et al.  Representation Learning of Knowledge Graphs with Hierarchical Types , 2016, IJCAI.

[18]  Volker Tresp,et al.  Type-Constrained Representation Learning in Knowledge Graphs , 2015, SEMWEB.

[19]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[20]  Steffen Staab,et al.  Knowledge graphs , 2021, Commun. ACM.

[21]  Zhen Wang,et al.  Aligning Knowledge and Text Embeddings by Entity Descriptions , 2015, EMNLP.

[22]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[23]  Zhiyuan Liu,et al.  Differentiating Concepts and Instances for Knowledge Graph Embedding , 2018, EMNLP.

[24]  Yizhou Sun,et al.  Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts , 2019, KDD.

[25]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[26]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[27]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[28]  Jason Weston,et al.  A semantic matching energy function for learning with multi-relational data , 2013, Machine Learning.

[29]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[30]  Wei Zhang,et al.  Knowledge vault: a web-scale approach to probabilistic knowledge fusion , 2014, KDD.