A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization

In this paper, we introduce an embedding model, named CapsE, exploring a capsule network to model relationship triples (subject, relation, object). Our CapsE represents each triple as a 3-column matrix where each column vector represents the embedding of an element in the triple. This 3-column matrix is then fed to a convolution layer where multiple filters are operated to generate different feature maps. These feature maps are reconstructed into corresponding capsules which are then routed to another capsule to produce a continuous vector. The length of this vector is used to measure the plausibility score of the triple. Our proposed CapsE obtains better performance than previous state-of-the-art embedding models for knowledge graph completion on two benchmark datasets WN18RR and FB15k-237, and outperforms strong search personalization baselines on SEARCH17.

[1]  Jaime Teevan,et al.  Understanding and predicting personal navigation , 2011, WSDM '11.

[2]  Dai Quoc Nguyen,et al.  A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network , 2017, NAACL.

[3]  Wei Chu,et al.  Modeling the impact of short- and long-term behavior on search personalization , 2012, SIGIR '12.

[4]  Wei Chu,et al.  Enhancing personalized search by mining and modeling task behavior , 2013, WWW.

[5]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[6]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[7]  Jacob Eisenstein,et al.  Predicting Semantic Relations using Global Graph Properties , 2018, EMNLP.

[8]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[9]  Lizhen Qu,et al.  STransE: a novel embedding model of entities and relationships in knowledge bases , 2016, NAACL.

[10]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[11]  Danqi Chen,et al.  Observed versus latent features for knowledge base and text inference , 2015, CVSC.

[12]  Xin Liu,et al.  Modeling Users' Dynamic Preference for Personalized Recommendation , 2015, IJCAI.

[13]  Meredith Ringel Morris,et al.  Discovering and using groups to improve personalized search , 2009, WSDM '09.

[14]  Susan T. Dumais,et al.  Personalizing Search via Automated Analysis of Interests and Activities , 2005, SIGIR.

[15]  Dawei Song,et al.  Temporal Latent Topic User Profiles for Search Personalisation , 2015, ECIR.

[16]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[17]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[18]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[19]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[20]  William Yang Wang,et al.  KBGAN: Adversarial Learning for Knowledge Graph Embeddings , 2017, NAACL.

[21]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[22]  Lizhen Qu,et al.  Neighborhood Mixture Model for Knowledge Base Completion , 2016, CoNLL.

[23]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[24]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[25]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[26]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[27]  Rahul Gupta,et al.  Knowledge base completion via search-based question answering , 2014, WWW.

[28]  Dat Quoc Nguyen.  An overview of embedding models of entities and relationships for knowledge base completion , 2017, ArXiv.

[29]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[30]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[31]  Evgeniy Gabrilovich,et al.  A Review of Relational Machine Learning for Knowledge Graphs , 2015, Proceedings of the IEEE.

[32]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[33]  Huanbo Luan,et al.  Modeling Relation Paths for Representation Learning of Knowledge Bases , 2015, EMNLP.

[34]  Jun Zhao,et al.  Knowledge Graph Embedding via Dynamic Mapping Matrix , 2015, ACL.

[35]  Fabio Crestani,et al.  Building user profiles from topic models for personalised search , 2013, CIKM.

[36]  Dat Quoc Nguyen,et al.  Search Personalization with Embeddings , 2017, ECIR.