A convolutional neural network-based model for knowledge base completion and its application to search personalization

In this paper, we propose a novel embedding model, named ConvKB, for knowledge base completion. Our model ConvKB advances state-of-the-art models by employing a convolutional neural network, so that it can capture global relationships and transitional characteristics between entities and relations in knowledge bases. In ConvKB, each triple (head entity, relation, tail entity) is represented as a 3-column matrix where each column vector represents a triple element. This 3-column matrix is then fed to a convolution layer where multiple filters are operated on the matrix to generate different feature maps. These feature maps are then concatenated into a single feature vector representing the input triple. The feature vector is multiplied with a weight vector via a dot product to return a score. This score is then used to predict whether the triple is valid or not. Experiments show that ConvKB obtains better link prediction and triple classification results than previous state-of-the-art models on benchmark datasets WN18RR, FB15k-237, WN11 and FB13. We further apply our ConvKB to a search personalization problem which aims to tailor the search results to each specific user based on the user’s personal interests and preferences. In particular, we model the potential relationship between the submitted query, the user and the search result (i.e., document) as a triple (query, user, document) on which the ConvKB is able to work. Experimental results on query logs from a commercial web search engine show that ConvKB achieves better performances than the standard ranker as well as strong search personalization baselines.

[1]  Gerhard Weikum,et al.  WWW 2007 / Track: Semantic Web Session: Ontologies ABSTRACT YAGO: A Core of Semantic Knowledge , 2022 .

[2]  Susan T. Dumais,et al.  Personalizing search via automated analysis of interests and activities , 2005, SIGIR '05.

[3]  Wei Chu,et al.  Modeling the impact of short- and long-term behavior on search personalization , 2012, SIGIR '12.

[4]  Hua Wu,et al.  An End-to-End Model for Question Answering over Knowledge Base with Cross-Attention Combining Global Knowledge , 2017, ACL.

[5]  Dat Quoc Nguyen An overview of embedding models of entities and relationships for knowledge base completion , 2017, ArXiv.

[6]  Wei Chu,et al.  Enhancing personalized search by mining and modeling task behavior , 2013, WWW.

[7]  Meredith Ringel Morris,et al.  Discovering and using groups to improve personalized search , 2009, WSDM '09.

[8]  Hua Wu,et al.  Question Answering over Knowledge Base with Neural Attention Combining Global Knowledge Information , 2016, ArXiv.

[9]  Ming-Wei Chang,et al.  Traversing Knowledge Graph in Vector Space without Symbolic Space Guidance , 2016 .

[10]  Rahul Gupta,et al.  Knowledge base completion via search-based question answering , 2014, WWW.

[11]  Juan-Zi Li,et al.  Text-Enhanced Representation Learning for Knowledge Graph , 2016, IJCAI.

[12]  Dawei Song,et al.  Temporal Latent Topic User Profiles for Search Personalisation , 2015, ECIR.

[13]  Simone Paolo Ponzetto,et al.  Knowledge-based graph document modeling , 2014, WSDM.

[14]  Dawei Song,et al.  Improving search personalisation with dynamic group formation , 2014, SIGIR.

[15]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Fabio Crestani,et al.  Building user profiles from topic models for personalised search , 2013, CIKM.

[18]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[19]  Jun Zhao,et al.  Knowledge Graph Embedding via Dynamic Mapping Matrix , 2015, ACL.

[20]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[22]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[23]  Manolis Vavalis,et al.  A network-based model for high-dimensional information filtering , 2010, SIGIR '10.

[24]  Lizhen Qu,et al.  Neighborhood Mixture Model for Knowledge Base Completion , 2016, CoNLL.

[25]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[26]  Xin Liu,et al.  Modeling Users' Dynamic Preference for Personalized Recommendation , 2015, IJCAI.

[27]  Evgeniy Gabrilovich,et al.  A Review of Relational Machine Learning for Knowledge Graphs , 2015, Proceedings of the IEEE.

[28]  Hoifung Poon,et al.  Compositional Learning of Embeddings for Relation Paths in Knowledge Base and Text , 2016, ACL.

[29]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[30]  Jun Zhao,et al.  Knowledge Graph Completion with Adaptive Sparse Transfer Matrix , 2016, AAAI.

[31]  Lizhen Qu,et al.  STransE: a novel embedding model of entities and relationships in knowledge bases , 2016, NAACL.

[32]  Han Xiao,et al.  From One Point to a Manifold: Knowledge Graph Embedding for Precise Link Prediction , 2015, IJCAI.

[33]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[34]  John Miller,et al.  Traversing Knowledge Graphs in Vector Space , 2015, EMNLP.

[35]  Jaime Teevan,et al.  Understanding and predicting personal navigation , 2011, WSDM '11.

[36]  Yu Hao,et al.  Knowlege Graph Embedding by Flexible Translation , 2015, ArXiv.

[37]  Li Guo,et al.  Context-Dependent Knowledge Graph Embedding , 2015, EMNLP.

[38]  Danqi Chen,et al.  Observed versus latent features for knowledge base and text inference , 2015, CVSC.

[39]  M. de Rijke,et al.  Behavior‐based personalization in web search , 2017, J. Assoc. Inf. Sci. Technol..

[40]  Han Xiao,et al.  TransG : A Generative Model for Knowledge Graph Embedding , 2015, ACL.

[41]  Tom M. Mitchell,et al.  Leveraging Knowledge Bases in LSTMs for Improving Machine Reading , 2017, ACL.

[42]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[43]  Seong-Bae Park,et al.  A Translation-Based Knowledge Graph Embedding Preserving Logical Property of Relations , 2016, HLT-NAACL.

[44]  Fan Yang,et al.  Differentiable Learning of Logical Rules for Knowledge Base Reasoning , 2017, NIPS.

[45]  Dai Quoc Nguyen,et al.  A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network , 2017, NAACL.

[46]  Huanbo Luan,et al.  Modeling Relation Paths for Representation Learning of Knowledge Bases , 2015, EMNLP.

[47]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[48]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[49]  Mihajlo Grbovic,et al.  Search Ranking And Personalization at Airbnb , 2017, RecSys.

[50]  Mathias Niepert,et al.  KBlrn: End-to-End Learning of Knowledge Base Representations with Latent, Relational, and Numerical Features , 2017, UAI.

[51]  Jens Lehmann,et al.  DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia , 2015, Semantic Web.

[52]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[53]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[54]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[55]  William Yang Wang,et al.  KBGAN: Adversarial Learning for Knowledge Graph Embeddings , 2017, NAACL.

[56]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[57]  Steven C. H. Hoi,et al.  On Effective Personalized Music Retrieval by Exploring Online User Behaviors , 2016, SIGIR.

[58]  Michael Gamon,et al.  Representing Text for Joint Embedding of Text and Knowledge Bases , 2015, EMNLP.

[59]  Ji-Rong Wen,et al.  A large-scale evaluation and analysis of personalized search strategies , 2007, WWW '07.

[60]  Dat Quoc Nguyen,et al.  Search Personalization with Embeddings , 2017, ECIR.

[61]  Gerhard Weikum,et al.  NAGA: Searching and Ranking Knowledge , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[62]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[63]  Udo Kruschwitz,et al.  Personalised Query Suggestion for Intranet Search with Temporal User Profiling , 2017, CHIIR.

[64]  Milad Shokouhi,et al.  Modeling User Interests for Zero-Query Ranking , 2016, ECIR.

[65]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[66]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[67]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[68]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[69]  James P. Callan,et al.  Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding , 2017, WWW.

[70]  Eduard H. Hovy,et al.  An Interpretable Knowledge Transfer Model for Knowledge Base Completion , 2017, ACL.