Contextual Path Retrieval: A Contextual Entity Relation Embedding-based Approach

Contextual path retrieval (CPR) refers to the task of finding contextual path(s) between a pair of entities in a knowledge graph that explains the connection between them in a given context. For this novel retrieval task, we propose the Embedding-based Contextual Path Retrieval (ECPR) framework. ECPR is based on a three-component structure that includes a context encoder and path encoder that encode query context and path, respectively, and a path ranker that assigns a ranking score to each candidate path to determine the one that should be the contextual path. For context encoding, we propose two novel context encoding methods, i.e., context-fused entity embeddings and contextualized embeddings. For path encoding, we propose PathVAE, an inductive embedding approach to generate path representations. Finally, we explore two path-ranking approaches. In our evaluation, we construct a synthetic dataset from Wikipedia and two real datasets of Wikinews articles constructed through crowdsourcing. Our experiments show that methods based on ECPR framework outperform baseline methods, and that our two proposed context encoders yield significantly better performance than baselines. We also analyze a few case studies to show the distinct features of ECPR-based methods.

[1]  Hongzhi Yin,et al.  Temporal Meta-path Guided Explainable Recommendation , 2021, WSDM.

[2]  Sebastian Bruch,et al.  An Alternative Cross Entropy Loss for Learning-to-Rank , 2019, WWW.

[3]  Zhiyuan Liu,et al.  KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation , 2019, Transactions of the Association for Computational Linguistics.

[4]  Laura Dietz,et al.  Why does this Entity matter?: Support Passage Retrieval for Entity Retrieval , 2019, ICTIR.

[5]  Zhe Zhao,et al.  K-BERT: Enabling Language Representation with Knowledge Graph , 2019, AAAI.

[6]  Chengsheng Mao,et al.  KG-BERT: BERT for Knowledge Graph Completion , 2019, ArXiv.

[7]  Yongfeng Zhang,et al.  Reinforcement Knowledge Graph Reasoning for Explainable Recommendation , 2019, SIGIR.

[8]  Yixin Cao,et al.  KGAT: Knowledge Graph Attention Network for Recommendation , 2019, KDD.

[9]  Lei Li,et al.  Dynamically Fused Graph Network for Multi-hop Reasoning , 2019, ACL.

[10]  Kamalakar Karlapalem,et al.  Scalable Knowledge Graph Construction over Text using Deep Learning based Predicate Mapping , 2019, WWW.

[11]  Yixin Cao,et al.  Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences , 2019, WWW.

[12]  Yixin Cao,et al.  Explainable Reasoning over Knowledge Graphs for Recommendation , 2018, AAAI.

[13]  Sumit Bhatia,et al.  That's Interesting, Tell Me More! Finding Descriptive Support Passages for Knowledge Graph Relationships , 2018, SEMWEB.

[14]  Jian-Yun Nie,et al.  RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space , 2018, ICLR.

[15]  Yoshua Bengio,et al.  HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering , 2018, EMNLP.

[16]  Richard Socher,et al.  Multi-Hop Knowledge Graph Reasoning with Reward Shaping , 2018, EMNLP.

[17]  Philip S. Yu,et al.  Leveraging Meta-path based Context for Top- N Recommendation with A Neural Co-Attention Model , 2018, KDD.

[18]  Jonathan Berant,et al.  The Web as a Knowledge-Base for Answering Complex Questions , 2018, NAACL.

[19]  Xu Chen,et al.  Learning over Knowledge-Base Embeddings for Recommendation , 2018, Algorithms.

[20]  Xiaoyan Zhu,et al.  An Interpretable Reasoning Network for Multi-Relation Question Answering , 2018, COLING.

[21]  Dai Quoc Nguyen,et al.  A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network , 2017, NAACL.

[22]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[23]  Sebastian Riedel,et al.  Constructing Datasets for Multi-hop Reading Comprehension Across Documents , 2017, TACL.

[24]  Amina Kadry,et al.  Open Relation Extraction for Support Passage Retrieval: Merit and Open Issues , 2017, SIGIR.

[25]  Dik Lun Lee,et al.  Meta-Graph Based Recommendation Fusion over Heterogeneous Information Networks , 2017, KDD.

[26]  Wenhan Xiong,et al.  DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning , 2017, EMNLP.

[27]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[28]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[29]  M. de Rijke,et al.  Generating Descriptions of Entity Relationships , 2017, ECIR.

[30]  Guillaume Bouchard,et al.  Knowledge Graph Completion via Complex Tensor Factorization , 2017, J. Mach. Learn. Res..

[31]  Tim Weninger,et al.  ProjE: Embedding Projection for Knowledge Graph Completion , 2016, AAAI.

[32]  Lei Zou,et al.  Semantic SPARQL Similarity Search Over RDF Knowledge Graphs , 2016, Proc. VLDB Endow..

[33]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[34]  Nitish Aggarwal,et al.  Connecting the Dots: Explaining Relationships Between Unconnected Entities in a Knowledge Graph , 2016, ESWC.

[35]  Dongyan Zhao,et al.  Question Answering on Freebase via Relation Extraction and Textual Evidence , 2016, ACL.

[36]  David Vandyke,et al.  Counter-fitting Word Vectors to Linguistic Constraints , 2016, NAACL.

[37]  Jun Zhao,et al.  Knowledge Graph Completion with Adaptive Sparse Transfer Matrix , 2016, AAAI.

[38]  Hiroyuki Shindo,et al.  Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation , 2016, CoNLL.

[39]  Giuseppe Pirrò,et al.  Explaining and Suggesting Relatedness in Knowledge Graphs , 2015, SEMWEB.

[40]  M. de Rijke,et al.  Learning to Explain Entity Relationships in Knowledge Graphs , 2015, ACL.

[41]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[42]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[43]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[44]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[45]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[46]  Rahul Gupta,et al.  Knowledge base completion via search-based question answering , 2014, WWW.

[47]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[48]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[49]  Gerhard Weikum,et al.  Robust Disambiguation of Named Entities in Text , 2011, EMNLP.

[50]  Roi Blanco,et al.  Finding support sentences for entities , 2010, SIGIR.

[51]  Christopher J. C. Burges,et al.  From RankNet to LambdaRank to LambdaMART: An Overview , 2010 .

[52]  Conor Hayes,et al.  Using Linked Data to Build Open, Collaborative Recommender Systems , 2010, AAAI Spring Symposium: Linked Data Meets Artificial Intelligence.

[53]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[54]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[55]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[56]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[57]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[58]  Jimmy J. Lin,et al.  Scalable Knowledge Graph Construction from Text Collections , 2019, EMNLP.

[59]  Tie-Yan Liu,et al.  A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS 2017.

[60]  Quanquan Gu,et al.  Collaborative Filtering with Entity Similarity Regularization in Heterogeneous Information Networks , 2013 .