Connecting emerging relationships from news via tensor factorization

Knowledge graphs (KGs) have been widely used to represent relationships among entities, while KGs cannot capture new relationships between entities emerging along time. Since news often provides the latest information regarding the new entities and relationships, there is an opportunity to connect emerging relationships from news timely. However, it is a challenging task due to the source heterogeneity of structured KGs and unstructured news texts. In order to address the issue, we propose a tensor-based framework to capture the complex interactions among multiple types of relations, entities and text descriptions. We further develop an efficient Text-Aware MUlti-RElational learning method (TAMURE) that can learn the embedding representations of entities and relation types from both KGs and news, by jointly factorizing the interaction parameters. Furthermore, the complexity of TAMURE is linear in the number of parameters, which makes it suitable to large-scale KGs and news texts. Extensive experiments via TensorFlow demonstrate the effectiveness of the proposed TAMURE model compared with nine state-of-the-art methods on real-world datasets.

[1]  Jun Zhao,et al.  Knowledge Graph Embedding via Dynamic Mapping Matrix , 2015, ACL.

[2]  Gerhard Weikum,et al.  Discovering emerging entities with ambiguous names , 2014, WWW.

[3]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[4]  Antoine Bordes,et al.  Effective Blending of Two and Three-way Interactions for Modeling Multi-relational Data , 2014, ECML/PKDD.

[5]  Tom M. Mitchell,et al.  Random Walk Inference and Learning in A Large Scale Knowledge Base , 2011, EMNLP.

[6]  Xueyan Jiang,et al.  Reducing the Rank in Relational Factorization Models by Including Observable Patterns , 2014, NIPS.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[9]  Zhen Wang,et al.  Knowledge Graph and Text Jointly Embedding , 2014, EMNLP.

[10]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[11]  Philip S. Yu,et al.  Multi-view Machines , 2015, WSDM.

[12]  Christopher D. Manning,et al.  Combining Distant and Partial Supervision for Relation Extraction , 2014, EMNLP.

[13]  Philip S. Yu,et al.  Multilinear Factorization Machines for Multi-Task Multi-View Learning , 2017, WSDM.

[14]  Satoshi Sekine,et al.  A survey of named entity recognition and classification , 2007 .

[15]  Wei Zhang,et al.  Knowledge vault: a web-scale approach to probabilistic knowledge fusion , 2014, KDD.

[16]  Grigorios Tsoumakas,et al.  Mining Multi-label Data , 2010, Data Mining and Knowledge Discovery Handbook.

[17]  Philip S. Yu,et al.  Identifying Connectivity Patterns for Brain Diseases via Multi-side-view Guided Deep Architectures , 2016, SDM.

[18]  Philip S. Yu,et al.  Mining Brain Networks Using Multiple Side Views for Neurological Disorder Identification , 2015, 2015 IEEE International Conference on Data Mining.

[19]  Chun-Ta Lu,et al.  HEER: Heterogeneous graph embedding for emerging relation detection from news , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[20]  Tom M. Mitchell,et al.  Incorporating Vector Space Similarity in Random Walk Inference over Knowledge Bases , 2014, EMNLP.

[21]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[22]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[23]  Qiaozhu Mei,et al.  PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks , 2015, KDD.

[24]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[25]  Andrew McCallum,et al.  Collective multi-label classification , 2005, CIKM '05.

[26]  Andrew McCallum,et al.  Compositional Vector Space Models for Knowledge Base Completion , 2015, ACL.

[27]  Philip S. Yu,et al.  Learning Entity Types from Query Logs via Graph-Based Modeling , 2015, CIKM.

[28]  Jun Zhao,et al.  Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks , 2015, EMNLP.

[29]  Michael Gamon,et al.  Representing Text for Joint Embedding of Text and Knowledge Bases , 2015, EMNLP.

[30]  Philip S. Yu,et al.  NCR: A Scalable Network-Based Approach to Co-Ranking in Question-and-Answer Sites , 2014, CIKM.

[31]  Kai-Wei Chang,et al.  Typed Tensor Decomposition of Knowledge Bases for Relation Extraction , 2014, EMNLP.

[32]  Tom M. Mitchell,et al.  Improving Learning and Inference in a Large Knowledge-Base using Latent Syntactic Cues , 2013, EMNLP.

[33]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[34]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[35]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[36]  Tom M. Mitchell,et al.  Efficient and Expressive Knowledge Base Completion Using Subgraph Feature Extraction , 2015, EMNLP.

[37]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[38]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.