One-Class Order Embedding for Dependency Relation Prediction

Learning the dependency relations among entities and the hierarchy formed by these relations by mapping entities into some order embedding space can effectively enable several important applications, including knowledge base completion and prerequisite relations prediction. Nevertheless, it is very challenging to learn a good order embedding due to the existence of partial ordering and missing relations in the observed data. Moreover, most application scenarios do not provide non-trivial negative dependency relation instances. We therefore propose a framework that performs dependency relation prediction by exploring both rich semantic and hierarchical structure information in the data. In particular, we propose several negative sampling strategies based on graph-specific centrality properties, which supplement the positive dependency relations with appropriate negative samples to effectively learn order embeddings. This research not only addresses the needs of automatically recovering missing dependency relations, but also unravels dependencies among entities using several real-world datasets, such as course dependency hierarchy involving course prerequisite relations, job hierarchy in organizations, and paper citation hierarchy. Extensive experiments are conducted on both synthetic and real-world datasets to demonstrate the prediction accuracy as well as to gain insights using the learned order embedding.

[1]  Premkumar Natarajan,et al.  Modeling Concept Dependencies in a Scientific Corpus , 2016, ACL.

[2]  Zhaohui Wu,et al.  Recovering Concept Prerequisite Relations from University Course Dependencies , 2017, AAAI.

[3]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[4]  Chih-Jen Lin,et al.  Selection of Negative Samples for One-class Matrix Factorization , 2017, SDM.

[5]  Chengjiang Li,et al.  Course Concept Extraction in MOOCs via Embedding-Based Graph Propagation , 2017, IJCNLP.

[6]  Xiang Li,et al.  Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures , 2018, ACL.

[7]  Chengjiang Li,et al.  Prerequisite Relation Learning for Concepts in MOOCs , 2017, ACL.

[8]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[9]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[10]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[11]  Sanja Fidler,et al.  Order-Embeddings of Images and Language , 2015, ICLR.

[12]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[13]  Jin Tian,et al.  Joint Discovery of Skill Prerequisite Graphs and Student Models , 2016, EDM.

[14]  Andrew McCallum,et al.  Word Representations via Gaussian Embedding , 2014, ICLR.

[15]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[16]  C. Lee Giles,et al.  Investigating Active Learning for Concept Prerequisite Learning , 2018, AAAI.

[17]  Dragomir R. Radev,et al.  TutorialBank: A Manually-Collected Corpus for Prerequisite Chains, Survey Extraction and Resource Recommendation , 2018, ACL.

[18]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[19]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[20]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[21]  Evgeniy Gabrilovich,et al.  A Review of Relational Machine Learning for Knowledge Graphs , 2015, Proceedings of the IEEE.

[22]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[23]  Yiming Yang,et al.  Learning Concept Graphs from Online Educational Data , 2016, J. Artif. Intell. Res..

[24]  Jack Minker,et al.  On Indefinite Databases and the Closed World Assumption , 1987, CADE.

[25]  ChengXiang Zhai,et al.  Mining MOOC Lecture Transcripts to Construct Concept Dependency Graphs , 2018, EDM.

[26]  Wei Zhang,et al.  From Data Fusion to Knowledge Fusion , 2014, Proc. VLDB Endow..

[27]  Andrew Gordon Wilson,et al.  Hierarchical Density Order Embeddings , 2018, ICLR.

[28]  Ee-Peng Lim,et al.  Talent Flow Analytics in Online Professional Network , 2018, Data Science and Engineering.