Probing a Semantic Dependency Parser for Translational Relation Embeddings

Translational relation models are primarily applied to the task of Knowledge Graph embedding. We present a structural probe for testing whether a state-of-the-art semantic dependency parser learns contextualized word representations which fit a translational relation model. We find that the parser does not explicitly learn a translational relation model. We do, however, find that a simple transformation of the word representations is enough to induce a TransE model with 73.45% label recall, indicating that translational relation models are at least implicitly learned by the parser. We believe that our findings can in the future be used to develop new Natural Language Understanding systems that are more useful for Knowledge Graph generation and completion.

[1]  Adam Tauman Kalai,et al.  Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings , 2016, NIPS.

[2]  Daniel Gildea,et al.  Automatic Labeling of Semantic Roles , 2000, ACL.

[3]  Stephan Oepen,et al.  Discriminant-Based MRS Banking , 2006, LREC.

[4]  Stephan Oepen,et al.  SemEval 2014 Task 8: Broad-Coverage Semantic Dependency Parsing , 2014, *SEMEVAL.

[5]  Zhen Wang,et al.  Knowledge Graph and Text Jointly Embedding , 2014, EMNLP.

[6]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[7]  Yoshua Bengio,et al.  Understanding intermediate layers using linear classifier probes , 2016, ICLR.

[8]  Han Xiao,et al.  From One Point to a Manifold: Knowledge Graph Embedding for Precise Link Prediction , 2015, IJCAI.

[9]  Yusuke Miyao,et al.  SemEval 2015 Task 18: Broad-Coverage Semantic Dependency Parsing , 2015, *SEMEVAL.

[10]  Christopher D. Manning,et al.  A Structural Probe for Finding Syntax in Word Representations , 2019, NAACL.

[11]  Goran Nenadic,et al.  Deep learning meets ontologies: experiments to anchor the cardiovascular disease ontology in the biomedical literature , 2018, Journal of Biomedical Semantics.

[12]  Diego Reforgiato Recupero,et al.  Semantic Web Machine Reading with FRED , 2017, Semantic Web.

[13]  Stephan Oepen,et al.  Broad-Coverage Semantic Dependency Parsing , 2014 .

[14]  Marco Kuhlmann,et al.  Improving Semantic Dependency Parsing with Syntactic Features , 2019 .

[15]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[16]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[17]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.