Second-Order Semantic Dependency Parsing with End-to-End Neural Networks

Semantic dependency parsing aims to identify semantic relationships between words in a sentence that form a graph. In this paper, we propose a second-order semantic dependency parser, which takes into consideration not only individual dependency edges but also interactions between pairs of edges. We show that second-order parsing can be approximated using mean field (MF) variational inference or loopy belief propagation (LBP). We can unfold both algorithms as recurrent layers of a neural network and therefore can train the parser in an end-to-end manner. Our experiments show that our approach achieves state-of-the-art performance.

[1]  David A. Smith,et al.  Dependency Parsing by Belief Propagation , 2008, EMNLP.

[2]  André F. T. Martins,et al.  Priberam: A Turbo Semantic Parser with Second Order Features , 2014, *SEMEVAL.

[3]  Mark Dredze,et al.  Approximation-Aware Dependency Parsing by Belief Propagation , 2015, TACL.

[4]  Koby Crammer,et al.  Online Large-Margin Training of Dependency Parsers , 2005, ACL.

[5]  Noah A. Smith,et al.  Dual Decomposition with Many Overlapping Components , 2011, EMNLP.

[6]  Sanjiv Kumar,et al.  On the Convergence of Adam and Beyond , 2018 .

[7]  Timothy Dozat,et al.  Simpler but More Accurate Semantic Dependency Parsing , 2018, ACL.

[8]  Xavier Carreras,et al.  Experiments with a Higher-Order Projective Dependency Parser , 2007, EMNLP.

[9]  Noah A. Smith,et al.  Deep Multitask Learning for Semantic Dependency Parsing , 2017, ACL.

[10]  Weiwei Sun,et al.  Quasi-Second-Order Parsing for 1-Endpoint-Crossing, Pagenumber-2 Graphs , 2017, EMNLP.

[11]  Noah A. Smith,et al.  Learning Joint Semantic Parsers from Disjoint Data , 2018, NAACL.

[12]  Stephan Oepen,et al.  Broad-Coverage Semantic Dependency Parsing , 2014 .

[13]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[14]  Hai Zhao,et al.  Fourth-Order Dependency Parsing , 2012, COLING.

[15]  Fernando Pereira,et al.  Online Learning of Approximate Dependency Parsing Algorithms , 2006, EACL.

[16]  Kewei Tu,et al.  Structured Attentions for Visual Question Answering , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Weiwei Sun,et al.  Peking: Building Semantic Dependency Graphs with a Hybrid Parser , 2015, SemEval@NAACL-HLT.

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Stephan Oepen,et al.  SemEval 2014 Task 8: Broad-Coverage Semantic Dependency Parsing , 2014, *SEMEVAL.

[20]  André F. T. Martins,et al.  Lisbon: Evaluating TurboSemanticParser on Multiple Languages and Out-of-Domain Data , 2015, *SEMEVAL.

[21]  Wanxiang Che,et al.  A Neural Transition-Based Approach for Semantic Dependency Graph Parsing , 2018, AAAI.

[22]  Yusuke Miyao,et al.  SemEval 2015 Task 18: Broad-Coverage Semantic Dependency Parsing , 2015, *SEMEVAL.

[23]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Timothy Dozat,et al.  Deep Biaffine Attention for Neural Dependency Parsing , 2016, ICLR.