A Dynamic Oracle for Arc-Eager Dependency Parsing

The standard training regime for transition-based dependency parsers makes use of an oracle, which predicts an optimal transition sequence for a sentence and its gold tree. We present an improved oracle for the arc-eager transition system, which provides a set of optimal transitions for every valid parser configuration, including configurations from which the gold tree is not reachable. In such cases, the oracle provides transitions that will lead to the best reachable tree from the given configuration. The oracle is efficient to implement and provably correct. We use the oracle to train a deterministic left-to-right dependency parser that is less sensitive to error propagation, using an online training procedure that also explores parser configurations resulting from non-optimal sequences of transitions. This new parser outperforms greedy parsers trained using conventional oracles on a range of data sets, with an average improvement of over 1.2 LAS points and up to almost 3 LAS points on some data sets.

[1]  H. Kucera,et al.  Computational analysis of present-day American English , 1967 .

[2]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[3]  Andrew McCallum,et al.  Maximum Entropy Markov Models for Information Extraction and Segmentation , 2000, ICML.

[4]  Joakim Nivre,et al.  An Efficient Algorithm for Projective Dependency Parsing , 2003, IWPT.

[5]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[6]  Díaz de Ilarraza.  Construction of a Basque Dependency Treebank , 2003 .

[7]  Dilek Z. Hakkani-Tür,et al.  Building a Turkish Treebank , 2003 .

[8]  Roberto Basili,et al.  Building the Italian Syntactic-Semantic Treebank , 2003 .

[9]  Jan Hajic,et al.  The Prague Dependency Treebank , 2003 .

[10]  Yuji Matsumoto,et al.  Statistical Dependency Analysis with Support Vector Machines , 2003, IWPT.

[11]  Joakim Nivre,et al.  Incrementality in Deterministic Dependency Parsing , 2004 .

[12]  Jan Hajic,et al.  Prague Arabic Dependency Treebank: Development in Data and Tools , 2004 .

[13]  Michael A. Covington,et al.  A Fundamental Algorithm for Dependency Parsing , 2004 .

[14]  Joakim Nivre,et al.  Memory-Based Dependency Parsing , 2004, CoNLL.

[15]  C. Luo,et al.  Sinica Treebank: Design Criteria, Representational Issues and Implementation , 2004 .

[16]  William W. Cohen,et al.  Stacked Sequential Learning , 2005, IJCAI.

[17]  Daniel Marcu,et al.  Learning as search optimization: approximate large margin methods for structured prediction , 2005, ICML.

[18]  Joakim Nivre,et al.  Pseudo-Projective Dependency Parsing , 2005, ACL.

[19]  János Csirik,et al.  The Szeged Treebank , 2005, TSD.

[20]  Stelios Piperidis,et al.  Theoretical and Practical Issues in the Construction of a Greek Dependency Treebank , 2005 .

[21]  Giuseppe Attardi,et al.  Experiments with a Multilanguage Non-Projective Dependency Parser , 2006, CoNLL.

[22]  Christopher D. Manning,et al.  Generating Typed Dependency Parses from Phrase Structure Parses , 2006, LREC.

[23]  Josef van Genabith,et al.  QuestionBank: Creating a Corpus of Parse-Annotated Questions , 2006, ACL.

[24]  Joakim Nivre,et al.  Inductive Dependency Parsing , 2006, Text, speech and language technology.

[25]  Sebastian Riedel,et al.  The CoNLL 2007 Shared Task on Dependency Parsing , 2007, EMNLP.

[26]  Lluís Màrquez i Villodre,et al.  Anotación semiautomática con papeles temáticos de los corpus CESS-ECE , 2007, Proces. del Leng. Natural.

[27]  Joakim Nivre,et al.  Single Malt or Blended? A Study in Multilingual Parser Optimization , 2007, EMNLP.

[28]  Joakim Nivre,et al.  Characterizing the Errors of Data-Driven Dependency Parsing Models , 2007, EMNLP.

[29]  Joakim Nivre,et al.  Algorithms for Deterministic Incremental Dependency Parsing , 2008, CL.

[30]  Josef van Genabith,et al.  Parser Evaluation and the BNC: Evaluating 4 constituency parsers with 3 metrics , 2008, LREC.

[31]  Eric P. Xing,et al.  Stacking Dependency Parsers , 2008, EMNLP.

[32]  Stephen Clark,et al.  A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing , 2008, EMNLP.

[33]  Joakim Nivre,et al.  Integrating Graph-Based and Transition-Based Dependency Parsers , 2008, ACL.

[34]  John Langford,et al.  Search-based structured prediction , 2009, Machine Learning.

[35]  Yoav Goldberg,et al.  An Efficient Algorithm for Easy-First Non-Directional Dependency Parsing , 2010, NAACL.

[36]  J. Andrew Bagnell,et al.  Efficient Reductions for Imitation Learning , 2010, AISTATS.

[37]  Martha Palmer,et al.  Getting the Most out of Transition-based Dependency Parsing , 2011, ACL.

[38]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[39]  Joakim Nivre,et al.  Transition-based Dependency Parsing with Rich Non-local Features , 2011, ACL.

[40]  Josef van Genabith,et al.  Comparing the Use of Edited and Unedited Text in Parser Self-Training , 2011, IWPT.

[41]  Giorgio Satta,et al.  Elimination of Spurious Ambiguity in Transition-Based Dependency Parsing , 2012, ArXiv.

[42]  Yang Guo,et al.  Structured Perceptron with Inexact Search , 2012, NAACL.

[43]  Slav Petrov,et al.  Overview of the 2012 Shared Task on Parsing the Web , 2012 .

[44]  Jonas Kuhn,et al.  The Best of Both Worlds – A Graph-based Completion Model for Transition-based Parsers , 2012, EACL.