Learning to Search for Dependencies

We demonstrate that a dependency parser can be built using a credit assignment compiler which removes the burden of worrying about low-level machine learning details from the parser implementation. The result is a simple parser which robustly applies to many languages that provides similar statistical and computational performance with best-to-date transition-based parsing approaches, while avoiding various downsides including randomization, extra feature requirements, and custom learning algorithms.

[1]  Alan Fern,et al.  On learning linear ranking functions for beam search , 2007, ICML '07.

[2]  Joakim Nivre,et al.  Incrementality in Deterministic Dependency Parsing , 2004 .

[3]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[4]  John Langford,et al.  A Credit Assignment Compiler for Joint Prediction , 2014, NIPS.

[5]  Yang Guo,et al.  Structured Perceptron with Inexact Search , 2012, NAACL.

[6]  Thomas G. Dietterich,et al.  Prune-and-Score: Learning for Greedy Coreference Resolution , 2014, EMNLP.

[7]  Alan Fern,et al.  Output Space Search for Structured Prediction , 2012, ICML.

[8]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[9]  Thomas A. Henzinger,et al.  Probabilistic programming , 2014, FOSE.

[10]  He He,et al.  Dynamic Feature Selection for Dependency Parsing , 2013, EMNLP.

[11]  Giorgio Satta,et al.  Dynamic Programming Algorithms for Transition-Based Dependency Parsers , 2011, ACL.

[12]  John Langford,et al.  Efficient programmable learning to search , 2014, ArXiv.

[13]  John Langford,et al.  Search-based structured prediction , 2009, Machine Learning.

[14]  John Langford,et al.  Normalized Online Learning , 2013, UAI.

[15]  Kilian Q. Weinberger,et al.  Feature hashing for large scale multitask learning , 2009, ICML '09.

[16]  J. Andrew Bagnell,et al.  Reinforcement and Imitation Learning via Interactive No-Regret Learning , 2014, ArXiv.

[17]  Alan Fern,et al.  HC-Search: A Learning Framework for Search-based Structured Prediction , 2014, J. Artif. Intell. Res..

[18]  John Langford,et al.  Online Importance Weight Aware Updates , 2010, UAI.

[19]  Alan Fern,et al.  Discriminative Learning of Beam-Search Heuristics for Planning , 2007, IJCAI.

[20]  Joakim Nivre,et al.  Transition-based Dependency Parsing with Rich Non-local Features , 2011, ACL.

[21]  Brian Roark,et al.  Incremental Parsing with the Perceptron Algorithm , 2004, ACL.

[22]  John Langford,et al.  Learning to Search Better than Your Teacher , 2015, ICML.

[23]  Koby Crammer,et al.  Online Large-Margin Training of Dependency Parsers , 2005, ACL.

[24]  Matthew J. Streeter,et al.  Adaptive Bound Optimization for Online Convex Optimization , 2010, COLT 2010.

[25]  Xavier Carreras,et al.  Simple Semi-supervised Dependency Parsing , 2008, ACL.

[26]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[27]  Giorgio Satta,et al.  A Tabular Method for Dynamic Oracles in Transition-Based Parsing , 2014, TACL.

[28]  Daniel Marcu,et al.  Learning as search optimization: approximate large margin methods for structured prediction , 2005, ICML.

[29]  Robert E. Schapire,et al.  A Reduction from Apprenticeship Learning to Classification , 2010, NIPS.

[30]  Andrew McCallum,et al.  FACTORIE: Probabilistic Programming via Imperatively Defined Factor Graphs , 2009, NIPS.

[31]  Sabine Buchholz,et al.  CoNLL-X Shared Task on Multilingual Dependency Parsing , 2006, CoNLL.

[32]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[33]  Yuji Matsumoto,et al.  Statistical Dependency Analysis with Support Vector Machines , 2003, IWPT.

[34]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[35]  Joakim Nivre,et al.  An Efficient Algorithm for Projective Dependency Parsing , 2003, IWPT.

[36]  Yoav Goldberg,et al.  An Efficient Algorithm for Easy-First Non-Directional Dependency Parsing , 2010, NAACL.

[37]  Joakim Nivre,et al.  Training Deterministic Parsers with Non-Deterministic Oracles , 2013, TACL.

[38]  David M. Bradley,et al.  Boosting Structured Prediction for Imitation Learning , 2006, NIPS.