CRF Autoencoder for Unsupervised Dependency Parsing

Unsupervised dependency parsing, which tries to discover linguistic dependency structures from unannotated data, is a very challenging task. Almost all previous work on this task focuses on learning generative models. In this paper, we develop an unsupervised dependency parsing model based on the CRF autoencoder. The encoder part of our model is discriminative and globally normalized which allows us to use rich features as well as universal linguistic priors. We propose an exact algorithm for parsing as well as a tractable learning algorithm. We evaluated the performance of our model on eight multilingual treebanks and found that our model achieved comparable performance with state-of-the-art approaches.

[1]  Kewei Tu,et al.  Unambiguity Regularization for Unsupervised Learning of Probabilistic Grammars , 2012, EMNLP.

[2]  Mark A. Paskin,et al.  Cubic-time Parsing and Learning Algorithms for Grammatical Bigram , 2001 .

[3]  Xavier Carreras,et al.  Structured Prediction Models via the Matrix-Tree Theorem , 2007, EMNLP.

[4]  Jason Eisner,et al.  Three New Probabilistic Models for Dependency Parsing: An Exploration , 1996, COLING.

[5]  Kewei Tu,et al.  Unsupervised Neural Dependency Parsing , 2016, EMNLP.

[6]  Valentin I. Spitkovsky,et al.  Viterbi Training Improves Unsupervised Dependency Parsing , 2010, CoNLL.

[7]  Phil Blunsom,et al.  Unsupervised Induction of Tree Substitution Grammars for Dependency Parsing , 2010, EMNLP.

[8]  Robert E. Tarjan,et al.  Finding optimum branchings , 1977, Networks.

[9]  Valentin I. Spitkovsky,et al.  Breaking Out of Local Optima with Count Transforms and Model Recombination: A Study in Grammar Induction , 2013, EMNLP.

[10]  Noah A. Smith,et al.  Conditional Random Field Autoencoders for Unsupervised Structured Prediction , 2014, NIPS.

[11]  Noah A. Smith,et al.  Logistic Normal Priors for Unsupervised Probabilistic Grammar Induction , 2008, NIPS.

[12]  Noémie Elhadad,et al.  A convex and feature-rich discriminative approach to dependency grammar induction , 2015, ACL.

[13]  Mark Johnson,et al.  Using Universal Linguistic Knowledge to Guide Grammar Induction , 2010, EMNLP.

[14]  Phil Blunsom,et al.  The PASCAL Challenge on Grammar Induction , 2012, HLT-NAACL 2012.

[15]  Koby Crammer,et al.  Online Large-Margin Training of Dependency Parsers , 2005, ACL.

[16]  Phong Le,et al.  Unsupervised Dependency Parsing: Let’s Use Supervised Parsers , 2015, NAACL.

[17]  John DeNero,et al.  Painless Unsupervised Learning with Features , 2010, NAACL.

[18]  Dan Klein,et al.  Corpus-Based Induction of Syntactic Structure: Models of Dependency and Constituency , 2004, ACL.