Semi-supervised Learning of Dependency Parsers using Generalized Expectation Criteria

In this paper, we propose a novel method for semi-supervised learning of non-projective log-linear dependency parsers using directly expressed linguistic prior knowledge (e.g. a noun's parent is often a verb). Model parameters are estimated using a generalized expectation (GE) objective function that penalizes the mismatch between model predictions and linguistic expectation constraints. In a comparison with two prominent "unsupervised" learning methods that require indirect biasing toward the correct syntactic structure, we show that GE can attain better accuracy with as few as 20 intuitive constraints. We also present positive experimental results on longer sentences in multiple languages.

[1]  Yoav Seginer,et al.  Fast Unsupervised Incremental Parsing , 2007, ACL.

[2]  Noah A. Smith,et al.  Annealing Structural Bias in Multilingual Weighted Grammar Induction , 2006, ACL.

[3]  Rohit J. Kate,et al.  Semi-Supervised Learning for Semantic Parsing using Support Vector Machines , 2007, NAACL.

[4]  David A. Smith,et al.  Bootstrapping Feature-Rich Dependency Parsers with Entropic Priors , 2007, EMNLP-CoNLL.

[5]  John D. Lafferty,et al.  Development and Evaluation of a Broad-Coverage Probabilistic Grammar of English-Language Computer Manuals , 1992, ACL.

[6]  Gideon S. Mann,et al.  Learning from labeled features using generalized expectation criteria , 2008, SIGIR '08.

[7]  Xavier Carreras,et al.  Simple Semi-supervised Dependency Parsing , 2008, ACL.

[8]  Jan Svartvik,et al.  A __ comprehensive grammar of the English language , 1988 .

[9]  Dan Klein,et al.  Prototype-Driven Grammar Induction , 2006, ACL.

[10]  Koby Crammer,et al.  Online Large-Margin Training of Dependency Parsers , 2005, ACL.

[11]  Gideon S. Mann,et al.  Generalized Expectation Criteria for Semi-Supervised Learning of Conditional Random Fields , 2008, ACL.

[12]  Ben Taskar,et al.  Dependency Grammar Induction via Bitext Projection Constraints , 2009, ACL/IJCNLP.

[13]  Noah A. Smith,et al.  Contrastive Estimation: Training Log-Linear Models on Unlabeled Data , 2005, ACL.

[14]  Noah A. Smith,et al.  Probabilistic Models of Nonprojective Dependency Trees , 2007, EMNLP.

[15]  Dale Schuurmans,et al.  Semi-Supervised Convex Training for Dependency Parsing , 2008, ACL.

[16]  Gideon S. Mann,et al.  Simple, robust, scalable semi-supervised learning via expectation regularization , 2007, ICML '07.

[17]  Eugene Charniak,et al.  Effective Self-Training for Parsing , 2006, NAACL.

[18]  Gert Smolka,et al.  A Relational Syntax-Semantics Interface Based on Dependency Grammar , 2004, COLING.

[19]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[20]  David A. Smith,et al.  Dependency Parsing by Belief Propagation , 2008, EMNLP.

[21]  Giorgio Satta,et al.  On the Complexity of Non-Projective Data-Driven Dependency Parsing , 2007, IWPT.

[22]  Eugene Charniak,et al.  Immediate-Head Parsing for Language Models , 2001, ACL.

[23]  Noah A. Smith,et al.  Parsing with Soft and Hard Constraints on Dependency Length , 2005 .

[24]  Noah A. Smith,et al.  Novel estimation methods for unsupervised discovery of latent structure in natural language text , 2007 .

[25]  Dan Klein,et al.  Corpus-Based Induction of Syntactic Structure: Models of Dependency and Constituency , 2004, ACL.

[26]  Rens Bod,et al.  An All-Subtrees Approach to Unsupervised Parsing , 2006, ACL.