Fast Exact Inference with a Factored Model for Natural Language Parsing

We present a novel generative model for natural language tree structures in which semantic (lexical dependency) and syntactic (PCFG) structures are scored with separate models. This factorization provides conceptual simplicity, straightforward opportunities for separately improving the component models, and a level of performance comparable to similar, non-factored models. Most importantly, unlike other modern parsing models, the factored model admits an extremely effective A* parsing algorithm, which enables efficient, exact inference.

[1]  J. Baker.  Trainable grammars for speech recognition , 1979 .

[2]  Igor Mel’čuk,et al.  Dependency Syntax: Theory and Practice , 1987 .

[3]  Mats Rooth,et al.  Structural Ambiguity and Lexical Relations , 1991, ACL.

[4]  John Lafferty,et al.  Grammatical Trigrams: A Probabilistic Model of Link Grammar , 1992 .

[5]  Structural Ambiguity and Lexical Relations , 1990, CL.

[6]  Michael Collins,et al.  Prepositional Phrase Attachment through a Backed-off Model , 1995, VLC@ACL.

[7]  David M. Magerman.  Statistical Decision-Tree Models for Parsing , 1995, ACL.

[8]  Michael Collins,et al.  A New Statistical Parser Based on Bigram Lexical Dependencies , 1996, ACL.

[9]  Eugene Charniak,et al.  Tree-Bank Grammars , 1996, AAAI/IAAI, Vol. 2.

[10]  Joshua Goodman,et al.  Parsing Algorithms and Metrics , 1996, ACL.

[11]  Mark Johnson,et al.  PCFG Models of Linguistic Tree Representations , 1998, CL.

[12]  Eugene Charniak,et al.  Edge-Based Best-First Chart Parsing , 1998, VLC@COLING/ACL.

[13]  Giorgio Satta,et al.  Efficient Parsing for Bilexical Context-Free Grammars and Head Automaton Grammars , 1999, ACL.

[14]  Eugene Charniak,et al.  A Maximum-Entropy-Inspired Parser , 2000, ANLP.

[15]  Dan Klein,et al.  Parsing with Treebank Grammars: Empirical Bounds, Theoretical Models, and the Structure of the Penn Treebank , 2001, ACL.

[16]  Dan Klein,et al.  Parsing and Hypergraphs , 2001, IWPT.

[17]  Rens Bod.  What is the Minimal Set of Fragments that Achieves Maximal Parse Accuracy? , 2001, ACL.

[18]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[19]  Michael Collins,et al.  Head-Driven Statistical Models for Natural Language Parsing , 2003, CL.