Exploiting Multi-Word Units in History-Based Probabilistic Generation

We present a simple history-based model for sentence generation from LFG f-structures, which improves on the accuracy of previous models by breaking down PCFG independence assumptions so that more f-structure conditioning context is used in the prediction of grammar rule expansions. In addition, we present work on experiments with named entities and other multi-word units, showing a statistically significant improvement of generation accuracy. Tested on section 23 of the Penn Wall Street Journal Treebank, the techniques described in this paper improve BLEU scores from 66.52 to 68.82, and coverage from 98.18% to 99.96%.

[1]  John D. Lafferty,et al.  Towards History-based Grammars: Using Richer Models for Probabilistic Parsing , 1993, ACL.

[2]  Jun'ichi Tsujii,et al.  Probabilistic Models for Disambiguation of an HPSG-Based Chart Generator , 2005, IWPT.

[3]  Ronald M. Kaplan,et al.  Low-Level Mark-Up and Large-scale LFG Grammar Processing , 2003 .

[4]  Mark Johnson,et al.  PCFG Models of Linguistic Tree Representations , 1998, CL.

[5]  Irene Langkilde.  Forest-Based Statistical Sentence Generation , 2000, ANLP.

[6]  Eugene Charniak,et al.  A Maximum-Entropy-Inspired Parser , 2000, ANLP.

[7]  Adwait Ratnaparkhi,et al.  Trainable Methods for Surface Natural Language Generation , 2000, ANLP.

[8]  Stefan Riezler,et al.  Grammatical Machine Translation , 2006, NAACL.

[9]  Josef van Genabith,et al.  Robust PCFG-Based Generation Using Automatically Acquired LFG Approximations , 2006, ACL.

[10]  Joakim Nivre,et al.  Multiword Units in Syntactic Parsing , 2004 .

[11]  Kevin Humphreys,et al.  Reusing a Statistical Language Model for Generation , 2001, EWNLG@ACL.

[12]  Stephan Oepen,et al.  High Efficiency Realization for a Wide-Coverage Unification Grammar , 2005, IJCNLP.

[13]  Ronald M. Kaplan,et al.  The Formal Architecture of Lexical-Functional Grammar , 1989, J. Inf. Sci. Eng..

[14]  Stephan Oepen,et al.  Maximum Entropy Models for Realization Ranking , 2005 .

[15]  Srinivas Bangalore,et al.  Exploiting a Probabilistic Hierarchical Model for Generation , 2000, COLING.

[16]  Kevin Knight,et al.  Generation that Exploits Corpus-Based Statistical Knowledge , 1998, ACL.

[17]  Irene Langkilde-Geary,et al.  An Empirical Verification of Coverage and Correctness for a General-Purpose Sentence Generator , 2002, INLG.

[18]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[19]  Michael Collins,et al.  Head-Driven Statistical Models for Natural Language Parsing , 2003, CL.

[20]  Andy Way,et al.  Long-Distance Dependency Resolution in Automatically Acquired Wide-Coverage PCFG-Based LFG Approximations , 2004, ACL.

[21]  Stefan Riezler,et al.  Statistical Sentence Condensation using Ambiguity Packing and Stochastic Disambiguation Methods for Lexical-Functional Grammar , 2003, NAACL.

[22]  Anja Belz.  Probabilistic Generation of Weather Forecast Texts , 2007, HLT-NAACL.

[23]  Martin Kay,et al.  Chart Generation , 1996, ACL.

[24]  Hwee Tou Ng,et al.  Named Entity Recognition with a Maximum Entropy Approach , 2003, CoNLL.