Comparing the Ambiguity Reduction Abilities of Probabilistic Context-Free Grammars

We present a measure for evaluating Probabilistic Context Free Grammars (PCFG) based on their ambiguity resolution capabilities.Probabilities in a PCFG can be seen as a filtering mechanism: For an ambiguous sentence, the trees bearing maximum probability aresingle out, while all others are discarded. The level of ambiguity is related to the size of the singled out set of trees. Under our measure, a grammar is better than other if the first one has reduced the level of ambiguity in a higher degree. The measure we present is computed over a finite sample set of sentence because, as we show, it can not be computed over the set of sentences accepted by the grammar.

[1]  Rens Bod,et al.  Beyond Grammar: An Experience-Based Theory of Language , 1998 .

[2]  Khalil Sima'an,et al.  Towards comparing parsers from different linguistic frameworks: An information theoretic approach , 2002 .

[3]  Jason Eisner,et al.  Three New Probabilistic Models for Dependency Parsing: An Exploration , 1996, COLING.

[4]  Ted Briscoe,et al.  Parser evaluation: a survey and a new proposal , 1998, LREC.

[5]  Klaus Wich.  Exponential ambiguity of context-free grammars , 1999, Developments in Language Theory.

[6]  Dekang Lin,et al.  A dependency-based method for evaluating broad-coverage parsers , 1995, Natural Language Engineering.

[7]  Suresh Manandhar,et al.  Translating Treebank Annotation for Evaluation , 2001, ACL 2001.

[8]  守屋 悦朗,et al.  J.E.Hopcroft, J.D. Ullman 著, "Introduction to Automata Theory, Languages, and Computation", Addison-Wesley, A5変形版, X+418, \6,670, 1979 , 1980 .

[9]  Michael Collins,et al.  Discriminative Reranking for Natural Language Parsing , 2000, CL.

[10]  Michael Collins,et al.  Parsing with a Single Neuron: Convolution Kernels for Natural Language Problems , 2001 .

[11]  Ann Bies,et al.  The Penn Treebank: Annotating Predicate Argument Structure , 1994, HLT.

[12]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[13]  Klaus Wich.  Characterization of Context-Free Languages with Polynomially Bounded Ambiguity , 2001, MFCS.

[14]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[15]  Rens Bod.  An efficient implementation of a new DOP model , 2003, EACL.

[16]  Shigeru Masuyama,et al.  On the Ambiguity Reduction Ability of a Probabilistic Context-Free Grammar(Special Section on Discrete Mathematics and Its Applications) , 1998 .

[17]  Michael Collins,et al.  Review of Beyond grammar: an experience-based theory of language by Rens Bod. CSLI Publications 1998. , 1999 .

[18]  Michael Collins,et al.  Three Generative, Lexicalised Models for Statistical Parsing , 1997, ACL.