Probabilistic Grammars and their Applications

Formal grammars are widely used in speech recognition, language translation, and language understanding systems. Grammars rich enough to accommodate natural language generate multiple interpretations of typical sentences. These ambiguities are a fundamental challenge to practical application. Grammars can be equipped with probability distributions, and the various parameters of these distributions can be estimated from data (e.g., acoustic representations of spoken words or a corpus of hand-parsed sentences). The resulting probabilistic grammars help to interpret spoken or written language unambiguously. This article reviews the main classes of probabilistic grammars and points to some active areas of research.

[1]  Christopher Culy,et al.  The complexity of the vocabulary of Bambara , 1985 .

[2]  Eugene Charniak,et al.  Tree-Bank Grammars , 1996, AAAI/IAAI, Vol. 2.

[3]  Mark Johnson,et al.  Estimators for Stochastic “Unification-Based” Grammars , 1999, ACL.

[4]  Geoffrey K. Pullum,et al.  Generalized Phrase Structure Grammar , 1985 .

[5]  Zhiyi Chi,et al.  Statistical Properties of Probabilistic Context-Free Grammars , 1999, CL.

[6]  Peter Norvig,et al.  Verbmobih A Translation System for Face-to-Face Dialog , 1994 .

[7]  Eugene Charniak,et al.  Statistical Parsing with a Context-Free Grammar and Word Statistics , 1997, AAAI/IAAI.

[8]  Stuart Gonan,et al.  Probability models for complex systems , 1998 .

[9]  Steve Young,et al.  Applications of stochastic context-free grammars using the Inside-Outside algorithm , 1990 .

[10]  Ben Taskar,et al.  Introduction to statistical relational learning , 2007 .

[11]  Stuart M. Shieber,et al.  Evidence against the context-freeness of natural language , 1985 .

[12]  T. E. Harris,et al.  The Theory of Branching Processes. , 1963 .

[13]  Michael Collins,et al.  A New Statistical Parser Based on Bigram Lexical Dependencies , 1996, ACL.

[14]  M. Baltin,et al.  The Mental representation of grammatical relations , 1985 .

[15]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[16]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[17]  Ivan A. Sag,et al.  Information-based syntax and semantics , 1987 .

[18]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[19]  Noah A. Smith,et al.  Covariance in Unsupervised Learning of Probabilistic Grammars , 2010, J. Mach. Learn. Res..

[20]  J. Besag.  Statistical Analysis of Non-Lattice Data , 1975 .

[21]  Fernando Pereira,et al.  Relating Probabilistic Grammars and Automata , 1999, ACL.

[22]  N. Curteanu.  Book Reviews: Lecture on Contemporary Syntactic Theories: An Introduction to Unification-Based Approaches to Grammar , 1987, CL.

[23]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[24]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[25]  Steven P. Abney.  Stochastic Attribute-Value Grammars , 1996, CL.

[26]  Ronald M. Kaplan,et al.  Lexical Functional Grammar A Formal System for Grammatical Representation , 2004 .

[27]  King-Sun Fu,et al.  Syntactic Methods in Pattern Recognition , 1974, IEEE Transactions on Systems, Man, and Cybernetics.

[28]  Robert P. Goldman,et al.  A probabilistic plan recognition algorithm based on plan tree grammars , 2009, Artif. Intell..