Probabilistic analogues of regular and context-free grammars are well known in computational linguistics, and currently the subject of intensive research. To date, however, no satisfactory probabilistic analogue of attribute-value grammars has been proposed: previous attempts have failed to define an adequate parameter-estimation algorithm.In the present paper, I define stochastic attribute-value grammars and give an algorithm for computing the maximum-likelihood estimate of their parameters. The estimation algorithm is adapted from Della Pietra, Della Pietra, and Lafferty (1995). To estimate model parameters, it is necessary to compute the expectations of certain functions under random fields. In the application discussed by Della Pietra, Della Pietra, and Lafferty (representing English orthographic constraints), Gibbs sampling can be used to estimate the needed expectations. The fact that attribute-value grammars generate constrained languages makes Gibbs sampling inapplicable, but I show that sampling can be done using the more general Metropolis-Hastings algorithm.
[1]
P. Lance.
[Elementary principles].
,
1956,
La Revue du praticien.
[2]
Ulf Grenander,et al.
Parameter Estimation for Constrained Context-Free Language Models
,
1992,
HLT.
[3]
Chris Brew,et al.
Stochastic HPSG
,
1995,
EACL.
[4]
Gerhard Winkler,et al.
Image analysis, random fields and dynamic Monte Carlo methods: a mathematical introduction
,
1995,
Applications of mathematics.
[5]
Stefan Riezler,et al.
Quantitative Constraint Logic Programming for Weighted Grammar Applications
,
1996,
LACL.
[6]
Stefan Riezler,et al.
Probabilistic Constraint Logic Programming
,
1997,
ArXiv.
[7]
John D. Lafferty,et al.
Inducing Features of Random Fields
,
1995,
IEEE Trans. Pattern Anal. Mach. Intell..