Combination of Estimation Algorithms and Grammatical Inference Techniques to Learn Stochastic Context-Free Grammars

Some of the most widely-known methods to obtain Stochastic Context-Free Grammars (SCFGs) are based on estimation algorithms. All of these algorithms maximize a certain criterion function from a training sample by using gradient descendent techniques. In this optimization process, the obtaining of the initial SCFGs is an important factor, given that it affects the convergence process and the maximum which can be achieved. Here, we show experimentally how the results can be improved in cases when structural information about the task is inductively incorporated into the initial SCFGs. In this work, we present a stochastic version of the well-known Sakakibara algorithm in order to learn these initial SCFGs. Finally, an experimental study on part of the Wall Street Journal corpus was carried out.

[1]  Francisco Casacuberta,et al.  Comparison Between the Inside-Outside Algorithm and the Viterbi Algorithm for Stochastic Context-Free Grammars , 1996, SSPR.

[2]  Erkki Mäkinen On the Structural Grammatical Inference Problem for Some Classes of Context-Free Grammars , 1992, Inf. Process. Lett..

[3]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[4]  José-Miguel Benedí,et al.  Learning of stochastic context-free grammars by means of estimation algorithms , 1999, EUROSPEECH.

[5]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[6]  Jared J. Wolf Speech Recognition and Understanding , 1980 .

[7]  Ronald Rosenfeld,et al.  The CMU Statistical Language Modeling Toolkit and its use in the 1994 ARPA CSR Evaluation , 1995 .

[8]  Joan-Andreu Sánchez,et al.  Consistency of Stochastic Context-Free Grammars From Probabilistic Estimation Based on Growth Transformations , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Yasubumi Sakakibara,et al.  Efficient Learning of Context-Free Grammars from Positive Structural Examples , 1992, Inf. Comput..

[10]  Steve Young,et al.  Applications of stochastic context-free grammars using the Inside-Outside algorithm , 1990 .

[11]  Azriel Rosenfeld,et al.  Advances in Structural and Syntactical Pattern Recognition , 1996 .

[12]  John D. Lafferty,et al.  Computation of the Probability of Initial Substring Generation by Stochastic Context-Free Grammars , 1991, Comput. Linguistics.

[13]  Hermann Ney,et al.  Stochastic Grammars and Pattern Recognition , 1992 .

[14]  Joan-Andreu Sánchez,et al.  Estimation of the probability distributions of stochastic context-free grammars from the k-best derivations , 1998, ICSLP.

[15]  Stanley F. Chen,et al.  Bayesian Grammar Induction for Language Modeling , 1995, ACL.