Comparison Between the Inside-Outside Algorithm and the Viterbi Algorithm for Stochastic Context-Free Grammars

The most popular algorithms for the estimation of the probabilities of a context-free grammar are the Inside-Outside algorithm and the Viterbi algorithm, which are Maximum Likelihood approaches. The difference between the logarithm of the likelihood of a string and the logarithm of the likelihood of the most probable parse of a string is upper bounded linearly by the length of the string and the logarithm of the number of non-terminal symbols. However, this theoretical bound is too pessimistic. For this reason, an experimental work to show the behaviour of the two functions in practical cases is necessary.

[1]  Andreas Stolcke,et al.  Bayesian learning of probabilistic language models , 1994 .

[2]  Sartaj Sahni,et al.  Concepts in discrete mathematics , 1981 .

[3]  Roger K. Moore.  Computer Speech and Language , 1986 .

[4]  John D. Lafferty,et al.  Computation of the Probability of Initial Substring Generation by Stochastic Context-Free Grammars , 1991, Comput. Linguistics.

[5]  R. C. Underwood,et al.  THE APPLICATION OF STOCHASTIC CONTEXT-FREE GRAMMARS TO FOLDING, ALIGNING AND MODELING HOMOLOGOUS RNA SEQUENCES , 1993 .

[6]  Taylor L. Booth,et al.  Applying Probability Measures to Abstract Languages , 1973, IEEE Transactions on Computers.

[7]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[8]  Francisco Casacuberta.  Growth Transformations for Probability Functions of Stochastic Grammars , 1996, Int. J. Pattern Recognit. Artif. Intell..

[9]  Neri Merhav,et al.  Maximum likelihood hidden Markov modeling using a dominant sequence of states , 1991, IEEE Trans. Signal Process..

[10]  N. Merhav,et al.  Hidden Markov modeling using a dominant state sequence with application to speech recognition , 1991 .

[11]  J. Kupiec.  Hidden Markov estimation for unrestricted stochastic context-free grammars , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[12]  Steve Young,et al.  Applications of stochastic context-free grammars using the Inside-Outside algorithm , 1990 .

[13]  Yasubumi Sakakibara,et al.  Learning context-free grammars from structural data in polynomial time , 1988, COLT '88.

[14]  J. Baker.  Trainable grammars for speech recognition , 1979 .

[15]  Ellis Horowitz,et al.  Fundamentals of Data Structures in Pascal , 1984 .

[16]  Hermann Ney,et al.  Stochastic Grammars and Pattern Recognition , 1992 .

[17]  Andreas Stolcke,et al.  An Efficient Probabilistic Context-Free Parsing Algorithm that Computes Prefix Probabilities , 1994, CL.