Computation of distances for regular and context-free probabilistic languages

Several mathematical distances between probabilistic languages have been investigated in the literature, motivated by applications in language modeling, computational biology, syntactic pattern matching and machine learning. In most cases, only pairs of probabilistic regular languages were considered. In this paper we extend previous results to pairs of languages generated by a probabilistic context-free grammar and a probabilistic finite automaton.

[1]  Mehryar Mohri,et al.  Efficient Computation of the Relative Entropy of Probabilistic Automata , 2006, LATIN.

[2]  Eugene S. Santos.  Probabilistic Grammars and Automata , 1972, Inf. Control..

[3]  Christos H. Papadimitriou,et al.  Elements of the Theory of Computation , 1997, SIGA.

[4]  John D. Lafferty,et al.  Computation of the Probability of Initial Substring Generation by Stochastic Context-Free Grammars , 1991, Comput. Linguistics.

[5]  Ulf Grenander,et al.  Lectures in pattern theory , 1978 .

[6]  Giorgio Satta,et al.  Probabilistic Parsing as Intersection , 2003, IWPT.

[7]  Christian N. S. Pedersen,et al.  The consensus string problem and the complexity of comparing hidden Markov models , 2002, J. Comput. Syst. Sci..

[8]  C. Kelley.  Iterative Methods for Linear and Nonlinear Equations , 1987 .

[9]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[10]  Jorge Calera-Rubio,et al.  Computing the Relative Entropy Between Regular Tree Languages , 1998, Inf. Process. Lett..

[11]  R. K. Shyamasundar,et al.  Introduction to algorithms , 1996 .

[12]  Mark-Jan Nederhof,et al.  A General Technique to Train Language Models on Language Models , 2005, Computational Linguistics.

[13]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[14]  Mehryar Mohri,et al.  On the Computation of Some Standard Distances Between Probabilistic Automata , 2006, CIAA.

[15]  Fernando Pereira,et al.  Relating Probabilistic Grammars and Automata , 1999, ACL.

[16]  Giorgio Satta,et al.  Computation of Probabilities for an Island-Driven Parser , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Kousha Etessami,et al.  Recursive Markov Chains, Stochastic Grammars, and Monotone Systems of Nonlinear Equations , 2005, STACS.

[18]  No Value,et al.  8th International Workshop on Parsing Technologies , 2003 .

[19]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[20]  Kousha Etessami,et al.  PReMo : An Analyzer for P robabilistic Re cursive Mo dels , 2007, TACAS.

[21]  T. E. Harris,et al.  The Theory of Branching Processes. , 1963 .

[22]  David G. Stork,et al.  Pattern Classification , 1973 .

[23]  Giorgio Satta,et al.  Optimal Probabilistic Evaluation Functions for Search Controlled by Stochastic Context-Free Grammars , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  M. W. Shields.  An Introduction to Automata Theory , 1988 .

[25]  Rafael C. Carrasco.  Accurate Computation of the Relative Entropy Between Stochastic Regular Grammars , 1997, RAIRO Theor. Informatics Appl..

[26]  Giorgio Satta,et al.  Computing Partition Functions of PCFGs , 2009, Research on Language and Computation.

[27]  Sean R. Eddy,et al.  Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids , 1998 .

[28]  Mariëlle Stoelinga,et al.  An Introduction to Probabilistic Automata , 2002, Bull. EATCS.

[29]  Professor Seppo Sippu,et al.  Parsing Theory , 1988, EATCS Monographs on Theoretical Computer Science.

[30]  Taylor L. Booth,et al.  Applying Probability Measures to Abstract Languages , 1973, IEEE Transactions on Computers.

[31]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[32]  Stephen Soule,et al.  Entropies of Probabilistic Grammars , 1974, Inf. Control..

[33]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[34]  Azaria Paz,et al.  Introduction to probabilistic automata (Computer science and applied mathematics) , 1971 .

[35]  Erich Kaltofen,et al.  Analysis of the binary complexity of asymptotically fast algorithms for linear system solving , 1988, SIGS.

[36]  Yehoshua Bar-Hillel,et al.  Language and information : selected essays on their theory and application , 1965 .

[37]  Francisco Casacuberta,et al.  Probabilistic finite-state machines - part I , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Zhiyi Chi,et al.  Statistical Properties of Probabilistic Context-Free Grammars , 1999, CL.

[39]  J. Baker.  Trainable grammars for speech recognition , 1979 .

[40]  Sandra E. Hutchins.  Moments of string and derivation lengths of stochastic context-free grammars , 1972, Inf. Sci..

[41]  Giorgio Satta,et al.  Kullback-Leibler Distance between Probabilistic Context-Free Grammars and Probabilistic Finite Automata , 2004, COLING.

[42]  Andreas Stolcke,et al.  An Efficient Probabilistic Context-Free Parsing Algorithm that Computes Prefix Probabilities , 1994, CL.

[43]  Mark-Jan Nederhof,et al.  Squibs and Discussions: Weighted Deductive Parsing and Knuth’s Algorithm , 2003, CL.

[44]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.