Tensors over Semirings for Latent-Variable Weighted Logic Programs

Semiring parsing is an elegant framework for describing parsers by using semiring weighted logic programs. In this paper we present a generalization of this concept: latent-variable semiring parsing. With our framework, any semiring weighted logic program can be latentified by transforming weights from scalar values of a semiring to rank-n arrays, or tensors, of semiring values, allowing the modelling of latent variables within the semiring parsing framework. Semiring is too strong a notion when dealing with tensors, and we have to resort to a weaker structure: a partial semiring. We prove that this generalization preserves all the desired properties of the original semiring framework while strictly increasing its expressiveness.

[1]  Joshua Goodman,et al.  Parsing Inside-Out , 1998, ArXiv.

[2]  Noah A. Smith,et al.  Dynamic Programming Algorithms as Products of Weighted Logic Programs , 2008, ICLP.

[3]  Joshua Goodman,et al.  Semiring Parsing , 1999, CL.

[4]  Mark-Jan Nederhof,et al.  Squibs and Discussions: Weighted Deductive Parsing and Knuth’s Algorithm , 2003, CL.

[5]  Klaas Sikkel,et al.  Parsing Schemata and Correctness of Parsing Algorithms , 1998, Theor. Comput. Sci..

[6]  Martha Edmay Steenstrup.  Sum-ordered partial semirings , 1985 .

[7]  Jun'ichi Tsujii,et al.  Probabilistic CFG with Latent Annotations , 2005, ACL.

[8]  Noah A. Smith,et al.  Compiling Comp Ling: Weighted Dynamic Programming and the Dyna Language , 2005, HLT.

[9]  W. A. Martin,et al.  Parsing , 1980, ACL.

[10]  Adam Lopez,et al.  Translation as Weighted Deduction , 2009, EACL.

[11]  Kilian Gebhardt,et al.  Generic refinement of expressive grammar formalisms with an application to discontinuous constituent parsing , 2018, COLING.

[12]  Dean Alderucci.  A SPECTRAL ALGORITHM FOR LEARNING HIDDEN MARKOV MODELS THAT HAVE SILENT STATES , 2015 .

[13]  Werner Kuich,et al.  Semirings and Formal Power Series: Their Relevance to Formal Languages and Automata , 1997, Handbook of Formal Languages.

[14]  Noah A. Smith,et al.  Cube Summing, Approximate Inference with Non-Local Features, and Dynamic Programming without Semirings , 2009, EACL.

[15]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[16]  Zhifei Li,et al.  First- and Second-Order Expectation Semirings with Applications to Minimum-Risk Training on Translation Forests , 2009, EMNLP.

[17]  Stuart M. Shieber,et al.  Principles and Implementation of Deductive Parsing , 1994, J. Log. Program..

[18]  Karl Stratos,et al.  Experiments with Spectral Learning of Latent-Variable PCFGs , 2013, HLT-NAACL.

[19]  Karl Stratos,et al.  Spectral learning of latent-variable PCFGs: algorithms and sample complexity , 2014, J. Mach. Learn. Res..

[20]  Jason Eisner,et al.  Parameter Estimation for Probabilistic Finite-State Transducers , 2002, ACL.

[21]  Pierre Boullier,et al.  Range Concatenation Grammars , 2000, IWPT.

[22]  Liva Ralaivola,et al.  Grammatical inference as a principal component analysis problem , 2009, ICML '09.

[23]  David H. D. Warren,et al.  Parsing as Deduction , 1983, ACL.