Weighted DAG Automata for Semantic Graphs

Graphs have a variety of uses in natural language processing, particularly as representations of linguistic meaning. A deficit in this area of research is a formal framework for creating, combining, and using models involving graphs that parallels the frameworks of finite automata for strings and finite tree automata for trees. A possible starting point for such a framework is the formalism of directed acyclic graph (DAG) automata, defined by Kamimura and Slutzki and extended by Quernheim and Knight. In this article, we study the latter in depth, demonstrating several new results, including a practical recognition algorithm that can be used for inference and learning with models defined on DAG automata. We also propose an extension to graphs with unbounded node degree and show that our results carry over to the extended formalism.

[1]  Tom A. B. Snijders,et al.  Markov Chain Monte Carlo Estimation of Exponential Random Graph Models , 2002, J. Soc. Struct..

[2]  Grzegorz Rozenberg,et al.  Handbook of Graph Grammars and Computing by Graph Transformations, Volume 1: Foundations , 1997 .

[3]  Stephan Oepen,et al.  Broad-Coverage Semantic Dependency Parsing , 2014 .

[4]  Wolfgang Thomas.  On Logics, Tilings, and Automata , 1991, ICALP.

[5]  Klaus-Jörn Lange,et al.  String grammars with disconnecting or a basic root of the difficulty in graph grammar parsing , 1987, Discret. Appl. Math..

[6]  Clemens Lautemann.  The complexity of graph languages generated by hyperedge replacement , 2004, Acta Informatica.

[7]  Mogens Nielsen,et al.  Decidability Issues for Petri Nets - a survey , 1994, Bull. EATCS.

[8]  Lutz Priese.  Finite Automata on Unranked and Unordered DAGs , 2007, Developments in Language Theory.

[9]  Andrzej Ehrenfeucht,et al.  On the membership problem for regular DNLC grammars , 1986, Discret. Appl. Math..

[10]  Max Dauchet,et al.  A Kleene Theorem for a Class of Planar Acyclic Graphs , 1995, Inf. Comput..

[11]  Bruno Courcelle,et al.  The Monadic Second-Order Logic of Graphs X: Linear Orderings , 1996, Theor. Comput. Sci..

[12]  Annegret Habel,et al.  Hyperedge Replacement: Grammars and Languages , 1992, Lecture Notes in Computer Science.

[13]  Mark Johnson,et al.  Estimators for Stochastic “Unification-Based” Grammars , 1999, ACL.

[14]  Robert L. Mercer,et al.  Word-Sense Disambiguation Using Statistical Methods , 1991, ACL.

[15]  David Chiang,et al.  Hope and Fear for Discriminative Training of Statistical Translation Models , 2012, J. Mach. Learn. Res..

[16]  Bruno Courcelle,et al.  The Monadic Second-Order Logic of Graphs. I. Recognizable Sets of Finite Graphs , 1990, Inf. Comput..

[17]  Raymond J. Mooney,et al.  Learning for Semantic Parsing , 2009, CICLing.

[18]  Steve Young,et al.  Applications of stochastic context-free grammars using the Inside-Outside algorithm , 1990 .

[19]  Jaime G. Carbonell,et al.  A Discriminative Graph-Based Parser for the Abstract Meaning Representation , 2014, ACL.

[20]  Zhiyi Chi,et al.  Statistical Properties of Probabilistic Context-Free Grammars , 1999, CL.

[21]  H. Vogler,et al.  Weighted Tree Automata and Tree Transducers , 2009 .

[22]  Hwee Tou Ng,et al.  A Machine Learning Approach to Coreference Resolution of Noun Phrases , 2001, CL.

[23]  Prakash P. Shenoy,et al.  Probability propagation , 1990, Annals of Mathematics and Artificial Intelligence.

[24]  Steffen L. Lauritzen,et al.  Bayesian updating in causal probabilistic networks by local computations , 1990 .

[25]  Johannes Blum,et al.  Properties of Regular DAG Languages , 2016, LATA.

[26]  Jonathan May.  SemEval-2016 Task 8: Meaning Representation Parsing , 2016, SemEval@NAACL-HLT.

[27]  Annegret Habel,et al.  Hyperedge Replacement, Graph Grammars , 1997, Handbook of Graph Grammars.

[28]  Chuan Wang,et al.  A Transition-based Algorithm for AMR Parsing , 2015, NAACL.

[29]  Paul Gastin,et al.  The Kleene-Schützenberger Theorem for Formal Power Series in Partially Commuting Variables , 1999, Inf. Comput..

[30]  Jérôme Leroux,et al.  Structurally Cyclic Petri Nets , 2015, Log. Methods Comput. Sci..

[31]  Giora Slutzki,et al.  Parallel and Two-Way Automata on Directed Ordered Acyclic Graphs , 1981, Inf. Control..

[32]  Cyril Allauzen,et al.  A Unified Construction of the Glushkov, Follow, and Antimirov Automata , 2006, MFCS.

[33]  Sylvain Salvati,et al.  MIX is a 2-MCFL and the word problem in Z2 is captured by the IO and the OI hierarchies , 2015, J. Comput. Syst. Sci..

[34]  Vladimir Solmon,et al.  The estimation of stochastic context-free grammars using the Inside-Outside algorithm , 2003 .

[35]  Derek G. Corneil,et al.  Complexity of finding embeddings in a k -tree , 1987 .

[36]  James R. Lee,et al.  Improved approximation algorithms for minimum-weight vertex separators , 2005, STOC '05.

[37]  Dan Klein,et al.  Learning Accurate, Compact, and Interpretable Tree Annotation , 2006, ACL.

[38]  Akio Fujiyoshi,et al.  Recognition of directed acyclic graphs by spanning tree automata , 2010, Theor. Comput. Sci..

[39]  Joshua Goodman,et al.  Semiring Parsing , 1999, CL.

[40]  Heng Ji,et al.  Unsupervised Entity Linking with Abstract Meaning Representation , 2015, NAACL.

[41]  Daniel Jurafsky,et al.  Automatic Labeling of Semantic Roles , 2002, CL.

[42]  Shlomit S. Pinter,et al.  Finite Automata on Directed Graphs , 1992, J. Comput. Syst. Sci..

[43]  Ove Frank,et al.  http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained , 2007 .

[44]  Ralph Grishman,et al.  Improving Event Detection with Abstract Meaning Representation , 2015 .

[45]  Max Dauchet,et al.  Automata and Rational Expressions on Planar Graphs , 1988, MFCS.

[46]  Jan Hajic,et al.  The Prague Dependency Treebank , 2003 .

[47]  Noah A. Smith,et al.  Weighted and Probabilistic Context-Free Grammars Are Equally Expressive , 2007, CL.

[48]  Mark Hopkins,et al.  The Power of Extended Top-Down Tree Transducers , 2009, SIAM J. Comput..

[49]  Stephan Oepen,et al.  Discriminant-Based MRS Banking , 2006, LREC.

[50]  D. Rose.  Triangulated graphs and the elimination process , 1970 .

[51]  Annegret Habel,et al.  May we introduce to you: hyperedge replacement , 1986, Graph-Grammars and Their Application to Computer Science.

[52]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[53]  Mitchell P. Marcus,et al.  OntoNotes: The 90% Solution , 2006, NAACL.

[54]  David J. Weir,et al.  Characterizing Structural Descriptions Produced by Various Grammatical Formalisms , 1987, ACL.

[55]  Jaime G. Carbonell,et al.  Generation from Abstract Meaning Representation using Tree Transducers , 2016, NAACL.

[56]  Hubert Comon,et al.  Tree automata techniques and applications , 1997 .

[57]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[58]  Henrik Björklund,et al.  Between a Rock and a Hard Place - Uniform Parsing for Hyperedge Replacement DAG Grammars , 2016, LATA.

[59]  Fernando Pereira,et al.  Relating Probabilistic Grammars and Automata , 1999, ACL.

[60]  Robert McNaughton,et al.  Regular Expressions and State Graphs for Automata , 1960, IRE Trans. Electron. Comput..

[61]  N. Curteanu.  Book Reviews: Lecture on Contemporary Syntactic Theories: An Introduction to Unification-Based Approaches to Grammar , 1987, CL.

[62]  Manfred Droste,et al.  Weighted Automata and Logics on Graphs , 2015, MFCS.

[63]  Witold Charatonik,et al.  Automata on DAG representations of finite trees , 1999 .

[64]  Jason Eisner,et al.  Parameter Estimation for Probabilistic Finite-State Transducers , 2002, ACL.

[65]  Stephan Oepen,et al.  SemEval 2014 Task 8: Broad-Coverage Semantic Dependency Parsing , 2014, *SEMEVAL.

[66]  Wolfgang Thomas,et al.  Elements of an automata theory over partial orders , 1997, Partial Order Methods in Verification.

[67]  Erik P. de Vink,et al.  Bisimulation of Labelled State-to-Function Transition Systems Coalgebraically , 2015, Log. Methods Comput. Sci..

[68]  Eugene Charniak,et al.  Statistical Parsing with a Context-Free Grammar and Word Statistics , 1997, AAAI/IAAI.

[69]  Vibhav Gogate,et al.  A Complete Anytime Algorithm for Treewidth , 2004, UAI.

[70]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[71]  Philipp Koehn,et al.  Abstract Meaning Representation for Sembanking , 2013, LAW@ACL.

[72]  Ari Rappoport,et al.  Universal Conceptual Cognitive Annotation (UCCA) , 2013, ACL.

[73]  Xiaochang Peng,et al.  A Synchronous Hyperedge Replacement Grammar based approach for AMR parsing , 2015, CoNLL.

[74]  Sebastian Maneth,et al.  The complexity of tree automata and XPath on grammar-compressed trees , 2006, Theor. Comput. Sci..

[75]  Ronald M. Kaplan,et al.  Lexical Functional Grammar A Formal System for Grammatical Representation , 2004 .

[76]  Johannes Blum.  DAG Automata - Variants, Languages and Properties , 2015 .

[77]  Michael Collins,et al.  Three Generative, Lexicalised Models for Statistical Parsing , 1997, ACL.

[78]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[79]  Christoph Koch,et al.  Query evaluation on compressed trees , 2003, 18th Annual IEEE Symposium of Logic in Computer Science, 2003. Proceedings..

[80]  Wilf R. LaLonde,et al.  Regular right part grammars and their parsers , 1977, CACM.

[81]  Edward Ochmanski,et al.  Regular behaviour of concurrent systems , 1985, Bull. EATCS.

[82]  Chuan Wang,et al.  Boosting Transition-based AMR Parsing with Refined Actions and Auxiliary Analyzers , 2015, ACL.

[83]  Bruno Courcelle,et al.  Graph expressions and graph rewritings , 1987, Mathematical systems theory.

[84]  Adwait Ratnaparkhi,et al.  A Maximum Entropy Model for Part-Of-Speech Tagging , 1996, EMNLP.

[85]  Mitchell P. Marcus,et al.  Text Chunking using Transformation-Based Learning , 1995, VLC@ACL.

[86]  C. Reutenauer.  The Mathematics of Petri Nets , 1990 .

[87]  Wolfgang Thomas,et al.  NONDETERMINISM VERSUS DETERMINISM OF FINITE AUTOMATA OVER DIRECTED ACYCLIC GRAPHS , 1994 .

[88]  Michaël Rusinowitch,et al.  Closure properties and decision problems of dag automata , 2005, Inf. Process. Lett..

[89]  Kevin Knight,et al.  Towards Probabilistic Acceptors and Transducers for Feature Structures , 2012, SSST@ACL.

[90]  Ivan A. Sag,et al.  Book Reviews: Head-driven Phrase Structure Grammar and German in Head-driven Phrase-structure Grammar , 1996, CL.

[91]  Trevor Darrell,et al.  Conditional Random Fields for Object Recognition , 2004, NIPS.

[92]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[93]  Michal Pilipczuk,et al.  Solving Connectivity Problems Parameterized by Treewidth in Single Exponential Time , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[94]  Alfred V. Aho,et al.  The Theory of Parsing, Translation, and Compiling , 1972 .

[95]  Noah A. Smith,et al.  Toward Abstractive Summarization Using Semantic Representations , 2018, NAACL.

[96]  Jacob Andreas,et al.  Parsing Graphs with Hyperedge Replacement Grammars , 2013, ACL.

[97]  Sylvain Salvati,et al.  of Third Order , 2012 .

[98]  Michael Collins,et al.  Prepositional Phrase Attachment through a Backed-off Model , 1995, VLC@ACL.