Semantic Operations for Transfer-based Machine Translation

Semantic Operations for Transfer-based Machine Translation Michael Wayne Goodman Co-Chairs of the Supervisory Committee: Chair Emily M. Bender Department of Linguistics Chair Francis Bond Division of Linguistics and Multilingual Studies, Nanyang Technological University This dissertation describes a new approach to the automatic extraction of semantic mappings (transfer rules) for rule-based machine translation. This approach continues previous work in combining HPSG rule-based grammars, whose precise bidirectional implementation facilitates deep semantic analysis of sentences and the enumeration of grammatical realizations of semantic representations, and data-driven techniques of machine translation, whose automatic extraction of knowledge and statistical inference allow models to be quickly built from bitexts and to rank extracted patterns by their frequency. I define two new methods for bilingually aligning semantic fragments (or semantic subgraphs) and a heuristic strategy for aligning nodes between source and target subgraphs, which together allow me to design transfer systems that meet, and at times exceed, the translation coverage and quality of the prior state of the art with a significantly reduced dependence on idiosyncratic language-pair definitions (i.e., improved language independence). These improvements are made possible by a number of semantic operations, either designed or implemented by me and defined within this dissertation, that fully model the semantic representations and allow for inspection and transformation as graph operations. I apply my methods to the task of translating Japanese sentences into English—a typologically distant language pair.

[1]  Christopher Culy,et al.  The limits of n-gram translation evaluation metrics , 2003, MTSUMMIT.

[2]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[3]  Berthold Crysmann,et al.  Relative Clause Extraposition in German: An Efficient and Portable Implementation , 2005 .

[4]  Jason Eisner,et al.  Learning Non-Isomorphic Tree Mappings for Machine Translation , 2003, ACL.

[5]  Emily M. Bender,et al.  Using information structure to improve transfer-based MT , 2011, Proceedings of the International Conference on Head-Driven Phrase Structure Grammar.

[6]  M. Steedman,et al.  Combinatory Categorial Grammar , 2011 .

[7]  Francis Bond,et al.  Using Generation for Grammar Analysis and Error Detection , 2009, ACL/IJCNLP.

[8]  Stefan Thater,et al.  Computing Weakest Readings , 2010, ACL.

[9]  Ann Copestake Semantic Composition with (Robust) Minimal Recursion Semantics , 2007, ACL 2007.

[10]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[11]  Stephan Oepen,et al.  LOGON — A Norwegian MT Effort , 2004 .

[12]  Martha Palmer,et al.  Verbnet: a broad-coverage, comprehensive verb lexicon , 2005 .

[13]  Johan Bos,et al.  Predicate logic unplugged , 1996 .

[14]  Yejin Choi,et al.  Neural AMR: Sequence-to-Sequence Models for Parsing and Generation , 2017, ACL.

[15]  Philipp Koehn,et al.  Abstract Meaning Representation for Sembanking , 2013, LAW@ACL.

[16]  Christopher D. Manning,et al.  LinGO Redwoods A Rich and Dynamic Treebank for HPSG , 2002 .

[17]  Stephan Oepen,et al.  SEM-I rational MT : enriching deep grammars with a semantic interface for scalable machine translation. , 2005 .

[18]  Kevin Knight,et al.  Generation that Exploits Corpus-Based Statistical Knowledge , 1998, ACL.

[19]  Mario Vento,et al.  An Improved Algorithm for Matching Large Graphs , 2001 .

[20]  Nianwen Xue,et al.  Not an Interlingua, But Close: Comparison of English AMRs to Chinese and Czech , 2014, LREC.

[21]  Francis Bond,et al.  Parse Ranking with Semantic Dependencies and WordNet , 2014, GWC.

[22]  Andy Way A hybrid architecture for robust MT using LFG-DOP , 1999, J. Exp. Theor. Artif. Intell..

[23]  Robert T. Kasper,et al.  A Flexible Interface for Linking Applications to Penman’s Sentence Generator , 1989, HLT.

[24]  Justus J. Randolph Free-Marginal Multirater Kappa (multirater K[free]): An Alternative to Fleiss' Fixed-Marginal Multirater Kappa. , 2005 .

[25]  Francis Bond,et al.  An HPSG-based Shared-Grammar for the Chinese Languages: ZHONG [|] , 2015 .

[26]  Alex Lascarides,et al.  An Algebra for Semantic Construction in Constraint-based Grammars , 2001, ACL.

[27]  Stefan Müller,et al.  HPSG Analysis of German , 2000 .

[28]  Yvette Graham,et al.  Deep Syntax in Statistical Machine Translation , 2011 .

[29]  Dan Flickinger,et al.  Translation using Minimal Recursion Semantics , 1995, TMI.

[30]  François Yvon,et al.  Hierarchical Sub-sentential Alignment with Anymalign , 2012, EAMT.

[31]  Ulrich Schäfer,et al.  Integrating deep and shallow natural language processing components: representations and hybrid architectures , 2006 .

[32]  Ming Zhou,et al.  Sequence-to-Dependency Neural Machine Translation , 2017, ACL.

[33]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[34]  Johan Bos,et al.  The Groningen Meaning Bank , 2013, JSSP.

[35]  John Carroll,et al.  An Efficient Chart Generator for (Semi-)Lexicalist Grammars , 2001 .

[36]  Koenraad De Smedt,et al.  CONSTRUCTING A PARSED CORPUS WITH A LARGE LFG GRAMMAR , 2005 .

[37]  Satoshi Sato,et al.  Toward Memory-based Translation , 1990, COLING.

[38]  Richard Edwin Stearns,et al.  Syntax-Directed Transduction , 1966, JACM.

[39]  Taku Kudo,et al.  MeCab : Yet Another Part-of-Speech and Morphological Analyzer , 2005 .

[40]  Hans-Ulrich Krieger,et al.  TDL-A Type Description Language for Constraint-Based Grammars , 1994, COLING.

[41]  Stephan Oepen,et al.  Who Did What to Whom? A Contrastive Study of Syntacto-Semantic Dependencies , 2012, LAW@ACL.

[42]  Wolfgang Wahlster,et al.  Mobile Speech-to-Speech Translation of Spontaneous Dialogs: An Overview of the Final Verbmobil System , 2000 .

[43]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[44]  Stephan Oepen,et al.  Parser Evaluation Using Elementary Dependency Matching , 2011, IWPT.

[45]  Francis Bond,et al.  Extracting Transfer Rules for Multiword Expressions from Parallel Corpora , 2011, MWE@ACL.

[46]  Berthold Crysmann,et al.  Some Fine Points of Hybrid Natural Language Parsing , 2008, LREC.

[47]  Johan Bos,et al.  Compositional Semantics in Verbmobil , 1996, COLING.

[48]  Daniel Gildea,et al.  Loosely Tree-Based Alignment for Machine Translation , 2003, ACL.

[49]  Stephan Oepen,et al.  Efficiency in Unification-Based N-Best Parsing , 2007, Trends in Parsing Technology.

[50]  Alexander Franz,et al.  Large-scale machine translation: an interlingua approach , 1994, IEA/AIE '94.

[51]  Helge Dyvik Theuniversality of f-structure : discovery or stipulation ? The case of modals , 2014 .

[52]  Dan Flickinger,et al.  On building a more effcient grammar by exploiting types , 2000, Natural Language Engineering.

[53]  Stephan Oepen,et al.  SemEval 2014 Task 8: Broad-Coverage Semantic Dependency Parsing , 2014, *SEMEVAL.

[54]  Kevin Knight,et al.  A Syntax-based Statistical Translation Model , 2001, ACL.

[55]  Y. Tanaka,et al.  Compilation of a multilingual parallel corpus , 2001 .

[56]  Stephan Oepen,et al.  High Efficiency Realization for a Wide-Coverage Unification Grammar , 2005, IJCNLP.

[57]  Andy Way,et al.  Seeing the wood for the trees: data-oriented translation , 2003, MTSUMMIT.

[58]  Ann A. Copestake,et al.  Invited Talk: Slacker Semantics: Why Superficiality, Dependency and Avoidance of Commitment can be the Right Way to Go , 2009, EACL.

[59]  Jason Lee,et al.  Fully Character-Level Neural Machine Translation without Explicit Segmentation , 2016, TACL.

[60]  Eiichiro Sumita,et al.  Translating with Examples: A New Approach to Machine Translation , 2005 .

[61]  Dan Flickinger,et al.  Minimal Recursion Semantics: An Introduction , 2005 .

[62]  Uwe Reyle,et al.  Dealing with Ambiguities by Underspecification: Construction, Representation and Deduction , 1993, J. Semant..

[63]  David Chiang,et al.  Learning to Translate with Source and Target Syntax , 2010, ACL.

[64]  Yusuke Miyao,et al.  SemEval 2015 Task 18: Broad-Coverage Semantic Dependency Parsing , 2015, *SEMEVAL.

[65]  Uwe Reyle,et al.  37. Discourse Representation Theory , 2011 .

[66]  Hermann Ney,et al.  A Systematic Comparison of Various Statistical Alignment Models , 2003, CL.

[67]  Noah A. Smith,et al.  A Simple, Fast, and Effective Reparameterization of IBM Model 2 , 2013, NAACL.

[68]  Makoto Nagao,et al.  A framework of a mechanical translation between Japanese and English by analogy principle , 1984 .

[69]  Aric Hagberg,et al.  Exploring Network Structure, Dynamics, and Function using NetworkX , 2008 .

[70]  Francis Bond,et al.  An Implemented Description of Japanese: The Lexeed Dictionary and the Hinoki Treebank , 2006, ACL.

[71]  Jacob Andreas,et al.  Semantics-Based Machine Translation with Hyperedge Replacement Grammars , 2012, COLING.

[72]  Gregory Norman Carlson,et al.  Reference to kinds in English , 1977 .

[73]  Dan Flickinger,et al.  A New Well-Formedness Criterion for Semantics Debugging , 2005 .

[74]  Chris Quirk,et al.  Dependency Treelet Translation: Syntactically Informed Phrasal SMT , 2005, ACL.

[75]  Vladimir Eidelman,et al.  cdec: A Decoder, Alignment, and Learning Framework for Finite- State and Context-Free Translation Models , 2010, ACL.

[76]  Chris Quirk,et al.  Dependency treelet translation: the convergence of statistical and example-based machine-translation? , 2006, MTSUMMIT.

[77]  Anton Bryl,et al.  F-structure transfer-based statistical machine translation , 2009 .

[78]  Christoph Teichmann,et al.  Graph parsing with s-graph grammars , 2015, ACL.

[79]  Bonnie J. Dorr,et al.  Machine Translation Divergences: A Formal Description and Proposed Solution , 1994, CL.

[80]  Johan Bos,et al.  Open-Domain Semantic Parsing with Boxer , 2015, NODALIDA.

[81]  Bill Byrne,et al.  Hierarchical Statistical Semantic Realization for Minimal Recursion Semantics , 2015, IWCS.

[82]  Stephan Oepen,et al.  RDF Triple Stores and a Custom SPARQL Front-End for Indexing and Searching (Very) Large Semantic Networks , 2014, COLING.

[83]  Alexander M. Rush,et al.  OpenNMT: Open-Source Toolkit for Neural Machine Translation , 2017, ACL.

[84]  SAMEER S. PRADHAN,et al.  Ontonotes: a Unified Relational Semantic Representation , 2007, Int. J. Semantic Comput..

[85]  Stephan Oepen,et al.  Discriminant-Based MRS Banking , 2006, LREC.

[86]  Emily M. Bender Linguistic I Ssues in L Anguage Technology Lilt on Achieving and Evaluating Language-independence in Nlp on Achieving and Evaluating Language-independence in Nlp , 2022 .

[87]  Dekai Wu,et al.  Stochastic Inversion Transduction Grammars and Bilingual Parsing of Parallel Corpora , 1997, CL.

[88]  Matic Horvat,et al.  Hierarchical statistical semantic translation and realization , 2017 .

[89]  Emily M. Bender,et al.  Layers of Interpretation: On Grammar and Compositionality , 2015, IWCS.

[90]  Chenchen Ding,et al.  Dependency Tree Abstraction for Long-Distance Reordering in Statistical Machine Translation , 2014, EACL.

[91]  Phil Blunsom,et al.  Robust Incremental Neural Semantic Graph Parsing , 2017, ACL.

[92]  Ann Copestake,et al.  Implementing typed feature structure grammars , 2001, CSLI lecture notes series.

[93]  Miles Osborne,et al.  Statistical Machine Translation , 2010, Encyclopedia of Machine Learning and Data Mining.

[94]  Alon Lavie,et al.  METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments , 2007, WMT@ACL.

[95]  Emily M. Bender,et al.  Simple Negation Scope Resolution through Deep Parsing: A Semantic Solution to a Semantic Problem , 2014, ACL.

[96]  Mary Dalrymple,et al.  Lexical Functional Grammar , 2001 .

[97]  Berthold Crysmann,et al.  Reduplication in a computational HPSG of Hausa , 2017 .

[98]  Stephan Oepen,et al.  Stochastic HPSG Parse Disambiguation using the Redwoods Corpus , 2005 .

[99]  Ivan A. Sag,et al.  Book Reviews: Head-driven Phrase Structure Grammar and German in Head-driven Phrase-structure Grammar , 1996, CL.

[100]  Nathan Schneider,et al.  Association for Computational Linguistics: Human Language Technologies , 2011 .

[101]  Khalil Sima'an,et al.  A dop model for phrase structure trees , 2003 .

[102]  Pushpak Bhattacharyya,et al.  Some Issues in Automatic Evaluation of English-Hindi MT: More Blues for BLEU , 2006 .

[103]  Francis Bond,et al.  Building an HPSG-based Indonesian Resource Grammar (INDRA) , 2015 .

[104]  Oren Etzioni,et al.  Evaluating Lemmatic Communication , 2010 .

[105]  Shujian Huang,et al.  Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder , 2017, ACL.

[106]  Erik Velldal,et al.  Empirical Realization Ranking , 2009 .

[107]  David Chiang,et al.  Hierarchical Phrase-Based Translation , 2007, CL.

[108]  Yi Zhang,et al.  Mapping between Compositional Semantic Representations and Lexical Semantic Resources: Towards Accurate Deep Semantic Parsing , 2008, ACL.

[109]  A. Lascarides,et al.  Resolving Fragments using Discourse Information , 2002 .

[110]  Oren Etzioni,et al.  Lemmatic Machine Translation , 2009, MTSUMMIT.

[111]  Andreas Eisele,et al.  The DeepThought Core Architecture Framework , 2004, LREC.

[112]  Eric Nichols,et al.  Deep open-source machine translation , 2011, Machine Translation.

[113]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[114]  Stephan Oepen,et al.  Towards systematic grammar profiling.Test suite technology 10 years after , 1998, Comput. Speech Lang..

[115]  Eric K. Ringger,et al.  Statistical Machine Translation Using Labeled Semantic Dependency Graphs , 2004 .

[116]  Michael Wayne Goodman,et al.  Resources for building applications with Dependency Minimal Recursion Semantics , 2016, LREC.

[117]  Yuan Ding,et al.  Machine Translation Using Probabilistic Synchronous Dependency Insertion Grammars , 2005, ACL.

[118]  John Chandioux METEO : un système opérationnel pour la traduction automatique des bulletins météorologiques destinés au grand public , 1976 .

[119]  Yuji Matsumoto,et al.  Combining resources for open source machine translation , 2007, TMI.

[120]  Yoav Goldberg,et al.  Towards String-To-Tree Neural Machine Translation , 2017, ACL.

[121]  Ying Zhang,et al.  Interpreting BLEU/NIST Scores: How Much Improvement do We Need to Have a Better System? , 2004, LREC.

[122]  Francis Bond,et al.  Extracting Semantic Transfer Rules from Parallel Corpora with SMT Phrase Aligners , 2012, SSST@ACL.

[123]  Hitoshi Isahara,et al.  Enhancing the Japanese WordNet , 2009, ALR7@IJCNLP.

[124]  Joachim Niehren,et al.  Minimal Recursion Semantics as Dominance Constraints: Translation, Evaluation, and Analysis , 2004, ACL.

[125]  Jan Tore Lønning,et al.  Towards hybrid quality-oriented machine translation – on linguistics and probabilities in MT , 2007, TMI.

[126]  sanghoun Song,et al.  Information Structure of Relative Clauses in English: a Flexible and Computationally Tractable Model , 2014 .

[127]  George R. Doddington,et al.  Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics , 2002 .

[128]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[129]  Stephan Oepen,et al.  Open Source Machine Translation with DELPH-IN , 2005, MTSUMMIT.