Towards Optimal Choice Selection for Improved Hybrid Machine Translation

Towards Optimal Choice Selection for Improved Hybrid Machine Translation In recent years, machine translation (MT) research focused on investigating how hybrid MT as well as MT combination systems can be designed so that the resulting translations give an improvement over the individual translations. As a first step towards achieving this objective we have developed a parallel corpus with source data and the output of a number of MT systems, annotated with metadata information, capturing aspects of the translation process performed by the different MT systems. As a second step, we have organised a shared task in which participants were requested to build Hybrid/System Combination systems using the annotated corpus as input. The main focus of the shared task is trying to answer the following question: Can Hybrid MT algorithms or System Combination techniques benefit from the extra information (linguistically motivated, decoding and runtime) from the different systems involved? In this paper, we describe the annotated corpus we have created. We provide an overview on the participating systems from the shared task as well as a discussion of the results.

[1]  W. A. Scott,et al.  Reliability of Content Analysis ; The Case of Nominal Scale Cording , 1955 .

[2]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[3]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[4]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[5]  Richard S. Kayne,et al.  Syntaxe du français : le cycle transformationnel , 1977 .

[6]  G. Pullum,et al.  CLITICIZATION VS. INFLECTION: ENGLISH N'T , 1983 .

[7]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[8]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[9]  Philip H. Miller,et al.  Clitics and constituents in phrase structure grammar , 1992 .

[10]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[11]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[12]  Ivan A. Sag,et al.  Book Reviews: Head-driven Phrase Structure Grammar and German in Head-driven Phrase-structure Grammar , 1996, CL.

[13]  Jean Carletta,et al.  Assessing Agreement on Classification Tasks: The Kappa Statistic , 1996, CL.

[14]  A. F. Fehri.  Arabic Modifying Adjectives and DP Structures , 1999 .

[15]  I. Dan Melamed,et al.  Models of translation equivalence among words , 2000, CL.

[16]  Ivan A. Sag,et al.  Syntactic Theory: A Formal Introduction , 1999, Computational Linguistics.

[17]  Shuly Wintner,et al.  Definiteness in the Hebrew noun phrase , 2000, Journal of Linguistics.

[18]  Shankar Kumar,et al.  Minimum Bayes-Risk Word Alignments of Bilingual Texts , 2002, EMNLP.

[19]  Hermann Ney,et al.  Discriminative Training and Maximum Entropy Models for Statistical Machine Translation , 2002, ACL.

[20]  Andreas Stolcke,et al.  SRILM - an extensible language modeling toolkit , 2002, INTERSPEECH.

[21]  Eiichiro Sumita,et al.  Toward a Broad-coverage Bilingual Corpus for Speech Translation of Travel Conversations in the Real World , 2002, LREC.

[22]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[23]  Franz Josef Och,et al.  Minimum Error Rate Training in Statistical Machine Translation , 2003, ACL.

[24]  The Comprendium Translator system , 2003, MTSUMMIT.

[25]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[26]  Joost Merijn Kremers,et al.  The Arabic noun phrase : a minimalist approach , 2003 .

[27]  Chin-Yew Lin,et al.  ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation , 2004, COLING.

[28]  K. Krippendorff.  Reliability in Content Analysis: Some Common Misconceptions and Recommendations , 2004 .

[29]  Alon Lavie,et al.  METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments , 2005, IEEvaluation@ACL.

[30]  Philipp Koehn,et al.  Europarl: A Parallel Corpus for Statistical Machine Translation , 2005, MTSUMMIT.

[31]  David Chiang,et al.  A Hierarchical Phrase-Based Model for Statistical Machine Translation , 2005, ACL.

[32]  Yang Liu,et al.  Log-Linear Models for Word Alignment , 2005, ACL.

[33]  Robert C. Moore.  A Discriminative Framework for Bilingual Word Alignment , 2005, HLT.

[34]  Tomaz Erjavec,et al.  The JRC-Acquis: A Multilingual Aligned Parallel Corpus with 20+ Languages , 2006, LREC.

[35]  Hermann Ney,et al.  AER: do we need to “improve” our alignments? , 2006, IWSLT.

[36]  Mikel L. Forcada,et al.  Opentrad Apertium open-source machine translation system: an opportunity for business and research , 2006 .

[37]  Taro Watanabe,et al.  Left-to-Right Target Generation for Hierarchical Phrase-Based Translation , 2006, ACL.

[38]  Ben Taskar,et al.  Alignment by Agreement , 2006, NAACL.

[39]  David Chiang,et al.  Hierarchical Phrase-Based Translation , 2007, CL.

[40]  Miles Osborne,et al.  Randomised Language Modelling for Statistical Machine Translation , 2007, ACL.

[41]  John DeNero,et al.  Tailoring Word Alignments to Syntactic Machine Translation , 2007, ACL.

[42]  Philipp Koehn,et al.  Moses: Open Source Toolkit for Statistical Machine Translation , 2007, ACL.

[43]  Philipp Koehn,et al.  Factored Translation Models , 2007, EMNLP.

[44]  Dorothee Beermann,et al.  The Definite Article and Possessive Marking in Amharic , 2007 .

[45]  Rafael E. Banchs,et al.  Discriminative Alignment Training without Annotated Data for Machine Translation , 2007, HLT-NAACL.

[46]  David Chiang,et al.  Forest Rescoring: Faster Decoding with Integrated Language Models , 2007, ACL.

[47]  Philipp Koehn,et al.  Proceedings of the Third Workshop on Statistical Machine Translation (StatMT '08) , 2008 .

[48]  Mauro Cettolo,et al.  IRSTLM: an open source toolkit for handling large scale language models , 2008, INTERSPEECH.

[49]  Rafael E. Banchs,et al.  Word association models and search strategies for discriminative word alignment , 2008, EAMT.

[50]  Philipp Koehn,et al.  Further Meta-Evaluation of Machine Translation , 2008, WMT@ACL.

[51]  Stella Markantonatou,et al.  Evaluation of a Machine Translation System for Low Resource Languages: METIS-II , 2008, LREC.

[52]  William J. Byrne,et al.  Rule Filtering by Pattern for Efficient Hierarchical Translation , 2009, EACL.

[53]  Omar Zaidan,et al.  Z-MERT: A Fully Configurable Open Source Tool for Minimum Error Rate Training of Machine Translation Systems , 2009, Prague Bull. Math. Linguistics.

[54]  Nitin Madnani,et al.  TER-Plus: paraphrase, semantic, and alignment enhancements to Translation Edit Rate , 2009, Machine Translation.

[55]  Adel Jebali.  La modélisation des marqueurs d'arguments de l'arabe standard dans le cadre des grammaires à base de contraintes , 2009 .

[56]  Philipp Koehn,et al.  Findings of the 2009 Workshop on Statistical Machine Translation , 2009, WMT@EACL.

[57]  Hao Yu,et al.  Discarding monotone composed rule for hierarchical phrase-based statistical machine translation , 2009, IUCS '09.

[58]  Chris Callison-Burch,et al.  Joshua 2.0: A Toolkit for Parsing-Based Machine Translation with Syntax, Semirings, Discriminative Training and Other Goodies , 2010, WMT@ACL.

[59]  Loïc Barrault.  Many: Open Source Machine Translation System Combination , 2010, Prague Bull. Math. Linguistics.

[60]  William J. Byrne,et al.  Hierarchical Phrase-Based Translation with Weighted Finite-State Transducers and Shallow-n Grammars , 2010, CL.

[61]  Andy Way,et al.  MaTrEx: The DCU MT System for WMT 2008 , 2008, WMT@ACL.

[62]  Yang Liu,et al.  Discriminative Word Alignment by Linear Modeling , 2010, CL.

[63]  Kimberly A. Neuendorf,et al.  Reliability for Content Analysis , 2010 .

[64]  Hermann Ney,et al.  Jane: Open Source Hierarchical Translation, Extended with Reordering and Lexicon Models , 2010, WMT@ACL.

[65]  Christian Federmann,et al.  Appraise: An Open-Source Toolkit for Manual Phrase-Based Evaluation of Translations , 2010, LREC.

[66]  C. Federmann,et al.  DFKI System Combination using Syntactic Information at ML4HMT-2011 , 2011 .

[67]  Adam Lopez,et al.  Joshua 3.0: Syntax-based Machine Translation with the Thrax Grammar Extractor , 2011, WMT@EMNLP.

[68]  Markus Freitag,et al.  A Guide to Jane, an Open Source Hierarchical Translation Toolkit , 2011, Prague Bull. Math. Linguistics.

[69]  Kenneth Heafield,et al.  KenLM: Faster and Smaller Language Model Queries , 2011, WMT@EMNLP.

[70]  Mark Hopkins,et al.  Tuning as Ranking , 2011, EMNLP.

[71]  Ondrej Bojar,et al.  A Grain of Salt for the WMT Manual Evaluation , 2011, WMT@EMNLP.

[72]  Gholamreza Haffari,et al.  Bayesian Extraction of Minimal SCFG Rules for Hierarchical Phrase-based Translation , 2011, WMT@EMNLP.

[73]  Eleftherios Avramidis,et al.  DFKI System Combination with Sentence Ranking at ML4HMT-2011 , 2011 .

[74]  Tsuyoshi Okita.  DCU Confusion Network-based System Combination for ML 4 HMT , 2011 .

[75]  Anoop Sarkar,et al.  Mixing Multiple Translation Models in Statistical Machine Translation , 2012, ACL.

[76]  Anoop Sarkar,et al.  Kriya - An end-to-end Hierarchical Phrase-based MT System , 2012, Prague Bull. Math. Linguistics.

[77]  Mahmoud Fawzi Mammeri,et al.  Le syntagme nominal défini en arabe standard contemporain , 2012, Prague Bull. Math. Linguistics.