Choosing the Right Translation: A Syntactically Informed Classification Approach

One style of Multi-Engine Machine Translation architecture involves choosing the best of a set of outputs from different systems. Choosing the best translation from an arbitrary set, even in the presence of human references, is a difficult problem; it may prove better to look at mechanisms for making such choices in more restricted contexts. In this paper we take a classification-based approach to choosing between candidates from syntactically informed translations. The idea is that using multiple parsers as part of a classifier could help detect syntactic problems in this context that lead to bad translations; these problems could be detected on either the source side---perhaps sentences with difficult or incorrect parses could lead to bad translations---or on the target side---perhaps the output quality could be measured in a more syntactically informed way, looking for syntactic abnormalities. We show that there is no evidence that the source side information is useful. However, a target-side classifier, when used to identify particularly bad translation candidates, can lead to significant improvements in BLEU score. Improvements are even greater when combined with existing language and alignment model approaches.

[1]  Tadashi Nomoto.  Multi-Engine Machine Translation with Voted Language Model , 2004, ACL.

[2]  Chao Wang,et al.  Chinese Syntactic Reordering for Statistical Machine Translation , 2007, EMNLP.

[3]  Sergei Nirenburg,et al.  Three Heads are Better than One , 1994, ANLP.

[4]  Philipp Koehn,et al.  Re-evaluating the Role of Bleu in Machine Translation Research , 2006, EACL.

[5]  Stefan Riezler,et al.  Grammatical Machine Translation , 2006, NAACL.

[6]  Mark Dras,et al.  This Phrase-Based SMT System is Out of Order: Generalised Word Reordering in Machine Translation , 2006, ALTA.

[7]  Philipp Koehn,et al.  Clause Restructuring for Statistical Machine Translation , 2005, ACL.

[8]  John D. Lafferty,et al.  A Robust Parsing Algorithm for Link Grammars , 1995, IWPT.

[9]  Tadashi Nomoto.  Predictive models of performance in multi-engine machine translation , 2003, MTSUMMIT.

[10]  Stephen Wan,et al.  GLEU: Automatic Evaluation of Sentence-Level Fluency , 2007, ACL.

[11]  Philipp Koehn,et al.  Pharaoh: A Beam Search Decoder for Phrase-Based Statistical Machine Translation Models , 2004, AMTA.

[12]  Michael Gamon,et al.  A Machine Learning Approach to the Automatic Evaluation of Machine Translation , 2001, ACL.

[13]  Dan Klein,et al.  Optimization, Maxent Models, and Conditional Estimation without Magic , 2003, NAACL.

[14]  Rebecca Hwa,et al.  Regression for Sentence-Level MT Evaluation with Pseudo References , 2007, ACL.

[15]  Richard M. Schwartz,et al.  Combining Outputs from Multiple Machine Translation Systems , 2007, NAACL.

[16]  Chris Callison-Burch,et al.  A program for automatically selecting the best output from multiple machine translation engines , 2001, MTSUMMIT.

[17]  Mark Dras,et al.  Syntax-based word reordering in phrase-based statistical machine translation: why does it work? , 2007, MTSUMMIT.

[18]  Fei Huang,et al.  Hierarchical System Combination for Machine Translation , 2007, EMNLP.

[19]  Michael Collins,et al.  Head-Driven Statistical Models for Natural Language Parsing , 2003, CL.

[20]  Gertjan van Noord,et al.  Alpino: Wide-coverage Computational Analysis of Dutch , 2000, CLIN.

[21]  Thorsten Joachims,et al.  Making large-scale support vector machine learning practical , 1999 .

[22]  Andreas Eisele.  First Steps towards Multi-Engine Machine Translation , 2005, ParallelText@ACL.

[23]  Taro Watanabe,et al.  Using Language and Translation Models to Select the Best among Outputs from Multiple MT Systems , 2002, COLING.

[24]  Kenji Yamada,et al.  Syntax-based language models for statistical machine translation , 2003, ACL 2003.