论文信息 - An Automatic Evaluation Method for Localization Oriented Lexicalised EBMT System

An Automatic Evaluation Method for Localization Oriented Lexicalised EBMT System

To help developing a localization oriented EBMT system, an automatic machine translation evaluation method is implemented which adopts edit distance, cosine correlation and Dice coefficient as criteria. Experiment shows that the evaluation method distinguishes well between "good" translations and "bad" ones. To prove that the method is consistent with human evaluation, 6 MT systems are scored and compared. Theoretical analysis is made to validate the experimental results. Correlation coefficient and significance tests at 0.01 level are made to ensure the reliability of the results. Linear regression equations are calculated to map the automatic scoring results to human scorings.

Tiejun Zhao | Ming Zhou | Hao Yu | Jian-Min Yao | Sheng Li

[1] Hinrich Schütze,et al. Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[2] A. Guessoum,et al. Semi-automatic evaluation of the grammatical coverage of machine translation systems , 2001, MTSUMMIT.

[3] Terumasa Ehara,et al. An automatic evaluation method for machine translation using two-way MT , 2001 .

[4] Vladimir I. Levenshtein,et al. Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[5] Douglas A. Jones,et al. Toward a Scoring Function for Quality-Driven Machine Translation , 2000, COLING.

[6] Eiichiro Sumita,et al. Using multiple edit distances to automatically rank machine translation output , 2001, MTSUMMIT.

[7] Martin Volk,et al. Evaluating Translation Quality as Input to Product Development , 2000, LREC.

[8] Michael J. Fischer,et al. The String-to-String Correction Problem , 1974, JACM.

[9] Toshiyuki Takezawa,et al. An automatic evaluation method of translation quality using translation answer candidates queried from a parallel corpus , 2001 .

[10] Sungryong Koh,et al. A test suite for evaluation of English-to-Korean machine translation systems , 2001 .

[11] Chris Brew,et al. Automatic Evaluation of Computer Generated Text: A Progress Report on the TextEval Project , 1994, HLT.

[12] Yu Shiwen. Automatic evaluation of output quality for Machine Translation systems , 1993 .