论文信息 - Precise measurement method of a speech translation system’s capability with a paired comparison method between the system and humans

Precise measurement method of a speech translation system’s capability with a paired comparison method between the system and humans

The main goal of the present paper is to propose new schemes for the overall evaluation of a speech translation system. These schemes are expected to support and improve the design of the target application system, and precisely determine its performance. Experiments are conducted on the Japanese-to-English speech translation system ATR-MATRIX, which was developed at ATR Interpreting Telecommunications Research Laboratories. In the proposed schemes, the system’s translations are compared with those of a native Japanese taking the Test of English for International Communication (TOEIC), which is used as a measure of one’s speech translation capability. Subjective and automatic comparisons are made and the results are compared. A regression analysis on the subjective results shows that the speech translation capability of ATR-MATRIX matches a Japanese person scoring around 500 on the TOEIC. The automatic comparisons also show promising results.

T. Takezawa | F. Sugaya | Seiichi Yamamoto | K. Yasuda

[1] Hitoshi Iida,et al. A speech and language database for speech translation research , 1994, ICSLP.

[2] Yoshinori Sagisaka,et al. Variable-order N-gram generation by word-class splitting and consecutive word grouping , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[3] Hitoshi Iida,et al. A Japanese-to-English speech translation system: ATR-MATRIX , 1998, ICSLP.

[4] Toshiyuki Takezawa,et al. End-to-end evaluation in ATR-MATRIX: speech translation system between English and Japanese , 1999, EUROSPEECH.

[5] Eiichiro Sumita,et al. Solutions to Problems Inherent in Spoken-language Translation: The ATR-MATRIX Approach , 1999 .

[6] Wolfgang Wahlster,et al. Verbmobil: Foundations of Speech-to-Speech Translation , 2000, Artificial Intelligence.

[7] Yoshinori Sagisaka,et al. Evaluation of the ATR-matrix speech translation system with a pair comparison method between the system and humans , 2000, INTERSPEECH.