In this paper a robust, adaptive approach for mining parallel sentences from a bilingual comparable news collection is described Sentence length models and lexicon-based models are combined under a maximum likelihood criterion. Specific models are proposed to handle insertions and deletions that are frequent in bilingual data collected from the web. The proposed approach is adaptive, updating the translation lexicon iteratively using the mined parallel data to get better vocabulary coverage and translation probability parameter estimation. Experiments are carried out on 10 years of Xinhua bilingual news collection. Using the mined data, we get significant improvement in word-to-word alignment accuracy in machine translation modeling.
[1]
Robert L. Mercer,et al.
The Mathematics of Statistical Machine Translation: Parameter Estimation
,
1993,
CL.
[2]
Kenneth Ward Church.
Char_align: A Program for Aligning Parallel Texts at the Character Level
,
1993,
ACL.
[3]
Hermann Ney,et al.
Improved Statistical Alignment Models
,
2000,
ACL.
[4]
Stanley F. Chen,et al.
Aligning Sentences in Bilingual Corpora Using Lexical Information
,
1993,
ACL.
[5]
Xiaoyi Ma,et al.
BITS: a method for bilingual text search over the Web
,
1999,
MTSUMMIT.