Empirical studies on the impact of lexical resources on CLIR performance

In this paper, we compile and review several experiments measuring cross-lingual information retrieval (CLIR) performance as a function of the following resources: bilingual term lists, parallel corpora, machine translation (MT), and stemmers. Our CLIR system uses a simple probabilistic language model; the studies used TREC test corpora over Chinese, Spanish and Arabic. Our findings include: • One can achieve an acceptable CLIR performance using only a bilingual term list (70-80% on Chinese and Arabic corpora). • However, if a bilingual term list and parallel corpora are available, CLIR performance can rival monolingual performance. • If no parallel corpus is available, pseudo-parallel texts produced by an MT system can partially overcome the lack of parallel text. • While stemming is useful normally, with a very large parallel corpus for Arabic-English, stemming hurt performance in our empirical studies with Arabic, a highly inflected language.

[1]  Ellen M. Voorhees,et al.  The Ninth Text REtrieval Conference (TREC-9) , 2001 .

[2]  Wessel Kraaij,et al.  TNO at CLEF-2001: Comparing Translation Resources , 2001, CLEF.

[3]  Gregory Grefenstette.  Evaluating the adequacy of a multilingual transfer dictionary for the cross language information retrieval , 1998 .

[4]  Richard M. Schwartz,et al.  A hidden Markov model information retrieval system , 1999, SIGIR '99.

[5]  Robert L. Mercer,et al.  The Mathematics of Statistical Machine Translation: Parameter Estimation , 1993, CL.

[6]  Douglas W. Oard,et al.  The effect of bilingual term list size on dictionary-based cross-language information retrieval , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[7]  Pascale Fung,et al.  Finding Terminology Translations from Non-parallel Corpora , 1997, VLC.

[8]  Fredric C. Gey,et al.  Manual Queries and Machine Translation in Cross-Language Retrieval and Interactive Retrieval with Cheshire II at TREC-7 , 1998, TREC.

[9]  David A. Hull.  Using statistical testing in the evaluation of retrieval experiments , 1993, SIGIR.

[10]  Alexander M. Fraser,et al.  TREC 2001 Cross-lingual Retrieval at BBN , 2001, TREC.

[11]  J. Scott McCarley.  Should we Translate the Documents or the Queries in Cross-language Information Retrieval? , 1999, ACL.

[12]  W. Bruce Croft,et al.  Corpus-based stemming using cooccurrence of word variants , 1998, TOIS.

[13]  Lisa Ballesteros,et al.  Improving stemming for Arabic information retrieval: light stemming and co-occurrence analysis , 2002, SIGIR '02.

[14]  Ellen M. Voorhees,et al.  The seventh text REtrieval conference (TREC-7) , 1999 .

[15]  Ellen M. Voorhees,et al.  The fifth text REtrieval conference (TREC-5) , 1997 .

[16]  Ellen M. Voorhees,et al.  The Tenth Text REtrieval Conference, TREC 2001 | NIST , 2002 .

[17]  David J. Goodman,et al.  Personal Communications , 1994, Mobile Communications.

[18]  Douglas W. Oard,et al.  A comparative study of query and document translation for cross-language information retrieval , 1998, AMTA.

[19]  Djoerd Hiemstra,et al.  Disambiguation Strategies for Cross-Language Information Retrieval , 1999, ECDL.

[20]  John D. Lafferty,et al.  Information Retrieval as Statistical Translation , 2017 .

[21]  Amit Singhal,et al.  Pivoted document length normalization , 1996, SIGIR 1996.

[22]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[23]  Kui-Lam Kwok,et al.  TREC-9 Cross Language, Web and Question-Answering Track Experiments using PIRCS , 2000, TREC.

[24]  Ralph Weischedel,et al.  A Probabilistic Approach to Term Translation for Cross-Lingual Retrieval , 2003 .

[25]  Jinxi Xu,et al.  Cross-lingual Information Retrieval Using Hidden Markov Models , 2000, EMNLP.

[26]  Yaser Al-Onaizan,et al.  Translating Named Entities Using Monolingual and Bilingual Resources , 2002, ACL.

[27]  Martin Franz,et al.  Quantifying the utility of parallel corpora , 2001, SIGIR '01.

[28]  W. Bruce Croft,et al.  A language modeling approach to information retrieval , 1998, SIGIR '98.

[29]  Donna K. Harman,et al.  How effective is suffixing? , 1991, J. Am. Soc. Inf. Sci..

[30]  Donna Harman,et al.  How effective is suffixing , 1991 .

[31]  James Mayfield,et al.  Comparing cross-language query expansion techniques by degrading translation resources , 2002, SIGIR '02.

[32]  Ellen M. Voorhees,et al.  The eleventh text REtrieval conference, TREC 2002 , 2003 .

[33]  Jinxi Xu,et al.  Evaluating a probabilistic model for cross-lingual information retrieval , 2001, SIGIR '01.

[34]  W. Bruce Croft,et al.  Resolving ambiguity for cross-language retrieval , 1998, SIGIR '98.

[35]  Hermann Ney,et al.  Improved Statistical Alignment Models , 2000, ACL.