Exploiting Wikipedia API for Hindi-english Cross-language Information Retrieval☆

Abstract The rapidly increasing demographics of the internet population and the abundance of multilingual content on the web increased the communication in multiple languages. Most of the people use their regional languages to express their needs and the language diversity becomes a great barrier. Cross-Language Information Retrieval (CLIR) provides a solution for that language barrier which allows a user to ask a query in the native language and get the relevant documents in the different language. In this paper, we proposed a Wikipedia API based query translation approach. Queries are tokenized and multi-words query terms are created using N-gram technique. Wikipedia title and inter-wiki link features are exploited for query translation. Target language documents are retrieved using vector space retrieval model and BM25 retrieval algorithm. Experiment results shows that the proposed approach achieves better results without exploiting any language resources.

[1]  Takahiro Hara,et al.  Improving the extraction of bilingual terminology from Wikipedia , 2009, TOMCCAP.

[2]  Vasudeva Varma,et al.  Language independent identification of parallel sentences using Wikipedia , 2011, WWW.

[3]  Vasudeva Varma,et al.  Language Independent Named Entity Identification using Wikipedia , 2012 .

[4]  Vasudeva Varma,et al.  Approximate String Matching Techniques for Effective CLIR Among Indian Languages , 2007, WILF.

[5]  Vasudeva Varma,et al.  Language-Independent Context Aware Query Translation using Wikipedia , 2011, BUCC@ACL.

[6]  A. Kumaran,et al.  Cross-Lingual Information Retrieval System for Indian Languages , 2008, IJCNLP.

[7]  Udhyakumar Nallasamy,et al.  Named entity transliteration for cross-language information retrieval using compressed word format mapping algorithm , 2008, iNEWS '08.

[8]  Malek Boualem,et al.  Query translation using Wikipedia-based resources for analysis and disambiguation , 2010, EAMT.

[9]  John Tait,et al.  Literature Review of Cross Language Information Retrieval , 2005, WEC.

[10]  Maarten de Rijke,et al.  Finding Similar Sentences across Multiple Languages in Wikipedia , 2006 .

[11]  Vasudeva Varma,et al.  Hindi and Telugu to English Cross Language Information Retrieval at CLEF 2006 , 2006, CLEF.

[12]  Nurul Amelina Nasharuddin,et al.  Cross-lingual Information Retrieval State-of-the-Art , 2010 .

[13]  PothulaSujatha,et al.  A Review on the Cross and Multilingual Information Retrieval , 2011 .

[14]  S. Saraswathi,et al.  State of Art: Cross Lingual Information Retrieval System for Indian Languages , 2011 .

[15]  Namita Mittal,et al.  Cross Lingual Information Retrieval (CLIR): Review of Tools, Challenges and Translation Approaches , 2016 .