Experiences in evaluating multilingual and text‐image information retrieval

One important step during the development of information retrieval (IR) processes is the evaluation of the output regarding the information needs of the user. The “high quality” of the output is related to the integration of different methods to be applied in the IR process and the information included in the retrieved documents, but how can “quality” be measured? Although some of these methods can be tested in a stand‐alone way, it is not always clear what will happen when several methods are integrated. For this reason, much effort has been put into establishing a good combination of several methods or to correctly tuning some of the algorithms involved. The current approach is to measure the precision and recall figures yielded when different combinations of methods are included in an IR process. In this article, a short description of the current techniques and methods included in an IR system is given, paying special attention to the multilingual aspect of the problem. Also a discussion of their influence on the final performance of the IR process is presented by explaining previous experiences in the evaluation process followed in two projects (MIRACLE and OmniPaper) related to multilingual information retrieval. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 655–677, 2006.