The CLEF Cross Language Image Retrieval Track (ImageCLEF) 2004

In this paper we describe ImageCLEF, the cross language image retrieval track of the Cross Language Evaluation Forum (CLEF). We instigated and ran a pilot experiment in 2003 where participants submitted entries for an ad hoc bilingual image retrieval task on a collection of historic photographs from St. Andrews University Library. This was designed to simulate the situation in which users would express their search request in natural language but require visual documents in return. For 2004 we have extended the tasks to include a medical image retrieval task and a user-centred evaluation.

[1]  Gregory Grefenstette,et al.  Cross-Language Information Retrieval , 1998, The Springer International Series on Information Retrieval.

[2]  Cyril Cleverdon,et al.  The Cranfield tests on index language devices , 1997 .

[3]  Hsin-Hsi Chen,et al.  Overview of CLIR Task at the Third NTCIR Workshop , 2002, NTCIR.

[4]  Ingemar J. Cox,et al.  Target testing and the PicHunter Bayesian multimedia retrieval system , 1996, Proceedings of the Third Forum on Research and Technology Advances in Digital Libraries,.

[5]  David A. Forsyth,et al.  Benchmarks for storage and retrieval in multimedia databases , 2001, IS&T/SPIE Electronic Imaging.

[6]  Mark D. Dunlop,et al.  Image retrieval by hypertext links , 1997, SIGIR '97.

[7]  Hinrich Schütze,et al.  Multimodal browsing of images in Web documents , 1999, Electronic Imaging.

[8]  Carol Peters,et al.  European research letter: Cross-language system evaluation: The CLEF campaigns , 2001, J. Assoc. Inf. Sci. Technol..

[9]  M. Markkula,et al.  Searching for Photos - Journalists' Practices in Pictorial IR , 1998 .

[10]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  John Tait,et al.  Evaluating a content based image retrieval system , 2001, SIGIR '01.

[12]  Thomas Martin Deserno,et al.  The CLEF 2005 Cross-Language Image Retrieval Track , 2003, CLEF.

[13]  Peter G. B. Enser,et al.  Analysis of user need in image archives , 1997, J. Inf. Sci..

[14]  Abby Goodrum,et al.  Image Information Retrieval: An Overview of Current Research , 2000, Informing Sci. Int. J. an Emerg. Transdiscipl..

[15]  Mark D. Dunlop.  Reflections on Mira: Interactive evaluation in information retrieval , 2000, J. Am. Soc. Inf. Sci..

[16]  Sharon Flank.  Cross-Language Multimedia Information Retrieval , 2000, ANLP.

[17]  Sharon Flank.  Sentences vs. Phrases: Syntactic Complexity in Multimedia Information Retrieval , 2000 .

[18]  Ellen M. Voorhees,et al.  Overview of TREC 2001 , 2001, TREC.

[19]  Thierry Pun,et al.  Performance evaluation in content-based image retrieval: overview and proposals , 2001, Pattern Recognit. Lett..

[20]  Mark Sanderson,et al.  Assessing Translation Quality for Cross Language Image Retrieval , 2003, CLEF.

[21]  Joshua R. Smith,et al.  Image retrieval evaluation , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).