General Overview of ImageCLEF at the CLEF 2015 Labs

This paper presents an overview of the ImageCLEF 2015 evaluation campaign, an event that was organized as part of the CLEF labs 2015. ImageCLEF is an ongoing initiative that promotes the evaluation of technologies for annotation, indexing and retrieval for providing information access to databases of images in various usage scenarios and domains. In 2015, the 13th edition of ImageCLEF, four main tasks were proposed: 1 automatic concept annotation, localization and sentence description generation for general images; 2 identification, multi-label classification and separation of compound figures from biomedical literature; 3 clustering of x-rays from all over the body; and 4 prediction of missing radiological annotations in reports of liver CT images. The x-ray task was the only fully novel task this year, although the other three tasks introduced modifications to keep up relevancy of the proposed challenges. The participation was considerably positive in this edition of the lab, receiving almost twice the number of submitted working notes papers as compared to previous years.

[1]  Sameer Antani,et al.  Creating a classification of image types in the medical literature for visual categorization , 2012, Other Conferences.

[2]  Tim Causer,et al.  Building A Volunteer Community: Results and Findings from Transcribe Bentham , 2012, Digit. Humanit. Q..

[3]  Roberto Paredes,et al.  Overview of the ImageCLEF 2016 Scalable Concept Image Annotation Task , 2016, CLEF.

[4]  Henning Müller,et al.  Assessing the Scholarly Impact of ImageCLEF , 2011, CLEF.

[5]  Miguel Cazorla,et al.  ImageCLEF 2014: Overview and Analysis of the Results , 2014, CLEF.

[6]  Marina Bosch,et al.  ImageCLEF, Experimental Evaluation in Visual Information Retrieval , 2010 .

[7]  Ioannis Pratikakis,et al.  Segmentation-free Word Spotting in Historical Printed Documents , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[8]  Alejandro Héctor Toselli,et al.  ICFHR2014 Competition on Handwritten Text Recognition on Transcriptorium Datasets (HTRtS) , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[9]  Adolfo Jonathan Salinas-López,et al.  CEUR Workshop Proceedings , 2015 .

[10]  Ioannis A. Kakadiaris,et al.  Results of the BioASQ Tasks of the Question Answering Lab at CLEF 2015 , 2015, CLEF.

[11]  M. Ashraful Amin,et al.  Teaching & Learning System for Diagnostic Imaging - Phase I: X-Ray Image Analysis & Retrieval , 2015, CSEDU.

[12]  Konstantinos Zagoris,et al.  ICFHR 2014 Competition on Handwritten Keyword Spotting (H-KWS 2014) , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[13]  Neda Barzegar Marvasti,et al.  ImageCLEF Liver CT Image Annotation Task 2014 , 2014, CLEF.

[14]  Alon Lavie,et al.  Meteor Universal: Language Specific Translation Evaluation for Any Target Language , 2014, WMT@ACL.

[15]  Volkmar Frinken,et al.  A Novel Word Spotting Algorithm Using Bidirectional Long Short-Term Memory Neural Networks , 2010, ANNPR.

[16]  Christophoros Nikou,et al.  Word Spotting in Handwritten Text Using Contour-Based Models , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[17]  Christoph M. Friedrich,et al.  FHDO Biomedical Computer Science Group at Medical Classification Task of ImageCLEF 2015 , 2015, CLEF.

[18]  Hervé Glotin,et al.  LifeCLEF 2014: Multimedia Life Species Identification Challenges , 2014, CLEF.

[19]  Andreas Keller,et al.  Lexicon-free handwritten word spotting using character HMMs , 2012, Pattern Recognit. Lett..

[20]  José Francisco Aldana Montes,et al.  Overview of the ImageCLEF 2015 liver CT annotation task , 2015, CLEF.

[21]  Gareth J. F. Jones,et al.  ShARe/CLEF eHealth Evaluation Lab 2014, Task 3: User-centred Health Information Retrieval , 2014, CLEF.

[22]  Henning Müller,et al.  Overview of the ImageCLEF 2013 Medical Tasks , 2013, CLEF.

[23]  Josep Lladós,et al.  Integrating Visual and Textual Cues for Query-by-String Word Spotting , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[24]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[25]  Mette Skov,et al.  CLEF 2013 Evaluation Labs and Workshop, Online Working Notes , 2013 .

[26]  Erhard Rahm,et al.  The Scholarly Impact of CLEF (2000-2009) , 2013, CLEF.

[27]  Emmanuel Dellandréa,et al.  Overview of the ImageCLEF 2015 Scalable Image Annotation, Localization and Sentence Generation task , 2015, CLEF.

[28]  Mauricio Villegas Santamaría,et al.  Overview of the ImageCLEF 2014 Scalable Concept Image Annotation Task , 2014 .

[29]  Alejandro Héctor Toselli Rossi,et al.  Context-aware lattice based filler approach for key word spotting in handwritten documents , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[30]  Alejandro Héctor Toselli,et al.  ICDAR 2015 competition HTRtS: Handwritten Text Recognition on the tranScriptorium dataset , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[31]  Alejandro Héctor Toselli,et al.  ICDAR2015 Competition on Keyword Spotting for Handwritten Documents , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[32]  Justus H. Piater,et al.  IIS at ImageCLEF 2015: Multi-label Classification Task , 2015, CLEF.

[33]  George R. Thoma,et al.  NLM at imageCLEF2015: Biomedical Multipanel Figure Separation , 2015, CLEF.

[34]  Roberto Paredes,et al.  Overview of the ImageCLEF 2012 Scalable Web Image Annotation Task , 2012, CLEF.

[35]  R. Manmatha,et al.  Holistic word recognition for handwritten historical documents , 2004, First International Workshop on Document Image Analysis for Libraries, 2004. Proceedings..

[36]  José A. Rodríguez-Serrano,et al.  Handwritten word-spotting using hidden Markov models and universal vocabularies , 2009, Pattern Recognit..

[37]  Robert J. Gaizauskas,et al.  Generating Image Descriptions with Gold Standard Visual Inputs: Motivation, Evaluation and Baselines , 2015, ENLG.

[38]  Henning Müller,et al.  Overview of the ImageCLEF 2016 Medical Task , 2016, CLEF.

[39]  Henning Müller,et al.  Evaluating performance of biomedical image retrieval systems - An overview of the medical image retrieval task at ImageCLEF 2004-2013 , 2015, Comput. Medical Imaging Graph..

[40]  Nikos Papamarkos,et al.  Image retrieval systems based on compact shape descriptor and relevance feedback information , 2011, J. Vis. Commun. Image Represent..

[41]  M. Ashraful Amin,et al.  Overview of the ImageCLEF 2015 Medical Clustering Task , 2015, CLEF.

[42]  Pinar Yolum,et al.  Semantic Description of Liver CT Images: An Ontological Approach , 2014, IEEE Journal of Biomedical and Health Informatics.

[43]  Sven Koitka,et al.  Traditional Feature Engineering and Deep Learning Approaches at Medical Classification Task of ImageCLEF 2016 , 2016, CLEF.

[44]  Alejandro Héctor Toselli,et al.  Overview of the ImageCLEF 2016 Handwritten Scanned Document Retrieval Task , 2016, CLEF.

[45]  Cyril Grouin,et al.  Overview of the CLEF eHealth Evaluation Lab 2015 , 2015, CLEF.

[46]  B. Thomee,et al.  Overview of the ImageCLEF 2013 Scalable Concept Image Annotation Subtask , 2013, CLEF.

[47]  Henning Müller,et al.  Overview of the ImageCLEF 2015 Medical Classification Task , 2015, CLEF.

[48]  Henning Müller,et al.  Overview of the ImageCLEF 2012 Medical Image Retrieval and Classification Tasks , 2012, CLEF.