Use case oriented medical visual information retrieval & system evaluation

Large amounts of medical visual data are produced daily in hospitals, while new imaging techniques continue to emerge. In addition, many images are made available continuously via publications in the scientific literature and can also be valuable for clinical routine, research and education. Information retrieval systems are useful tools to provide access to the biomedical literature and fulfil the information needs of medical professionals. The tools developed in this thesis can potentially help clinicians make decisions about difficult diagnoses via a case-based retrieval system based on a use case associated with a specific evaluation task. This system retrieves articles from the biomedical literature when querying with a case description and attached images. This thesis proposes a multimodal approach for medical case-based retrieval with focus on the integration of visual information connected to text. Furthermore, the ImageCLEFmed evaluation campaign was organised during this thesis promoting medical retrieval system evaluation.

[1]  Franz Kainberger,et al.  Khresmoi: Multimodal Multilingual Medical Information Search. , 2012 .

[2]  Sameer Antani,et al.  Creating a classification of image types in the medical literature for visual categorization , 2012, Other Conferences.

[3]  Henning Müller,et al.  Log analysis to understand medical professionals' image searching behaviour. , 2012, Studies in health technology and informatics.

[4]  S. Sclaroff,et al.  Combining textual and visual cues for content-based image retrieval on the World Wide Web , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[5]  K. Cohen,et al.  Biomedical language processing: what's beyond PubMed? , 2006, Molecular cell.

[6]  Selim Aksoy,et al.  Proceedings of the 20th International conference on Recognizing patterns in signals, speech, images, and videos , 2010 .

[7]  Yiannis Gkoufas,et al.  Combining Textual and Visual Information for Image Retrieval in the Medical Domain , 2011, The open medical informatics journal.

[8]  Noriko Kando,et al.  Overview of IR tasks , 1999, NTCIR.

[9]  Eero Sormunen,et al.  Liberal relevance criteria of TREC -: counting on negligible documents? , 2002, SIGIR '02.

[10]  Rong Yan,et al.  The combination limit in multimedia retrieval , 2003, MULTIMEDIA '03.

[11]  Henning Müller,et al.  The medGIFT Group in ImageCLEFmed 2013 , 2013, CLEF.

[12]  Henning Müller,et al.  Meaningful Bags of Words for Medical Image Classification and Retrieval , 2015, Health Monitoring and Personalized Feedback using Multimedia Data.

[13]  V. B. Gaikwad Content based Medical Image Retrieval with SVM Classification and Relevance Feedback , 2013 .

[14]  E. A. Fox,et al.  Combining the Evidence of Multiple Query Representations for Information Retrieval , 1995, Inf. Process. Manag..

[15]  James Allan,et al.  The SMART Information Retrieval Project , 1993, HLT.

[16]  Luis Alfonso Ureña López,et al.  Query expansion with a medical ontology to improve a multimodal information retrieval system , 2009, Comput. Biol. Medicine.

[17]  Paul Over,et al.  Blind Men and Elephants: Six Approaches to TREC data , 1999, Information Retrieval.

[18]  William R. Hersh,et al.  Using Media Fusion and Domain Dimensions to Improve Precision in Medical Image Retrieval , 2009, CLEF.

[19]  Nicola Ferro,et al.  PROMISE - Participative Research labOratory for Multimedia and Multilingual Information Systems Evaluation , 2011, IRCDL.

[20]  Kai-Kuang Ma,et al.  Fuzzy color histogram and its use in color image retrieval , 2002, IEEE Trans. Image Process..

[21]  Péter Jacsó,et al.  The pros and cons of computing the h-index using Google Scholar , 2008, Online Inf. Rev..

[22]  Tetsuya Sakai RD-004 NTCIREVAL : A Generic Toolkit for Information Access Evaluation , 2011 .

[23]  Henning Müller,et al.  Evaluating performance of biomedical image retrieval systems - An overview of the medical image retrieval task at ImageCLEF 2004-2013 , 2015, Comput. Medical Imaging Graph..

[24]  Noriko Kando Overview of the Fourth NTCIR Workshop , 2004, NTCIR.

[25]  Henning Müller,et al.  Semi-supervised Learning for Image Modality Classification , 2015, MRDM@ECIR.

[26]  Joseph M. Hellerstein,et al.  Adaptive query processing: Why, How, When, and What Next? , 2007, VLDB.

[27]  Ellen M. Voorhees,et al.  Overview of the TREC 2012 Medical Records Track , 2012, TREC.

[28]  Ellen M. Voorhees,et al.  Retrieval System Evaluation , 2005 .

[29]  Charles L. A. Clarke,et al.  Reciprocal rank fusion outperforms condorcet and individual rank learning methods , 2009, SIGIR.

[30]  Gabriela Csurka,et al.  Medical image modality classification and retrieval , 2011, 2011 9th International Workshop on Content-Based Multimedia Indexing (CBMI).

[31]  Giuseppe Santucci,et al.  Measuring and Analyzing the Scholarly Impact of Experimental Evaluation Initiatives , 2014, IRCDL.

[32]  Orcun Goksel,et al.  VISCERAL | VISual Concept Extraction challenge in RAdioLogy: ISBI 2014 Challenge Organization , 2014 .

[33]  Dina Demner-Fushman,et al.  Biomedical Text Mining: A Survey of Recent Progress , 2012, Mining Text Data.

[34]  Noriko Kando Overview of the Second NTCIR Workshop , 2001, NTCIR.

[35]  Allen Kent,et al.  Machine literature searching VII. Machine functions and organization of semantic units , 1955 .

[36]  Yiannis S. Boutalis,et al.  FCTH: Fuzzy Color and Texture Histogram - A Low Level Feature for Accurate Image Retrieval , 2008, 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services.

[37]  Haejun Lee,et al.  Medical Image Retrieval: Past and Present , 2012, Healthcare informatics research.

[38]  Nozha Boujemaa,et al.  Generalized histogram intersection kernel for image recognition , 2005, IEEE International Conference on Image Processing 2005.

[39]  João Magalhães,et al.  NovaSearch on Medical ImageCLEF 2013 , 2013, CLEF.

[40]  Jayashree Kalpathy-Cramer,et al.  Multimodal medical image retrieval: image categorization to improve search precision , 2010, MIR '10.

[41]  Wei Song,et al.  BUAA AUDR at ImageCLEF 2012 Medical Retrieval Task , 2012, CLEF.

[42]  William R. Hersh,et al.  Automatic Image Modality Based Classification and Annotation to Improve Medical Image Retrieval , 2007, MedInfo.

[43]  John R. Smith,et al.  IBM T.J. Watson Research Center, Multimedia Analytics: Modality Classification and Case-Based Retrieval Tasks of ImageCLEF2012 , 2012, CLEF.

[44]  Gerard Salton,et al.  The State of Retrieval System Evaluation , 1992, Inf. Process. Manag..

[45]  Cord Spreckelsen,et al.  Towards case-based medical learning in radiological decision making using content-based image retrieval , 2011, BMC Medical Informatics Decis. Mak..

[46]  Henning Müller,et al.  Separating compound figures in journal articles to allow for subfigure classification , 2013, Medical Imaging.

[47]  Alex Pentland,et al.  Photobook: tools for content-based manipulation of image databases , 1994, Other Conferences.

[48]  Mohammad Shahram Moin,et al.  A New Content-Based Image Retrieval Approach Based on Pattern Orientation Histogram , 2007, MIRAGE.

[49]  Betsy L. Humphreys,et al.  Relationships in Medical Subject Headings (MeSH) , 2001 .

[50]  Roger Levy,et al.  A new approach to cross-modal multimedia retrieval , 2010, ACM Multimedia.

[51]  André Mourão,et al.  NovaMedSearch: a multimodal search engine for medical case-based retrieval , 2013, OAIR.

[52]  Yiannis Gkoufas,et al.  IPL at ImageCLEF 2011 Medical Retrieval Task , 2011, CLEF.

[53]  Noriko Kando,et al.  Overview of the Sixth NTCIR Workshop , 2007, NTCIR.

[54]  O. Ratib,et al.  Casimage Project: A Digital Teaching Files Authoring Environment , 2004, Journal of thoracic imaging.

[55]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Mark Gerstein,et al.  Training set expansion: an approach to improving the reconstruction of biological networks from limited and uneven reliable interactions , 2008, Bioinform..

[57]  S. Selvarajah,et al.  Analysis and Comparison of Texture Features for Content Based Image Retrieval , 2011 .

[58]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[59]  Siu-Tsen Shen,et al.  My favorites (bookmarks) schema: one solution to online information storage and retrieval , 2013, ISDOC.

[60]  Tetsuya Sakai,et al.  Overview of NTCIR-9 , 2011, NTCIR.

[61]  Fernando Llopis,et al.  Diversity Promotion: Is Reordering Top-Ranked Documents Sufficient? , 2009, CLEF.

[62]  Pierre Tirilly,et al.  On modality classification and its use in text-based image retrieval in medical databases , 2011, 2011 9th International Workshop on Content-Based Multimedia Indexing (CBMI).

[63]  Chih-Fong Tsai,et al.  Bag-of-Words Representation in Image Annotation: A Review , 2012 .

[64]  Fleur Mougin,et al.  Query Expansion using External Resources for Improving Information Retrieval in the Biomedical Domain , 2014, CLEF.

[65]  Stefano Mizzaro,et al.  The Good, the Bad, the Difficult, and the Easy: Something Wrong with Information Retrieval Evaluation? , 2008, ECIR.

[66]  Henning Müller,et al.  Gesture Interaction for Content--based Medical Image Retrieval , 2014, ICMR.

[67]  Mohan S. Kankanhalli,et al.  Multimodal fusion for multimedia analysis: a survey , 2010, Multimedia Systems.

[68]  Erhard Rahm,et al.  The Scholarly Impact of CLEF (2000-2009) , 2013, CLEF.

[69]  Cyril W. Cleverdon,et al.  Factors determining the performance of indexing systems , 1966 .

[70]  J. Michael Cherry,et al.  Concept Extraction and Synonymy Management for Biomedical Information Retrieval , 2004, TREC.

[71]  Arjen Hoogendam,et al.  Answers to Questions Posed During Daily Patient Care Are More Likely to Be Answered by UpToDate Than PubMed , 2008, Journal of medical Internet research.

[72]  Henning Müller,et al.  Overview of the CLEF 2011 Medical Image Classification and Retrieval Tasks , 2011, CLEF.

[73]  Chris Buckley,et al.  New Retrieval Approaches Using SMART: TREC 4 , 1995, TREC.

[74]  Péter Jacsó,et al.  Deflated, inflated and phantom citation counts , 2006, Online Inf. Rev..

[75]  Stéphane Marchand-Maillet,et al.  Benchmarking Image Retrieval Applications , 2004 .

[76]  Charles E. Kahn,et al.  Multi-modal Medical Image Retrieval , 2011 .

[77]  Rainer Lienhart,et al.  The Holy Grail of Multimedia Information Retrieval: So Close or Yet So Far Away? , 2008 .

[78]  Chris Buckley,et al.  OHSUMED: an interactive retrieval evaluation and new large test collection for research , 1994, SIGIR '94.

[79]  Markus Kreuzthaler,et al.  Navigating through Very Large Sets of Medical Records: An Information Retrieval Evaluation Architecture for Non-standardized Text , 2011, USAB.

[80]  Noriko Kando,et al.  Overview of the Third NTCIR Workshop , 2002, NTCIR.

[81]  Daniel L. Rubin,et al.  A hierarchical knowledge-based approach for retrieving similar medical images described with semantic annotations , 2014, J. Biomed. Informatics.

[82]  Hossein Nezamabadi-pour,et al.  A simultaneous feature adaptation and feature selection method for content-based image retrieval systems , 2013, Knowl. Based Syst..

[83]  Riccardo Bellazzi,et al.  Supporting decisions in medical applications: the knowledge management perspective , 2002, Int. J. Medical Informatics.

[84]  T. Peto,et al.  Crowdsourcing as a Novel Technique for Retinal Fundus Photography Classification: Analysis of Images in the EPIC Norfolk Cohort on Behalf of the UKBiobank Eye and Vision Consortium , 2013, PloS one.

[85]  Anil K. Jain,et al.  Image retrieval using color and shape , 1996, Pattern Recognit..

[86]  Daniel E. Rose,et al.  Understanding user goals in web search , 2004, WWW '04.

[87]  R. Manmatha,et al.  A Model for Learning the Semantics of Pictures , 2003, NIPS.

[88]  Elizabeth D. Liddy,et al.  Enhanced Text Retrieval Using Natural Language Processing , 2005 .

[89]  Yiannis S. Boutalis,et al.  CEDD: Color and Edge Directivity Descriptor: A Compact Descriptor for Image Indexing and Retrieval , 2008, ICVS.

[90]  Daekeun You,et al.  Multimodal biomedical image retrieval using hierarchical classification and modality fusion , 2013, International Journal of Multimedia Information Retrieval.

[91]  Henning Müller,et al.  Overview of the ImageCLEFmed 2008 Medical Image Retrieval Task , 2008, CLEF.

[92]  Dimitris K. Iakovidis,et al.  Fusion of multimodal temporal clinical data for the retrieval of similar patient cases , 2011, 2011 10th International Workshop on Biomedical Engineering.

[93]  M. Kwiatkowska,et al.  Case Representation and Retrieval in the Diagnosis and Treatment of Obstructive Sleep Apnea : A Semio-fuzzy Approach , 2004 .

[94]  Rafael Llobet,et al.  Training Set Expansion in Handwritten Character Recognition , 2002, SSPR/SPR.

[95]  Henning Müller,et al.  Multi-scale visual words for hierarchical medical image categorisation , 2012, Other Conferences.

[96]  Eugene Kim,et al.  Overview of the ImageCLEFmed 2006 Medical Retrieval and Medical Annotation Tasks , 2006, CLEF.

[97]  Henning Müller,et al.  Overview of the ImageCLEF 2013 Medical Tasks , 2013, CLEF.

[98]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[99]  George R. Thoma,et al.  Design and Development of a Multimodal Biomedical Information Retrieval System , 2012, J. Comput. Sci. Eng..

[100]  Joo-Hwee Lim,et al.  Latent semantic fusion model for image retrieval and annotation , 2007, CIKM '07.

[101]  Stéphane Marchand-Maillet,et al.  Information Fusion in Multimedia Information Retrieval , 2007, Adaptive Multimedia Retrieval.

[102]  Joo-Hwee Lim,et al.  Medical-Image Retrieval Based on Knowledge-Assisted Text and Image Indexing , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[103]  Noriko Kando Overview of the Eighth NTCIR Workshop , 2010, NTCIR.

[104]  Theodore Kalamboukis,et al.  IPL at CLEF 2013 Medical Retrieval Task , 2013, CLEF.

[105]  Z. Popovic,et al.  Crystal structure of a monomeric retroviral protease solved by protein folding game players , 2011, Nature Structural &Molecular Biology.

[106]  Andrei Broder,et al.  A taxonomy of web search , 2002, SIGF.

[107]  Ivan Kitanovski,et al.  FCSE at Medical Tasks of ImageCLEF 2013 , 2013, CLEF.

[108]  James Ze Wang Region-based retrieval of biomedical images , 2000, ACM Multimedia.

[109]  Henning Müller,et al.  Overview of the ImageCLEF 2012 Medical Image Retrieval and Classification Tasks , 2012, CLEF.

[110]  Shih-Fu Chang,et al.  Query-Adaptive Fusion for Multimodal Search , 2008, Proceedings of the IEEE.

[111]  Thomas Martin Deserno,et al.  The CLEF 2005 Cross-Language Image Retrieval Track , 2005, CLEF.

[112]  Karen Spärck Jones Reflections on TREC , 1995, Inf. Process. Manag..

[113]  S ViswaS Efficient Retrieval of Images for Search Engine by Visual Similarity and Re Ranking , 2013 .

[114]  Shih-Fu Chang,et al.  Reranking Methods for Visual Search , 2007, IEEE MultiMedia.

[115]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[116]  W R Hersh,et al.  How well do physicians use electronic information retrieval systems? A framework for investigation and systematic review. , 1998, JAMA.

[117]  Ricardo da Silva Torres,et al.  RECOD at ImageCLEF 2011: Medical Modality Classification using Genetic Programming , 2011, CLEF.

[118]  Tefko Saracevic,et al.  Evaluation of evaluation in information retrieval , 1995, SIGIR '95.

[119]  Zachary F. Meisel,et al.  Crowdsourcing—Harnessing the Masses to Advance Health and Medicine, a Systematic Review , 2013, Journal of General Internal Medicine.

[120]  Frank Weichert,et al.  Analysis of the Accuracy and Robustness of the Leap Motion Controller , 2013, Sensors.

[121]  D. Frank Hsu,et al.  Comparing Rank and Score Combination Methods for Data Fusion in Information Retrieval , 2005, Information Retrieval.

[122]  Constantino Carlos Reyes-Aldasoro,et al.  Volumetric Texture Description and Discriminant Feature Selection for MRI , 2003, EUROCAST.

[123]  Dina Demner-Fushman,et al.  Application of Information Technology: Essie: A Concept-based Search Engine for Structured Biomedical Text , 2007, J. Am. Medical Informatics Assoc..

[124]  Guido Zuccon,et al.  Why Assessing Relevance in Medical IR is Demanding , 2014, MedIR@SIGIR.

[125]  Alexandr Andoni,et al.  Near-Optimal Hashing Algorithms for Approximate Nearest Neighbor in High Dimensions , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[126]  H Müller,et al.  A Survey on Visual Information Search Behavior and Requirements of Radiologists , 2012, Methods of Information in Medicine.

[127]  James Ze Wang,et al.  Toward Bridging the Annotation-Retrieval Gap in Image Search , 2007, IEEE MultiMedia.

[128]  Martha Larson,et al.  Overview of VideoCLEF 2008: Automatic Generation of Topic-based Feeds for Dual Language Audio-Visual Content , 2008, CLEF.

[129]  Gertjan J. Burghouts,et al.  Performance evaluation of local colour invariants , 2009, Comput. Vis. Image Underst..

[130]  Antoine Geissbühler,et al.  Analyzing Web Log Files of the Health On the Net HONmedia Search Engine to Define Typical Image Search Tasks for Image Retrieval Evaluation , 2007, MedInfo.

[131]  Indriyati Atmosukarto,et al.  Similarity-Based Retrieval for Biomedical Applications , 2008, Case-Based Reasoning on Images and Signals.

[132]  Andreas Thor,et al.  Citation analysis of database publications , 2005, SGMD.

[133]  D. Frank Hsu,et al.  Fusion analysis of information retrieval models on biomedical collections , 2011, 14th International Conference on Information Fusion.

[134]  Henning Müller,et al.  Fusion Techniques for Combining Textual and Visual Information Retrieval , 2010, ImageCLEF.

[135]  Paul Clough,et al.  ImageCLEF: Experimental Evaluation in Visual Information Retrieval , 2010 .

[136]  Offer Drori Improving Display of Search Results in Information Retrieval Systems Users Study , 2001, NDDL.

[137]  Qiang Li,et al.  Fast Filtering Techniques in Medical Image Classification and Retrieval , 2013, CLEF.

[138]  Henning Müller,et al.  Architecture and applications of the Parallel Distributed Image Search Engine (ParaDISE) , 2014, GI-Jahrestagung.

[139]  Mark Sanderson,et al.  The CLEF 2004 Cross-Language Image Retrieval Track , 2004, CLEF.

[140]  Guojun Lu,et al.  Review of shape representation and description techniques , 2004, Pattern Recognit..

[141]  Ellen M. Voorhees,et al.  Bias and the limits of pooling for large collections , 2007, Information Retrieval.

[142]  Shuzhong Shi,et al.  Estimating High Dimensional Covariance Matrices and its Applications , 2011 .

[143]  Winston H. Hsu,et al.  Adaptive Learning for Multimodal Fusion in Video Search , 2009, PCM.

[144]  Henning Müller,et al.  Mobile Medical Visual Information Retrieval , 2012, IEEE Transactions on Information Technology in Biomedicine.

[145]  Daekeun You,et al.  ITI's Participation in the 2013 Medical Track of ImageCLEF , 2013, CLEF.

[146]  Xin Tong,et al.  Component Retrieval Using Conversational Case-Based Reasoning , 2004, Intelligent Information Processing.

[147]  George R. Thoma,et al.  Multimodal biomedical image indexing and retrieval using descriptive text and global feature mapping , 2013, Information Retrieval.

[148]  Mobyen Uddin Ahmed,et al.  Case-Based Reasoning Systems in the Health Sciences: A Survey of Recent Trends and Developments , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[149]  Akio Yamada,et al.  The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[150]  Ivar Jacobson,et al.  Object-oriented software engineering - a use case driven approach , 1993, TOOLS.

[151]  Henning Müller,et al.  Medical image retrieval using bag of meaningful visual words: unsupervised visual vocabulary pruning with PLSA , 2013, MIIRH '13.

[152]  Thomas Mandl,et al.  Analyzing Information Retrieval Results With a Focus on Named Entities , 2008, ROCLING/IJCLCLP.

[153]  Antoine Geissbühler,et al.  Medical Visual Information Retrieval: State of the Art and Challenges Ahead , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[154]  Shan Gao,et al.  Performance evaluation of early and late fusion methods for generic semantics indexing , 2013, Pattern Analysis and Applications.

[155]  Mark Sanderson,et al.  BEST PRACTICES FOR TEST COLLECTION CREATION AND INFORMATION RETRIEVAL SYSTEM EVALUATION , 2010 .

[156]  Ana M. García-Serrano,et al.  Using Visual Concept Features in a Multimodal Retrieval System for the Medical Collection at ImageCLEF2012 , 2012, CLEF.

[157]  L. Böszörményi,et al.  Textual Methods for Medical Case Retrieval , 2014 .

[158]  José Luis Martínez-Fernández,et al.  Combining Textual and Visual Features for Image Retrieval , 2005, CLEF.

[159]  Adeniran Oluwaranti,et al.  Development of an Image Retrieval Model for Biomedical Image Databases , 2011 .

[160]  Daekeun You,et al.  ITI's Participation in the ImageCLEF 2012 Medical Retrieval and Classification Tasks , 2012, CLEF.

[161]  James Ostell The Entrez Search and Retrieval System , 2014 .

[162]  Yi Zhang,et al.  UESTC at ImageCLEF 2012 Medical Tasks , 2012, CLEF.

[163]  Ivar Jacobson,et al.  Object-oriented development in an industrial environment , 1987, OOPSLA '87.

[164]  Shengli Wu,et al.  Linear combination of component results in information retrieval , 2012, Data Knowl. Eng..

[165]  Henning Müller,et al.  The ImageCLEFmed Medical Image Retrieval Task Test Collection , 2009, Journal of Digital Imaging.

[166]  Thomas Martin Deserno,et al.  Overview of the ImageCLEFmed 2007 Medical Retrieval and Medical Annotation Tasks , 2007, CLEF.

[167]  Noriko Kando,et al.  Pooling for a Large-Scale Test Collection: An Analysis of the Search Results from the First NTCIR Workshop , 2004, Information Retrieval.

[168]  Henning Müller,et al.  Comparing fusion techniques for the ImageCLEF 2013 medical case retrieval task , 2015, Comput. Medical Imaging Graph..

[169]  Ellen M. Voorhees,et al.  The fifteenth text retrieval conference TREC 2006 , 2007 .

[170]  Noriko Kando Overview of the Fifth NTCIR Workshop , 2005, NTCIR.

[171]  Donna Harman he Text Retrieval Conferences , 1993, TIPSTER.

[172]  Donna K. Harman,et al.  The TREC Test Collections , 2005 .

[173]  John Collins,et al.  A Comparative Study of Similarity Measures for Content-Based Medical Image Retrieval , 2012, CLEF.

[174]  Henning Müller,et al.  Medical case-based retrieval: integrating query MeSH terms for query-adaptive multi-modal fusion , 2015, Medical Imaging.

[175]  Fabio A. González,et al.  Bioingenium at ImageClefmed 2010: A Latent Semantic Approach , 2010, CLEF.

[176]  Igor Jurisica,et al.  Integration of Case-Based and Image-Based Reasoning , 1998 .

[177]  N. Fuhr PAN-Uncovering Plagiarism , Authorship , and Social Software Misuse ImageCLEF 2013-Cross Language Image Annotation and Retrieval INEX-INitiative for the Evaluation of XML retrieval , 2002 .

[178]  Henning Müllera,et al.  Health care professionals ’ image use and search behaviour , 2006 .

[179]  Albert N. Link,et al.  Economic impact assessment of NIST's text REtrieval conference (TREC) program. Final report , 2010 .

[180]  Zhongmin Shi,et al.  Synonym-Based Expansion and Boosting-Based Re-Ranking: A Two-phase Approach for Genomic Information Retrieval , 2005, TREC.

[181]  Joo-Hwee Lim,et al.  Inter-media Concept-Based Medical Image Indexing and Retrieval with UMLS at IPAL , 2006, CLEF.

[182]  George R. Thoma,et al.  Text and Content-based Approaches to Image Modality Classification and Retrieval for the ImageCLEF 2011 Medical Retrieval Track , 2011, CLEF.

[183]  J. Hirsch An index to quantify an individual's scientific output , 2005 .

[184]  Franz Kainberger,et al.  Khresmoi - Multilingual Semantic Search of Medical Text and Images , 2013, MedInfo.

[185]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[186]  Henning Müller,et al.  Fusion Techniques in Biomedical Information Retrieval , 2014, Fusion in Computer Vision.

[187]  Saïd Radhouani,et al.  Improving Early Precision in the ImageCLEF Medical Retrieval Task , 2010, ImageCLEF.

[188]  Stéphane Marchand-Maillet,et al.  Can feature information interaction help for information fusion in multimedia problems? , 2009, Multimedia Tools and Applications.

[189]  Mark Sanderson,et al.  Seven Years of Image Retrieval Evaluation , 2010, ImageCLEF.

[190]  Alistair Cockburn,et al.  Agile Software Development , 2001 .

[191]  Henning Müller,et al.  Facilitating medical information search using Google Glass connected to a content-based medical image retrieval system , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[192]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[193]  Henning Müller,et al.  Assessing the Scholarly Impact of ImageCLEF , 2011, CLEF.

[194]  Catherine L. Smith,et al.  User adaptation: good results from poor systems , 2008, SIGIR '08.

[195]  Bei Yu,et al.  Crowdsourcing Participatory Evaluation of Medical Pictograms Using Amazon Mechanical Turk , 2013, Journal of medical Internet research.

[196]  Henning Müller,et al.  Overview of the CLEF 2009 Medical Image Retrieval Track , 2009, CLEF.

[197]  Stefano Mizzaro,et al.  Evaluating user interfaces to information retrieval systems: a case study on user support , 1996, SIGIR '96.

[198]  João Magalhães,et al.  Assisted query formulation for multimodal medical case-based retrieval , 2013 .

[199]  Otis Gospodnetic,et al.  Lucene in Action, Second Edition: Covers Apache Lucene 3.0 , 2010 .

[200]  Noriko Kando Overview of the Seventh NTCIR Workshop , 2008, NTCIR.

[201]  Miguel E. Ruiz,et al.  Integrating an automatic classification method into the medical image retrieval process , 2008, AMIA.

[202]  Alan F. Smeaton,et al.  The scholarly impact of TRECVid (2003-2009) , 2011, J. Assoc. Inf. Sci. Technol..

[203]  Henning Müller,et al.  Crowdsourcing for Medical Image Classification , 2014 .

[204]  Noriko Kando,et al.  Preface from NTCIR-10 General Chairs , 2013, NTCIR.

[205]  Henning Müller,et al.  Ground truth generation in medical imaging: a crowdsourcing-based iterative approach , 2012, CrowdMM '12.

[206]  Daniel L. Rubin,et al.  Utilisation de relations ontologiques pour la comparaison d'images décrites par des annotations sémantiques , 2014, EGC.

[207]  Gabriela Csurka,et al.  XRCE's Participation at Medical Image Modality Classification and Ad-hoc Retrieval Tasks of Image CLEF2011 , 2011, CLEF.

[208]  Soo-Hyun Kim,et al.  The effects of high dimensional covariance matrix estimation on asset pricing and generalized least squares , 2010 .

[209]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[210]  Mark Sanderson,et al.  The CLEF Cross Language Image Retrieval Track (ImageCLEF) 2004 , 2004, CLEF.

[211]  Judit Bar-Ilan,et al.  Which h-index? — A comparison of WoS, Scopus and Google Scholar , 2008, Scientometrics.

[212]  Benjamin M. Good,et al.  Crowdsourcing for bioinformatics , 2013, Bioinform..

[213]  Jun Yu Li,et al.  Two Sample Tests for High Dimensional Covariance Matrices , 2012, 1206.0917.

[214]  Henning Müller,et al.  Initial specification of the evaluation tasks "Use cases to bridge validation and benchmarking" PROMISE Deliverable 2.1 , 2011 .

[215]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[216]  Hayit Greenspan,et al.  Content-Based Image Retrieval in Radiology: Current Status and Future Directions , 2010, Journal of Digital Imaging.

[217]  A. F. Rodriguez,et al.  Description and retrieval of medical visual information based on language modelling , 2014 .

[218]  Shamik Sural,et al.  Segmentation and histogram generation using the HSV color space for image retrieval , 2002, Proceedings. International Conference on Image Processing.

[219]  Daniel L Rubin,et al.  Imaging informatics: essential tools for the delivery of imaging services. , 2013, Academic radiology.

[220]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[221]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[222]  Jayashree Kalpathy-Cramer,et al.  Systematic Evaluations and Ground Truth , 2010 .

[223]  Falk Scholer,et al.  User performance versus precision measures for simple search tasks , 2006, SIGIR.

[224]  Christoph U. Lehmann,et al.  A Crowdsourcing Model for Creating Preclinical Medical Education Study Tools , 2013, Academic medicine : journal of the Association of American Medical Colleges.

[225]  Henning Müller,et al.  Information Fusion for Combining Visual and Textual Image Retrieval , 2010, 2010 20th International Conference on Pattern Recognition.

[226]  Koen E. A. van de Sande,et al.  The University of Amsterdam's Concept Detection System at ImageCLEF 2009 , 2009, CLEF.

[227]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[228]  Jeongeun Lee,et al.  SNUMedinfo at ImageCLEF 2013: Medical Retrieval Task , 2013, CLEF.

[229]  Xiannong Meng,et al.  A Study of Color Histogram Based Image Retrieval , 2009, 2009 Sixth International Conference on Information Technology: New Generations.

[230]  Henning Müller,et al.  Bag-of-Colors for Biomedical Document Image Classification , 2012, MCBR-CDS.

[231]  Gwénolé Quellec,et al.  Medical Case Retrieval From a Committee of Decision Trees , 2010, IEEE Transactions on Information Technology in Biomedicine.

[232]  Mark Sanderson,et al.  Evaluating the performance of information retrieval systems using test collections , 2013, Inf. Res..