Multi-modal Relevance Feedback for Medical Image Retrieval

Medical image retrieval can assist physicians in nding information supporting their diagnosis. Systems that allow searching for medical images need to provide tools for quick and easy navigation and query renement as the time for information search is often short. Relevance feedback is a powerful tool in information retrieval. This study evaluates relevance feedback techniques with regard to the content they use. A novel relevance feedback technique that uses both text and visual information of the results is proposed. Results show the potential of relevance feedback techniques in medical image retrieval and the superiority of the proposed algorithm over commonly used approaches. Future steps include integrating semantics into relevance feedback techniques to benet of the structured knowledge of ontologies and experimenting on the fusion of text and visual information.

[1]  Mounia Lalmas,et al.  A survey on the use of relevance feedback for information access systems , 2003, The Knowledge Engineering Review.

[2]  Yue Li,et al.  Mammogram Retrieval: Image Selection Strategy of Relevance Feedback for Locating Similar Lesions , 2011, Int. J. Digit. Libr. Syst..

[3]  Bipin C. Desai,et al.  A Framework for Medical Image Retrieval Using Machine Learning and Statistical Similarity Matching Techniques With Relevance Feedback , 2007, IEEE Transactions on Information Technology in Biomedicine.

[4]  H Müller,et al.  A Survey on Visual Information Search Behavior and Requirements of Radiologists , 2012, Methods of Information in Medicine.

[5]  Henning Müller,et al.  User Tests for Assessing a Medical Image Retrieval System: A Pilot Study , 2013, MedInfo.

[6]  Edward A. Fox,et al.  Combination of Multiple Searches , 1993, TREC.

[7]  Neill W. Campbell,et al.  Iterative refinement by relevance feedback in content-based digital image retrieval , 1998, MULTIMEDIA '98.

[8]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[9]  Thierry Pun,et al.  Strategies for positive and negative relevance feedback in image retrieval , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[10]  Thierry Pun,et al.  Learning from User Behavior in Image Retrieval: Application of Market Basket Analysis , 2004, International Journal of Computer Vision.

[11]  Marco La Cascia,et al.  Image Digestion and Relevance Feedback in the ImageRover WWW Search Engine , 1997 .

[12]  Henning Müller,et al.  Fusion Techniques for Combining Textual and Visual Information Retrieval , 2010, ImageCLEF.

[13]  Gerard Salton,et al.  Improving retrieval performance by relevance feedback , 1997, J. Am. Soc. Inf. Sci..

[14]  Thierry Pun,et al.  Content-based query of image databases: inspirations from text retrieval , 2000, Pattern Recognit. Lett..

[15]  Henning Müller,et al.  Overview of the ImageCLEF 2012 Medical Image Retrieval and Classification Tasks , 2012, CLEF.

[16]  Jing Li,et al.  Relevance Feedback in Content-Based Image Retrieval: A Survey , 2013, Handbook on Neural Information Processing.

[17]  Henning Müller,et al.  Bag-of-Colors for Biomedical Document Image Classification , 2012, MCBR-CDS.

[18]  Henning Müller,et al.  The medGIFT Group in ImageCLEFmed 2013 , 2013, CLEF.

[19]  Thomas S. Huang,et al.  Relevance feedback techniques in interactive content-based image retrieval , 1997, Electronic Imaging.