Combining Text and Content Based Image Retrieval on Medical Resource Database

An approach of medical image retrieval by combining diagnosis text and image content features is presented in this paper. In this approach, the system firstly extracts fusion features and semantic content from images and related text in medical resource database. In the second stage, a multi-modal query strategy is generated based on the semantic model and medical ontology, and then the mixed retrieval rendering is interactively shown in intelligent user interface. We evaluate our approach on real medical image data collected from the hospital, and this method achieves encouraging results which are basically satisfying the clinical needs.

[1]  T M Lehmann,et al.  Content-based Image Retrieval in Medical Applications , 2004, Methods of Information in Medicine.

[2]  Bruce I. Reiner,et al.  Medical Image Resource Center 2002: An Update on the RSNA's Medical Image Resource Center , 2002, Journal of Digital Imaging.

[3]  Carla E. Brodley,et al.  ASSERT: A Physician-in-the-Loop Content-Based Retrieval System for HRCT Image Databases , 1999, Comput. Vis. Image Underst..

[4]  Shih-Fu Chang,et al.  Tools and techniques for color image retrieval , 1996, Electronic Imaging.

[5]  Jitendra Malik,et al.  Efficient shape matching using shape contexts , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[7]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[8]  Shih-Fu Chang,et al.  VisualSEEk: a fully automated content-based image query system , 1997, MULTIMEDIA '96.

[9]  David S. Channin,et al.  BRISC—An Open Source Pulmonary Nodule Image Retrieval Framework , 2007, Journal of Digital Imaging.

[10]  Jitendra Malik,et al.  Shape contexts enable efficient retrieval of similar shapes , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[11]  Alex Pentland,et al.  Photobook: Content-based manipulation of image databases , 1996, International Journal of Computer Vision.

[12]  Katherine P Andriole,et al.  Informatics in radiology: integration of the medical imaging resource center into a teaching hospital network to allow single sign-on access. , 2009, Radiographics : a review publication of the Radiological Society of North America, Inc.

[13]  Carla E. Brodley,et al.  Unsupervised Feature Selection Applied to Content-Based Retrieval of Lung Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.