Using relevance feedback with short-term memory for content-based spine X-ray image retrieval

Managing large medical image databases has become a challenging task as more medical images are produced and stored in digital format. Computer-aided decision support for content-based image retrieval (CBIR) is an essential tool for medical image management. This paper presents a novel hybrid relevance feedback (RF) system for shape-based retrieval of spine X-ray images. A new shape similarity measure that considers both whole shape and partial shape matching is presented. The proposed RF architecture includes separate retrieval and feedback modes to solicit user's opinion for refining retrieval results. A unique short-term memory approach is implemented to avoid repeated request for user's feedback on the same, already approved, and retrieved relevant images. An automatic weight updating scheme is developed to present the images on which it is best for the user to provide feedback. Incorporating all these unique features, the proposed RF retrieval system is able to reduce the gap between high-level human visual perception and low-level computerized features. Experimental results show overall retrieval accuracy improvement of 22.0% and 17.5% after the second feedback iteration for retrieving spine X-ray images with similar osteophytes severity and type, respectively.

[1]  Shi-Min Hu,et al.  Optimal adaptive learning for image retrieval , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[2]  Djemel Ziou,et al.  Relevance feedback for CBIR: a new approach based on probabilistic feature weighting with positive and negative examples , 2006, IEEE Transactions on Image Processing.

[3]  Bipin C. Desai,et al.  A Framework for Medical Image Retrieval Using Machine Learning and Statistical Similarity Matching Techniques With Relevance Feedback , 2007, IEEE Transactions on Information Technology in Biomedicine.

[4]  B. J. Doherty,et al.  Morphologic Study of Lumbar Vertebral Osteophytes , 1998, Southern medical journal.

[5]  Chih-Yi Chiu,et al.  Learning human perceptual concepts in a fuzzy CBIR system , 2003, Proceedings Fifth International Conference on Computational Intelligence and Multimedia Applications. ICCIMA 2003.

[6]  Hao Wu,et al.  A New Way for Multidimensional Medical Data Management: Volume of Interest (VOI)-Based Retrieval of Medical Images With Visual and Functional Features , 2006, IEEE Transactions on Information Technology in Biomedicine.

[7]  Anthony K. H. Tung,et al.  iSearch: mining retrieval history for content-based image retrieval , 2003, Eighth International Conference on Database Systems for Advanced Applications, 2003. (DASFAA 2003). Proceedings..

[8]  Zhong Jin,et al.  Integrated probability function and its application to content-based image retrieval by relevance feedback , 2003, Pattern Recognit..

[9]  I Macnab,et al.  The traction spur. An indicator of segmental instability. , 1971, The Journal of bone and joint surgery. American volume.

[10]  Shu-Ming Hsieh,et al.  Graph-based representation for similarity retrieval of symbolic images , 2008, Data Knowl. Eng..

[11]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Chih-Yi Chiu,et al.  A fuzzy logic CBIR system , 2003, The 12th IEEE International Conference on Fuzzy Systems, 2003. FUZZ '03..

[13]  Sameer Antani,et al.  Evaluating Partial Shape Queries for Pathology-based Retrieval of Vertebra , 2004 .

[14]  James S. Duncan,et al.  Synthesis of Research: Medical Image Databases: A Content-based Retrieval Approach , 1997, J. Am. Medical Informatics Assoc..

[15]  Irwin King,et al.  Relevance feedback based on parameter estimation of target distribution , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[16]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[17]  L. Rodney Long,et al.  Evaluation of shape similarity measurement methods for spine X-ray images , 2004, J. Vis. Commun. Image Represent..

[18]  Bipin C. Desai,et al.  Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion , 2008, Comput. Medical Imaging Graph..

[19]  Hichem Frigui,et al.  Interactive image retrieval using fuzzy sets , 2001, Pattern Recognit. Lett..

[20]  Marin Ferecatu,et al.  Interactive Remote-Sensing Image Retrieval Using Active Relevance Feedback , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[21]  D Resnick,et al.  Traction osteophytes of the lumbar spine: radiographic-pathologic correlation. , 1988, Radiology.

[22]  P. Muneesawang,et al.  A neural network approach for learning image similarity in adaptive CBIR , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[23]  Arun K. Majumdar,et al.  An Object-Oriented Fuzzy Data Model for Similarity Detection in Image Databases , 2002, IEEE Trans. Knowl. Data Eng..

[24]  Nicu Sebe,et al.  How to complete performance graphs in content-based image retrieval: add generality and normalize scope , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Nam Ik Cho,et al.  Relevance feedback in content-based image retrieval system by selective region growing in the feature space , 2003, Signal Process. Image Commun..

[26]  Dah-Jye Lee,et al.  A Spine X-Ray Image Retrieval System Using Partial Shape Matching , 2008, IEEE Transactions on Information Technology in Biomedicine.

[27]  Hayit Greenspan,et al.  Medical Image Categorization and Retrieval for PACS Using the GMM-KL Framework , 2007, IEEE Transactions on Information Technology in Biomedicine.

[28]  Arnold W. M. Smeulders,et al.  Concept-based retrieval of biomedical images , 2003, SPIE Medical Imaging.

[29]  Xuelong Li,et al.  Which Components are Important for Interactive Image Searching? , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Horace Ho-Shing Ip,et al.  On relevance feedback and similarity measure for image retrieval with synergetic neural nets , 2003, Neurocomputing.

[31]  Bao-Gang Hu,et al.  Kernel-based similarity learning , 2002, Proceedings. International Conference on Machine Learning and Cybernetics.

[32]  Fabio Roli,et al.  Bayesian relevance feedback for content-based image retrieval , 2004, Pattern Recognit..

[33]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[34]  Wei-Ying Ma,et al.  Learning similarity measure for natural image retrieval with relevance feedback , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[35]  Carla E. Brodley,et al.  Interactive Content-based Image Retrieval Using Relevance Feedback , 2002 .

[36]  Rong Jin,et al.  A unified log-based relevance feedback scheme for image retrieval , 2006 .

[37]  Esther de Ves,et al.  Applying logistic regression to relevance feedback in image retrieval systems , 2007, Pattern Recognit..

[38]  Ingemar J. Cox,et al.  The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments , 2000, IEEE Trans. Image Process..

[39]  Seiji Yamada,et al.  Non-Relevance Feedback Document Retrieval based on One Class SVM and SVDD , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[40]  Rujie Liu,et al.  SVM-based active feedback in image retrieval using clustering and unlabeled data , 2008, Pattern Recognit..

[41]  Masafumi Hagiwara,et al.  An image retrieval system by impression words and specific object names - IRIS , 2002, Neurocomputing.

[42]  Thomas Martin Deserno,et al.  Ontology of Gaps in Content-Based Image Retrieval , 2009, Journal of Digital Imaging.

[43]  W. Liu,et al.  Medical Image Retrieval using Salient Point Detector , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[44]  Chiou-Ting Hsu,et al.  Image retrieval with relevance feedback based on graph-theoretic region correspondence estimation , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[45]  Li Fang,et al.  Image retrieval with relevance feedback , 2000, Proceedings 29th Applied Imagery Pattern Recognition Workshop.

[46]  George R Thoma,et al.  Image informatics at a national research center. , 2005, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[47]  Nikolas P. Galatsanos,et al.  Relevance feedback based on incremental learning for mammogram retrieval , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[48]  Thomas Martin Deserno,et al.  A monohierarchical multiaxial classification code for medical images in content-based retrieval , 2002, Proceedings IEEE International Symposium on Biomedical Imaging.

[49]  Zhongfei Zhang,et al.  Automatic medical image annotation and retrieval , 2008, Neurocomputing.

[50]  Amit Kumar Das,et al.  Image retrieval based on indexing and relevance feedback , 2007, Pattern Recognit. Lett..

[51]  Aleksandra Mojsilovic,et al.  Semantic based categorization, browsing and retrieval in medical image databases , 2002, Proceedings. International Conference on Image Processing.

[52]  Nikolas P. Galatsanos,et al.  A similarity learning approach to content-based image retrieval: application to digital mammography , 2004, IEEE Transactions on Medical Imaging.

[53]  Djemel Ziou,et al.  Combining positive and negative examples in relevance feedback for content-based image retrieval , 2003, J. Vis. Commun. Image Represent..

[54]  Yanchun Zhang,et al.  An overview of content-based image retrieval techniques , 2004, 18th International Conference on Advanced Information Networking and Applications, 2004. AINA 2004..