Implementing Relevance Feedback for Content-Based Medical Image Retrieval

Content-based image medical retrieval (CBMIR) is a technique for retrieving medical images on the basis of automatically derived image features such as colour, texture and shape. There are many applications of CBMIR, such as teaching, research, diagnosis and electronic patient records. The retrieval performance of a CBMIR system depends mainly on the representation of image features, which researchers have studied extensively for decades. Although a number of methods and approaches have been suggested, it remains one of the most challenging problems in current (CBMIR) studies, largely due to the well-known “semantic gap” issue that exists between machine-captured low-level image features and human-perceived high-level semantic concepts. There have been many techniques proposed to bridge this gap. This study proposes a novel relevance feedback retrieval method (RFRM) for CBMIR. The feedback implemented here is based on voting values performed by each class in the image repository. Here, eighteen using colour moments and GLCM texture features were extracted to represent each image and eight common similarity coefficients were used as similarity measures. After briefly researching using a single random image query, the top images retrieved from each class are used as voters to select the most effective similarity coefficient that will be used for the final searching process. Our proposed method is implemented on the Kvasir dataset, which has 4,000 images divided into eight classes and was recently widely used for gastrointestinal disease detection. Intensive statistical analysis of the results shows that our proposed RFRM method has the best performance for enhancing both recall and precision when it uses any group of similarity coefficients.

[1]  V. K. Govindan,et al.  Content based medical image retrieval using topic and location model , 2019, J. Biomed. Informatics.

[2]  Yimo Tao,et al.  Multi-Level Learning Approaches for Medical Image Understanding and Computer-aided Detection and Diagnosis , 2010 .

[3]  Jianyong Sun,et al.  Using deep learning for content-based medical image retrieval , 2017, Medical Imaging.

[4]  N. J. Leite,et al.  An architecture for content-based retrieval of remote sensing images , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[5]  Payel Ghosh,et al.  Review of medical image retrieval systems and future directions , 2011, 2011 24th International Symposium on Computer-Based Medical Systems (CBMS).

[6]  Abdolraheem Khader Alhassan,et al.  Color and texture fusion-based method for content-based Image Retrieval , 2017, 2017 International Conference on Communication, Control, Computing and Electronics Engineering (ICCCCEE).

[7]  Naomie Salim,et al.  Ligand-Based Virtual Screening Using Bayesian Inference Network and Reweighted Fragments , 2012, TheScientificWorldJournal.

[8]  Teh Ying Wah,et al.  A Comparison Study on Similarity and Dissimilarity Measures in Clustering Continuous Data , 2015, PloS one.

[9]  Naomie Salim,et al.  Condorcet and borda count fusion method for ligand-based virtual screening , 2014, Journal of Cheminformatics.

[10]  Faïez Gargouri,et al.  Vector space model adaptation and pseudo relevance feedback for content-based image retrieval , 2018, Multimedia Tools and Applications.

[11]  Sung Wook Baik,et al.  Medical Image Retrieval with Compact Binary Codes Generated in Frequency Domain Using Highly Reactive Convolutional Features , 2018, Journal of Medical Systems.

[12]  Parmit K. Chilana,et al.  Using content‐based image retrieval of dermoscopic images for interpretation and education: A pilot study , 2019, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[13]  Jie Ma,et al.  Medical Image Retrieval Based on Convolutional Neural Network and Supervised Hashing , 2019, IEEE Access.

[14]  Naomie Salim,et al.  Ligand expansion in ligand-based virtual screening using relevance feedback , 2012, Journal of Computer-Aided Molecular Design.

[15]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Michael Riegler,et al.  KVASIR: A Multi-Class Image Dataset for Computer Aided Gastrointestinal Disease Detection , 2017, MMSys.

[17]  K. Park,et al.  Effective Diagnosis and Treatment through Content-Based Medical Image Retrieval (CBMIR) by Using Artificial Intelligence , 2019, Journal of clinical medicine.

[18]  Mei Chen,et al.  Dictionary pruning with visual word significance for medical image retrieval , 2016, Neurocomputing.

[19]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[20]  H. Kasban,et al.  A robust medical image retrieval system based on wavelet optimization and adaptive block truncation coding , 2019, Multimedia Tools and Applications.

[21]  C. Krishna Mohan,et al.  Content based medical image retrieval using dictionary learning , 2015, Neurocomputing.

[22]  V. K. Govindan,et al.  Content-based medical image retrieval by spatial matching of visual words , 2018, J. King Saud Univ. Comput. Inf. Sci..

[23]  Hamid Hassanpour,et al.  A short-term learning framework based on relevance feedback for content-based image retrieval , 2017, 2017 3rd Iranian Conference on Intelligent Systems and Signal Processing (ICSPIS).

[24]  Naomie Salim,et al.  Implementing Relevance Feedback in Ligand-Based Virtual Screening Using Bayesian Inference Network , 2011, Journal of biomolecular screening.

[25]  Muhammad Awais,et al.  Medical image retrieval using deep convolutional neural network , 2017, Neurocomputing.

[26]  Daniel L. Rubin,et al.  Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: Application to bone tumor radiographs , 2018, J. Biomed. Informatics.

[27]  Sudeesh Chouhan,et al.  Content based image retrieval system techniques: a review and analysis , 2017 .

[28]  Mouna Torjmen Khemakhem,et al.  Document/query expansion based on selecting significant concepts for context based retrieval of medical images , 2019, J. Biomed. Informatics.

[29]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[30]  Murat Erten,et al.  HSV Color Histogram Based Image Retrieval with Background Elimination , 2019, 2019 1st International Informatics and Software Engineering Conference (UBMYK).

[31]  Majid Razmara,et al.  Diagnostic accuracy of content‐based dermatoscopic image retrieval with deep classification features† , 2018, The British journal of dermatology.

[32]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..