CBIR Using Features Derived by Deep Learning

In a Content-based Image Retrieval (CBIR) System, the task is to retrieve similar images from a large database given a query image. The usual procedure is to extract some useful features from the query image and retrieve images that have a similar set of features. For this purpose, a suitable similarity measure is chosen, and images with high similarity scores are retrieved. Naturally, the choice of these features play a very important role in the success of this system, and high-level features are required to reduce the “semantic gap.” In this article, we propose to use features derived from pre-trained network models from a deep-learning convolution network trained for a large image classification problem. This approach appears to produce vastly superior results for a variety of databases, and it outperforms many contemporary CBIR systems. We analyse the retrieval time of the method and also propose a pre-clustering of the database based on the above-mentioned features, which yields comparable results in a much shorter time in most of the cases.

[1]  Shalini Batra,et al.  An efficient bi-layer content based image retrieval system , 2020, Multimedia Tools and Applications.

[2]  P. G. Krishna Mohan,et al.  Local mean differential excitation pattern for content based image retrieval , 2018, SN Applied Sciences.

[3]  Harald Kosch,et al.  Distributed Multimedia Database Technologies Supported by MPEG-7 and MPEG-21 , 2003 .

[4]  Rehan Ashraf,et al.  Content Based Image Retrieval Using Embedded Neural Networks with Bandletized Regions , 2015, Entropy.

[5]  Gwangil Jeon,et al.  Content Based Image Retrieval by Using Color Descriptor and Discrete Wavelet Transform , 2018, Journal of Medical Systems.

[6]  Abdolreza Rashno,et al.  Content-based image retrieval with color and texture features in neutrosophic domain , 2017, 2017 3rd International Conference on Pattern Recognition and Image Analysis (IPRIA).

[7]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[8]  Patrick Siarry,et al.  Boosting content based image retrieval performance through integration of parametric & nonparametric approaches , 2019, J. Vis. Commun. Image Represent..

[9]  Zahid Mehmood,et al.  A novel method for content-based image retrieval to improve the effectiveness of the bag-of-words model using a support vector machine , 2018, J. Inf. Sci..

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Mr. Yogen Mahesh Lohite,et al.  A novel method for Content Based Image retrieval using Local features and SVM classifier , 2017 .

[12]  Hani Hagras,et al.  Interval Type-2 Beta Fuzzy Near Sets Approach to Content-Based Image Retrieval , 2018, IEEE Transactions on Fuzzy Systems.

[13]  Qi Tian,et al.  Recent Advance in Content-based Image Retrieval: A Literature Survey , 2017, ArXiv.

[14]  Zahid Mehmood,et al.  Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine , 2017, Applied Intelligence.

[15]  Jonathon Shlens,et al.  A Tutorial on Principal Component Analysis , 2014, ArXiv.

[16]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Tae-Sun Choi,et al.  Image indexing by modified color co-occurrence matrix , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[18]  Robert M. Haralick,et al.  Probabilistic vs. geometric similarity measures for image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[19]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  D. Surendran,et al.  The efficient fast-response content-based image retrieval using spark and MapReduce model framework , 2020, Journal of Ambient Intelligence and Humanized Computing.

[21]  Jing Huang,et al.  Color-Spatial Image Indexing and Applications , 1998 .

[22]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[23]  Chunheng Wang,et al.  Unsupervised Part-Based Weighting Aggregation of Deep Convolutional Features for Image Retrieval , 2017, AAAI.

[24]  Timo Ojala,et al.  Semantic image retrieval with hsv correlograms , 2001 .

[25]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[26]  Tanzila Saba,et al.  Convolution, Approximation and Spatial Information Based Object and Color Signatures for Content Based Image Retrieval , 2019, 2019 International Conference on Computer and Information Sciences (ICCIS).

[27]  Gaurav Dhiman,et al.  A novel content-based image retrieval approach for classification using GLCM features and texture fused LBP variants , 2020, Neural Computing and Applications.

[28]  Amita Pal,et al.  Improved Content-Based Image Retrieval via Discriminant Analysis , 2017 .

[29]  Eugenio Culurciello,et al.  An Analysis of Deep Neural Network Models for Practical Applications , 2016, ArXiv.

[30]  Mutasem K. Alsmadi,et al.  Content-Based Image Retrieval Using Color, Shape and Texture Descriptors and Features , 2020, Arabian Journal for Science and Engineering.

[31]  François Chollet,et al.  Deep Learning with Python , 2017 .

[32]  Zahid Mehmood,et al.  A Novel Technique Based on Visual Words Fusion Analysis of Sparse Features for Effective Content-Based Image Retrieval , 2018 .

[33]  Bir Bhanu,et al.  Probabilistic Feature Relevance Learning for Content-Based Image Retrieval , 1999, Comput. Vis. Image Underst..

[34]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[35]  L. Sumalatha,et al.  Content based Image Retrieval Using Multi Motif Co-Occurrence Matrix , 2018 .

[36]  Anand Singh Jalal,et al.  A Visual Saliency-Based Approach for Content-Based Image Retrieval , 2021, Int. J. Cogn. Informatics Nat. Intell..

[37]  Vijay Vasudevan,et al.  Learning Transferable Architectures for Scalable Image Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[38]  Graham W. Taylor,et al.  Adaptive deconvolutional networks for mid and high level feature learning , 2011, 2011 International Conference on Computer Vision.

[39]  Zahid Mehmood,et al.  An effective content-based image retrieval technique for image visuals representation based on the bag-of-visual-words model , 2018, PloS one.

[40]  Khawaja Tehseen Ahmed,et al.  Content based image retrieval using image features information fusion , 2019, Inf. Fusion.

[41]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[42]  Ramakrishnan Kannan,et al.  CBIR System Based On Prediction Errors , 2017, J. Inf. Sci. Eng..

[43]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[44]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[45]  Claus Nebauer,et al.  Evaluation of convolutional neural networks for visual recognition , 1998, IEEE Trans. Neural Networks.

[46]  Zahid Mehmood,et al.  Scene analysis and search using local features and support vector machine for effective content-based image retrieval , 2018, Artificial Intelligence Review.

[47]  Gavriel Salomon,et al.  T RANSFER OF LEARNING , 1992 .