Query-by-visual-search: multimodal framework for content-based image retrieval

Content-based image retrieval (CBIR) states the procedure of recovering images having similar visual content against a query image from image datasets. In CBIR, the selection of redundant and irrelevant features from images results in the semantic gap issue, which occurs during feature representation and machine learning process. The robust image representation for effective and efficient image retrieval mainly depends upon robust feature selection and classification, which also reduces the semantic gap problem of CBIR. This paper proposed an innovative method for effective and efficient CBIR. The method uses sparse complementary features for vigorous image representation, optimal feature selection based on locality-preserving projection, fuzzy c-means clustering, and soft label support vector machine for robust image classification. In CBIR, smaller and larger sizes of codebook improve the recall and precision (accuracy) of the system, respectively. Due to this reason, the proposed method introduces complementary features based on a larger size codebook, which is assembled using two small sizes of codebooks to increase CBIR performance. The three well-known image datasets (i.e. Corel-1000, Corel-1500, and Holidays) are used to assess the performance of the proposed method. The experimental evaluation highlights promising results as compared to recent methods of CBIR.

[1]  Patrick Gros,et al.  Asymmetric hamming embedding: taking the best of our bits for large scale image search , 2011, ACM Multimedia.

[2]  Prashant Srivastava,et al.  Integration of wavelet transform, Local Binary Patterns and moments for content-based image retrieval , 2017, J. Vis. Commun. Image Represent..

[3]  Xiaodong Liu,et al.  Image retrieval based on effective feature extraction and diffusion process , 2018, Multimedia Tools and Applications.

[4]  Paul L. Rosin.  Measuring Corner Properties , 1999, Comput. Vis. Image Underst..

[5]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[7]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[8]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.

[9]  Zahid Mehmood,et al.  A novel method for content-based image retrieval to improve the effectiveness of the bag-of-words model using a support vector machine , 2018, J. Inf. Sci..

[10]  Sung Wook Baik,et al.  Integrating salient colors with rotational invariant texture features for image representation in retrieval systems , 2017, Multimedia Tools and Applications.

[11]  D. Surendran,et al.  The efficient fast-response content-based image retrieval using spark and MapReduce model framework , 2020, Journal of Ambient Intelligence and Humanized Computing.

[12]  Abbes Amira,et al.  Improving content-based image retrieval with compact global and local multi-features , 2016, International Journal of Multimedia Information Retrieval.

[13]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[14]  Xiaodong Liu,et al.  A new fusion approach for content based image retrieval with color histogram and local directional pattern , 2016, International Journal of Machine Learning and Cybernetics.

[15]  Prashant Srivastava,et al.  Content-Based Image Retrieval using Scale Invariant Feature Transform and moments , 2016, 2016 IEEE Uttar Pradesh Section International Conference on Electrical, Computer and Electronics Engineering (UPCON).

[16]  Munish Kumar,et al.  Content-based image retrieval system using ORB and SIFT features , 2018, Neural Computing and Applications.

[17]  Chokri Ben Amar,et al.  A hybrid approach for Content-Based Image Retrieval based on Fast Beta Wavelet network and fuzzy decision support system , 2016, Machine Vision and Applications.

[18]  Gwangil Jeon,et al.  Content Based Image Retrieval by Using Color Descriptor and Discrete Wavelet Transform , 2018, Journal of Medical Systems.

[19]  T. Sree Sharmila,et al.  An efficient framework for image retrieval using color, texture and edge features , 2017, Comput. Electr. Eng..

[20]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[21]  Zahid Mehmood,et al.  Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine , 2017, Applied Intelligence.

[22]  Zahid Mehmood,et al.  A novel image retrieval based on rectangular spatial histograms of visual words , 2018 .

[23]  Zahid Mehmood,et al.  Image retrieval by addition of spatial information based on histograms of triangular regions , 2016, Comput. Electr. Eng..

[24]  Nam-Chul Kim,et al.  Content-based image retrieval using combined texture and color features based on multi-resolution multi-direction filtering and color autocorrelogram , 2019, Journal of Ambient Intelligence and Humanized Computing.

[25]  Shan-Shan Zhang,et al.  Improving image retrieval by integrating shape and texture features , 2018, Multimedia Tools and Applications.

[26]  Yadong Wang,et al.  Improving fuzzy c-means clustering based on feature-weight learning , 2004, Pattern Recognit. Lett..

[27]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[28]  S. Suganya,et al.  Content based image retrieval system based on semantic information using color, texture and shape features , 2016, 2016 International Conference on Computing Technologies and Intelligent Data Engineering (ICCTIDE'16).

[29]  Tom Drummond,et al.  Faster and Better: A Machine Learning Approach to Corner Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Matthieu Cord,et al.  An application of swarm intelligence to distributed image retrieval , 2012, Inf. Sci..

[31]  Xi Zhang,et al.  Feature integration analysis of bag-of-features model for image retrieval , 2013, Neurocomputing.

[32]  Zahid Mehmood,et al.  Scene analysis and search using local features and support vector machine for effective content-based image retrieval , 2018, Artificial Intelligence Review.

[33]  Licheng Jiao,et al.  Feature integration of EODH and Color-SIFT: Application to image retrieval based on codebook , 2014, Signal Process. Image Commun..

[34]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[35]  Hassan Farsi,et al.  Content-based image retrieval by combining convolutional neural networks and sparse representation , 2019, Multimedia Tools and Applications.

[36]  Haibing Wang,et al.  Image retrieval using spatiograms of colors quantized by Gaussian Mixture Models , 2016, Neurocomputing.

[37]  Zahid Mehmood,et al.  Scene search based on the adapted triangular regions and soft clustering to improve the effectiveness of the visual-bag-of-words model , 2018, EURASIP J. Image Video Process..

[38]  Shichao Kan,et al.  Compressed sensing based feature fusion for image retrieval , 2018, Journal of Ambient Intelligence and Humanized Computing.

[39]  Zahid Mehmood,et al.  Effect of complementary visual words versus complementary features on clustering for effective content-based image search , 2018, J. Intell. Fuzzy Syst..

[40]  Zahid Mehmood,et al.  A Novel Image Retrieval Based on a Combination of Local and Global Histograms of Visual Words , 2016 .

[41]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Zahid Mehmood,et al.  Boosting the Performance of the BoVW Model Using SURF–CoHOG-Based Sparse Features with Relevance Feedback for CBIR , 2020, Iranian Journal of Science and Technology, Transactions of Electrical Engineering.

[44]  Wu Hao,et al.  Optimized CNN Based Image Recognition Through Target Region Selection , 2018 .

[45]  Abdolreza Rashno,et al.  An efficient content-based image retrieval with ant colony optimization feature selection schema based on wavelet and color features , 2015, 2015 The International Symposium on Artificial Intelligence and Signal Processing (AISP).

[46]  Xiangyang Wang,et al.  Content-based image retrieval by integrating color and texture features , 2012, Multimedia Tools and Applications.

[47]  Min Huang,et al.  Content-based image retrieval technology using multi-feature fusion , 2015 .

[48]  Gholam Ali Montazer,et al.  A new image feature descriptor for content based image retrieval using scale invariant feature transform and local derivative pattern , 2017 .

[49]  Myo-Taeg Lim,et al.  MDGHM-SURF: A robust local image descriptor based on modified discrete Gaussian-Hermite moment , 2015, Pattern Recognit..

[50]  Konstantinos N. Plataniotis,et al.  Distance measures for color image retrieval , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[51]  Yonghuai Liu,et al.  Human consistency evaluation of static video summaries , 2018, Multimedia Tools and Applications.

[52]  Chandan Singh,et al.  A fast and efficient image retrieval system based on color and texture features , 2016, J. Vis. Commun. Image Represent..