Color and texture applied to a signature-based bag of visual words method for image retrieval

This article addresses the problem of representation, indexing and retrieval of images through the signature-based bag of visual words (S-BoVW) paradigm, which maps features extracted from image blocks into a set of words without the need of clustering processes. Here, we propose the first ever method based on the S-BoVW paradigm that considers information of texture to generate textual signatures of image blocks. We also propose a strategy that represents image blocks with words which are generated based on both color as well as texture information. The textual representation generated by this strategy allows the application of traditional text retrieval and ranking techniques to compute the similarity between images. We have performed experiments with distinct similarity functions and weighting schemes, comparing the proposed strategy to the well-known cluster-based bag of visual words (C-BoVW) and S-BoVW methods proposed previously. Our results show that the proposed strategy for representing images is a competitive alternative for image retrieval, and overcomes the baselines in many scenarios.

[1]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[2]  Cordelia Schmid,et al.  Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search , 2008, ECCV.

[3]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Peter Wiemer-Hastings,et al.  Latent semantic analysis , 2004, Annu. Rev. Inf. Sci. Technol..

[5]  Cordelia Schmid,et al.  A sparse texture representation using local affine regions , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[8]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[9]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Ricardo da Silva Torres,et al.  Evaluating Retrieval Effectiveness of Descriptors for Searching in Large Image Databases , 2011, J. Inf. Data Manag..

[11]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[13]  Victor S. Lempitsky,et al.  Aggregating Local Deep Features for Image Retrieval , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Ricardo da Silva Torres,et al.  A multimodal query expansion based on genetic programming for visually-oriented e-commerce applications , 2016, Inf. Process. Manag..

[16]  Matthijs Douze,et al.  The Yael Library , 2014, ACM Multimedia.

[17]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Matti Pietikäinen,et al.  Block-Based Methods for Image Retrieval Using Local Binary Patterns , 2005, SCIA.

[19]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[20]  Stephen E. Robertson,et al.  Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval , 1994, SIGIR '94.

[21]  Andrew Zisserman,et al.  Representing shape with a spatial pyramid kernel , 2007, CIVR '07.

[22]  Ricardo da Silva Torres,et al.  A signature-based bag of visual words method for image indexing and search , 2015, Pattern Recognit. Lett..

[23]  Andrew Zisserman,et al.  Image Classification using Random Forests and Ferns , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[24]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[25]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[26]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[27]  Ricardo da Silva Torres,et al.  Sorted dominant local color for searching large and heterogeneous image databases , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[28]  Gang Hua,et al.  Descriptive visual words and visual phrases for image applications , 2009, ACM Multimedia.

[29]  Yiannis S. Boutalis,et al.  Accurate Image Retrieval Based on Compact Composite Descriptors and Relevance Feedback Information , 2010, Int. J. Pattern Recognit. Artif. Intell..