A Novel Discriminating and Relative Global Spatial Image Representation with Applications in CBIR

The requirement for effective image search, which motivates the use of Content-Based Image Retrieval (CBIR) and the search of similar multimedia contents on the basis of user query, remains an open research problem for computer vision applications. The application domains for Bag of Visual Words (BoVW) based image representations are object recognition, image classification and content-based image analysis. Interest point detectors are quantized in the feature space and the final histogram or image signature do not retain any detail about co-occurrences of features in the 2D image space. This spatial information is crucial, as it adversely affects the performance of an image classification-based model. The most notable contribution in this context is Spatial Pyramid Matching (SPM), which captures the absolute spatial distribution of visual words. However, SPM is sensitive to image transformations such as rotation, flipping and translation. When images are not well-aligned, SPM may lose its discriminative power. This paper introduces a novel approach to encoding the relative spatial information for histogram-based representation of the BoVW model. This is established by computing the global geometric relationship between pairs of identical visual words with respect to the centroid of an image. The proposed research is evaluated by using five different datasets. Comprehensive experiments demonstrate the robustness of the proposed image representation as compared to the state-of-the-art methods in terms of precision and recall values.

[1]  Baochang Zhang,et al.  Spatial Weighting for Bag-of-Features Based Image Retrieval , 2013, IUKM.

[2]  Gang Hua,et al.  Integrated feature selection and higher-order spatial feature extraction for object categorization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Awais Ahmad,et al.  Intelligent image classification-based on spatial weighted histograms of concentric circles , 2018, Comput. Sci. Inf. Syst..

[4]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[5]  James Ze Wang,et al.  Real-Time Computerized Annotation of Pictures , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Hermann Ney,et al.  Features for image retrieval: an experimental comparison , 2008, Information Retrieval.

[7]  Haibing Wang,et al.  Image retrieval using spatiograms of colors quantized by Gaussian Mixture Models , 2016, Neurocomputing.

[8]  Zahid Mehmood,et al.  A Novel Image Retrieval Based on a Combination of Local and Global Histograms of Visual Words , 2016 .

[9]  A. S. Zadgaonkar,et al.  Image Classification Using Fusion of Holistic Visual Descriptions , 2016 .

[10]  Zahid Mehmood,et al.  Image retrieval by addition of spatial information based on histograms of triangular regions , 2016, Comput. Electr. Eng..

[11]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[12]  Li Li,et al.  Exploiting global and local features for image retrieval , 2018, Journal of Central South University.

[13]  Mihai Datcu,et al.  Exploiting Deep Features for Remote Sensing Image Retrieval: A Systematic Investigation , 2017, IEEE Transactions on Big Data.

[14]  Martin Kampel,et al.  Ancient Coin Classification Using Reverse Motif Recognition: Image-based classification of Roman Republican coins , 2015, IEEE Signal Processing Magazine.

[15]  Jun Wu,et al.  Perceptual uniform descriptor and ranking on manifold for image retrieval , 2018, Inf. Sci..

[16]  Xiaodong Liu,et al.  Image retrieval based on effective feature extraction and diffusion process , 2018, Multimedia Tools and Applications.

[17]  Rehan Ashraf,et al.  Image classification by addition of spatial information based on histograms of orthogonal vectors , 2018, PloS one.

[18]  Abbes Amira,et al.  Semantic content-based image retrieval: A comprehensive study , 2015, J. Vis. Commun. Image Represent..

[19]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[20]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[21]  Bushra Zafar,et al.  A Hybrid Geometric Spatial Image Representation for scene classification , 2018, PloS one.

[22]  Aun Irtaza,et al.  Fusion of local and global features for effective image extraction , 2017, Applied Intelligence.

[23]  Zhigang Zhu,et al.  Real-time indoor assistive localization with mobile omnidirectional vision and cloud GPU acceleration , 2017 .

[24]  Haohui Yin Scene classification using spatial pyramid matching and hierarchical Dirichlet processes , 2010 .

[25]  A. K. Pal,et al.  Image retrieval based on non-uniform bins of color histogram and dual tree complex wavelet transform , 2017, Multimedia Tools and Applications.

[26]  Jana Kosecka,et al.  Deep Convolutional Features for Image Based Retrieval and Scene Categorization , 2015, ArXiv.

[27]  Jun Wu,et al.  Image retrieval framework based on texton uniform descriptor and modified manifold ranking , 2017, J. Vis. Commun. Image Represent..

[28]  Rehan Ashraf,et al.  Content-Based Image Retrieval Based on Late Fusion of Binary and Local Descriptors , 2017, ArXiv.

[29]  Zhi-Hua Zhou,et al.  On the relation between multi-instance learning and semi-supervised learning , 2007, ICML '07.

[30]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Thomas Hofmann,et al.  Support Vector Machines for Multiple-Instance Learning , 2002, NIPS.

[32]  Ekta Walia,et al.  Boosting local texture descriptors with Log-Gabor filters response for improved image retrieval , 2016, International Journal of Multimedia Information Retrieval.

[33]  Ali Javed,et al.  An Ensemble Based Evolutionary Approach to the Class Imbalance Problem with Applications in CBIR , 2018 .

[34]  Xi Zhang,et al.  Feature integration analysis of bag-of-features model for image retrieval , 2013, Neurocomputing.

[35]  Bruce A. Draper,et al.  Introduction to the Bag of Features Paradigm for Image Classification and Retrieval , 2011, ArXiv.

[36]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[37]  Cécile Barat,et al.  Spatial orientations of visual word pairs to improve Bag-of-Visual-Words model , 2012, BMVC.

[38]  Qi Tian,et al.  Spatial pooling for transformation invariant image representation , 2011, MM '11.

[39]  Savvas A. Chatzichristofis,et al.  CoMo: A Compact Composite Moment-Based Descriptor for Image Retrieval , 2017, CBMI.

[40]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[41]  Takumi Kobayashi,et al.  Acoustic Scene Classification Using Efficient Summary Statistics and Multiple Spectro-Temporal Descriptor Fusion , 2018, Applied Sciences.

[42]  Guojun Lu,et al.  Rotation Invariant Spatial Pyramid Matching for Image Classification , 2015, 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[43]  Giorgio Giacinto,et al.  Information fusion in content based image retrieval: A comprehensive overview , 2017, Inf. Fusion.

[44]  Rehan Ashraf,et al.  Content based image retrieval system by using HSV color histogram, discrete wavelet transform and edge histogram descriptor , 2018, 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET).

[45]  Lei Zhu,et al.  Unsupervised Visual Hashing with Semantic Assistant for Content-Based Image Retrieval , 2017, IEEE Transactions on Knowledge and Data Engineering.

[46]  Tong Liu,et al.  A pooled Object Bank descriptor for image scene classification , 2018, Expert Syst. Appl..

[47]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[48]  S. Geva,et al.  Content-Based Image (object) Retrieval with Rotational Invariant Bag-of-Visual Words representation , 2015, 2015 IEEE 10th International Conference on Industrial and Information Systems (ICIIS).

[49]  Tinne Tuytelaars,et al.  Dense interest points , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[50]  David Filliat,et al.  A visual bag of words method for interactive qualitative localization and mapping , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[51]  Zhuang Miao,et al.  Adding spatial distribution clue to aggregated vector in image retrieval , 2018, EURASIP Journal on Image and Video Processing.

[52]  Martin Kampel,et al.  Encoding Spatial Arrangements of Visual Words for Rotation-Invariant Image Classification , 2014, GCPR.

[53]  Tong Zhang,et al.  Deep Learning Based Feature Selection for Remote Sensing Scene Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[54]  Savvas A. Chatzichristofis,et al.  A Novel Image Retrieval Based on Visual Words Integration of SIFT and SURF , 2016, PloS one.