Comparative study of global color and texture descriptors for web image retrieval

This paper presents a comparative study of color and texture descriptors considering the Web as the environment of use. We take into account the diversity and large-scale aspects of the Web considering a large number of descriptors (24 color and 28 texture descriptors, including both traditional and recently proposed ones). The evaluation is made on two levels: a theoretical analysis in terms of algorithms complexities and an experimental comparison considering efficiency and effectiveness aspects. The experimental comparison contrasts the performances of the descriptors in small-scale datasets and in a large heterogeneous database containing more than 230 thousand images. Although there is a significant correlation between descriptors performances in the two settings, there are notable deviations, which must be taken into account when selecting the descriptors for large-scale tasks. An analysis of the correlation is provided for the best descriptors, which hints at the best opportunities of their use in combination.

[1]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Lai-Man Po,et al.  A Compact and Efficient Color Descriptor for Image Retrieval , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[3]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[4]  Stefan M. Rüger,et al.  Evaluation of Texture Features for Content-Based Image Retrieval , 2004, CIVR.

[5]  Quan Liu,et al.  An Orientation Independent Texture Descriptor for Image Retrieval , 2007, 2007 International Conference on Communications, Circuits and Systems.

[6]  Jun Liu,et al.  Maximum entropy random fields for texture analysis , 2002, Pattern Recognit. Lett..

[7]  Mario A. Nascimento,et al.  A compact and efficient image retrieval approach based on border/interior pixel classification , 2002, CIKM '02.

[8]  Dong-Gyu Sim,et al.  Fast texture description and retrieval of DCT-based compressed images , 2001 .

[9]  Moncef Gabbouj,et al.  A Generic Shape/Texture Descriptor Over Multiscale Edge Field: 2-D Walking Ant Histogram , 2008, IEEE Transactions on Image Processing.

[10]  Ricardo da Silva Torres,et al.  Combining Global with Local Texture Information for Image Retrieval Applications , 2008, 2008 Tenth IEEE International Symposium on Multimedia.

[11]  Ricardo da Silva Torres,et al.  Eva: an evaluation tool for comparing descriptors in content-based image retrieval tasks , 2010, MIR '10.

[12]  Georgios S. Paschos,et al.  Image Content-Based Retrieval Using Chromaticity Moments , 2003, IEEE Trans. Knowl. Data Eng..

[13]  Carlo Tomasi,et al.  Texture-based image retrieval without segmentation , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  Wei-Han Chang,et al.  A fast MPEG-7 dominant color extraction with new similarity measure for image retrieval , 2008, J. Vis. Commun. Image Represent..

[15]  U.A. Ahmad,et al.  Texture features based on Fourier transform and Gabor filters: an empirical comparison , 2007, 2007 International Conference on Machine Vision.

[16]  Zheru Chi,et al.  B-spline over-complete wavelet based fractal signature analysis for texture image retrieval , 2004, Proceedings of 2004 International Symposium on Intelligent Multimedia, Video and Speech Processing, 2004..

[17]  Bo Tao,et al.  Texture Recognition and Image Retrieval Using Gradient Indexing , 2000, J. Vis. Commun. Image Represent..

[18]  Dong-Gyu Sim,et al.  Invariant texture retrieval using modified Zernike moments , 2004, Image Vis. Comput..

[19]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[20]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Guojun Lu,et al.  Review of shape representation and description techniques , 2004, Pattern Recognit..

[22]  Neucimar J. Leite,et al.  Wavelet-based fingerprint image retrieval , 2009 .

[23]  Mario A. Nascimento,et al.  An adaptive and efficient clustering-based approach for content-based image retrieval in image databases , 2001, Proceedings 2001 International Database Engineering and Applications Symposium.

[24]  S. K. Dai,et al.  Texture image retrieval and image segmentation using composite sub-band gradient vectors , 2006, J. Vis. Commun. Image Represent..

[25]  Anil K. Jain,et al.  Texture Analysis , 2018, Handbook of Image Processing and Computer Vision.

[26]  Pranam Janney,et al.  Invariant Features of Local Textures—a rotation invariant local texture , 2008 .

[27]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[28]  Jiangtao Cui,et al.  Image retrieval based on color distribution entropy , 2006, Pattern Recognit. Lett..

[29]  Mario A. Nascimento,et al.  Cell Histograms Versus Color Histograms for Image Representation and Retrieval , 2003, Knowledge and Information Systems.

[30]  Xuelong Li,et al.  Image retrieval based on perceptive weighted color blocks , 2003, Pattern Recognit. Lett..

[31]  Empirical evaluation of MPEG-7 XM color descriptors in content-based retrieval of semantic image categories , 2002, Object recognition supported by user interaction for service robots.

[32]  K. Nallaperumal,et al.  Content Based Image Indexing and Retrieval Using Color Descriptor in Wavelet Domain , 2007, International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 2007).

[33]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[35]  Laurent Amsaleg,et al.  Scalability of local image descriptors: a comparative study , 2006, MM '06.

[36]  Guojun Lu,et al.  A Comparative Study of Fourier Descriptors for Shape Representation and Retrieval , 2002 .

[37]  Adam Williams,et al.  Content-based image retrieval using joint correlograms , 2007, Multimedia Tools and Applications.

[38]  J.-P. Renno,et al.  Evaluation of MPEG7 color descriptors for visual surveillance retrieval , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[39]  Markus A. Stricker,et al.  Similarity of color images , 1995, Electronic Imaging.

[40]  A. K. Ray,et al.  Fuzzy measures for color image retrieval , 2005, Fuzzy Sets Syst..

[41]  Hamid Abrishami Moghaddam,et al.  Wavelet correlogram: A new approach for image indexing and retrieval , 2005, Pattern Recognit..

[42]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[43]  Claudio Gutierrez,et al.  Survey of graph database models , 2008, CSUR.

[44]  Neucimar J. Leite,et al.  Rotation-Invariant and Scale-Invariant Steerable Pyramid Decomposition for Texture Image Retrieval , 2007 .

[45]  Hermann Ney,et al.  Features for image retrieval: an experimental comparison , 2008, Information Retrieval.

[46]  Guojun Lu,et al.  Evaluation of MPEG-7 shape descriptors against other shape descriptors , 2003, Multimedia Systems.

[47]  Ricardo da Silva Torres,et al.  Content-Based Image Retrieval: Theory and Applications , 2006, RITA.

[48]  Matti Pietikäinen,et al.  Block-Based Methods for Image Retrieval Using Local Binary Patterns , 2005, SCIA.

[49]  Ricardo da Silva Torres,et al.  Color Descriptors for Web Image Retrieval: A Comparative Study , 2008, 2008 XXI Brazilian Symposium on Computer Graphics and Image Processing.

[50]  Fatos T. Yarman-Vural,et al.  SASI: a new texture descriptor for content based image retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[51]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[52]  Sven Loncaric,et al.  A survey of shape analysis techniques , 1998, Pattern Recognit..

[53]  A. Khawne,et al.  Color descriptor for image retrieval in wavelet domain , 2006, 2006 8th International Conference Advanced Communication Technology.

[54]  Yeong-Ho Ha,et al.  Spatial color descriptor for image retrieval and video segmentation , 2003, IEEE Trans. Multim..

[55]  Prabir Kumar Biswas,et al.  Cosine-modulated wavelet based texture features for content-based image retrieval , 2004, Pattern Recognit. Lett..

[56]  Kai-Kuang Ma,et al.  Rotation-invariant and scale-invariant Gabor features for texture image retrieval , 2007, Image Vis. Comput..

[57]  Djemel Ziou,et al.  Image Retrieval from the World Wide Web: Issues, Techniques, and Systems , 2004, CSUR.

[58]  David Zhang,et al.  Scale-orientation histogram for texture image retrieval , 2003, Pattern Recognit..

[59]  Fatos T. Yarman-Vural,et al.  SASI: a generic texture descriptor for image retrieval , 2003, Pattern Recognit..

[60]  L. KherfiM.,et al.  Image Retrieval from the World Wide Web , 2004 .

[61]  Jun Ma,et al.  Rotation Invariant Image Classification Based on MPEG-7 Homogeneous Texture Descriptor , 2007 .

[62]  Fillia Makedon,et al.  R-Histogram: quantitative representation of spatial relations for similarity-based image retrieval , 2003, MULTIMEDIA '03.

[63]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[64]  Cyrus Shahabi,et al.  Image retrieval by shape: a comparative study , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[65]  Stephan Volmer,et al.  Color co-occurrence descriptors for querying-by-example , 1998, Proceedings 1998 MultiMedia Modeling. MMM'98 (Cat. No.98EX200).

[66]  Michael Unser,et al.  Sum and Difference Histograms for Texture Classification , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Feng Xu,et al.  Evaluation and comparison of texture descriptors proposed in MPEG-7 , 2006, J. Vis. Commun. Image Represent..

[68]  Shree K. Nayar,et al.  Multiresolution histograms and their use for recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  Ling-Hwei Chen,et al.  An efficient computation method for the texture browsing descriptor of MPEG-7 , 2005, Image Vis. Comput..

[71]  Marta Rukoz,et al.  Estimating the indexability of multimedia descriptors for similarity searching , 2010, RIAO.

[72]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[73]  Ricardo da Silva Torres,et al.  Evaluating Retrieval Effectiveness of Descriptors for Searching in Large Image Databases , 2011, J. Inf. Data Manag..

[74]  Alberto Del Bimbo,et al.  Visual information retrieval , 1999 .

[75]  B. S. Manjunath,et al.  An efficient color representation for image retrieval , 2001, IEEE Trans. Image Process..

[76]  B. S. Manjunath,et al.  A texture descriptor for browsing and similarity retrieval , 2000, Signal Process. Image Commun..

[77]  Chin-Chen Chang,et al.  Color image retrieval technique based on color features and image bitmap , 2007, Inf. Process. Manag..

[78]  Ramin Zabih,et al.  Comparing images using color coherence vectors , 1997, MULTIMEDIA '96.

[79]  Yong Man Ro,et al.  Hierarchical rotational invariant similarity measurement for MPEG-7 homogeneous texture descriptor , 2000 .