Comparative Evaluation of Hand-Crafted Image Descriptors vs. Off-the-Shelf CNN-Based Features for Colour Texture Classification under Ideal and Realistic Conditions

Convolutional Neural Networks (CNN) have brought spectacular improvements in several fields of machine vision including object, scene and face recognition. Nonetheless, the impact of this new paradigm on the classification of fine-grained images—such as colour textures—is still controversial. In this work, we evaluate the effectiveness of traditional, hand-crafted descriptors against off-the-shelf CNN-based features for the classification of different types of colour textures under a range of imaging conditions. The study covers 68 image descriptors (35 hand-crafted and 33 CNN-based) and 46 compilations of 23 colour texture datasets divided into 10 experimental conditions. On average, the results indicate a marked superiority of deep networks, particularly with non-stationary textures and in the presence of multiple changes in the acquisition conditions. By contrast, hand-crafted descriptors were better at discriminating stationary textures under steady imaging conditions and proved more robust than CNN-based features to image rotation.

[1]  Amit Jain,et al.  A multiscale representation including opponent color features for texture recognition , 1998, IEEE Trans. Image Process..

[2]  Jana Reinhard,et al.  Textures A Photographic Album For Artists And Designers , 2016 .

[3]  Paul Southam,et al.  Theoretical and experimental comparison of different approaches for color texture classification , 2011, J. Electronic Imaging.

[4]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[5]  Stanislav Kovacic,et al.  Rotation-invariant texture classification , 2003, Pattern Recognit. Lett..

[6]  Maria Petrou,et al.  Image processing - dealing with texture , 2020 .

[7]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Luiz Eduardo Soares de Oliveira,et al.  A database for automatic classification of forest species , 2012, Machine Vision and Applications.

[9]  Christoph Palm,et al.  Color texture classification by integrative Co-occurrence matrices , 2004, Pattern Recognit..

[10]  Paolo Napoletano,et al.  Improved opponent color local binary patterns: an effective local image descriptor for color texture classification , 2017, J. Electronic Imaging.

[11]  Iasonas Kokkinos,et al.  Describing Textures in the Wild , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Dong-Chen He,et al.  Texture Unit, Texture Spectrum, And Texture Analysis , 1990 .

[13]  Stefanos Zafeiriou,et al.  Fine-Grained Material Classification Using Micro-geometry and Reflectance , 2016, ECCV.

[14]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Fernando López-García,et al.  Performance evaluation of soft color texture descriptors for surface grading using experimental design and logistic regression , 2008, Pattern Recognit..

[17]  Marcos X. Álvarez-Cid,et al.  Texture Description Through Histograms of Equivalent Patterns , 2012, Journal of Mathematical Imaging and Vision.

[18]  Francesco Bianconi,et al.  Rotation invariant co-occurrence features based on digital circles and discrete Fourier transform , 2014, Pattern Recognit. Lett..

[19]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[21]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[22]  Iasonas Kokkinos,et al.  Deep Filter Banks for Texture Recognition, Description, and Segmentation , 2015, International Journal of Computer Vision.

[23]  Ko Nishino,et al.  The Scale of Geometric Texture , 2012, ECCV.

[24]  M. Pietikäinen,et al.  TEXTURE ANALYSIS WITH LOCAL BINARY PATTERNS , 2004 .

[25]  Eckehard G. Steinbach,et al.  Multimodal Feature-Based Surface Material Classification , 2017, IEEE Transactions on Haptics.

[26]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[27]  Arivazhagan Selvaraj,et al.  Texture classification using wavelet transform , 2003, Pattern Recognit. Lett..

[28]  Matti Pietikäinen,et al.  Classification with color and texture: jointly or separately? , 2004, Pattern Recognit..

[29]  Matti Pietikäinen,et al.  Accurate color discrimination with classification based on feature distributions , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[30]  Andrew Zisserman,et al.  A Statistical Approach to Texture Classification from Single Images , 2004, International Journal of Computer Vision.

[31]  Francesco Bianconi,et al.  On Comparing Colour Spaces From a Performance Perspective: Application to Automated Classification of Polished Natural Stones , 2015, ICIAP Workshops.

[32]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[33]  Manuel Fernández Delgado,et al.  Influence of normalization and color space to color texture classification , 2017, Pattern Recognit..

[34]  Safia Abdelmounaime,et al.  New Brodatz-Based Image Databases for Grayscale Color and Multiband Texture Analysis , 2013 .

[35]  Paolo Napoletano,et al.  Combining multiple features for color texture classification , 2016, J. Electronic Imaging.

[36]  A. Benassi,et al.  GENERALIZATION OF THE COOCCURRENCE MATRIX FOR COLOUR IMAGES: APPLICATION TO COLOUR TEXTURE CLASSIFICATION , 2011 .

[37]  Paolo Napoletano,et al.  Evaluating color texture descriptors under large variations of controlled lighting conditions , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[38]  André Ricardo Backes,et al.  Color texture analysis based on fractal descriptors , 2012, Pattern Recognit..

[39]  C. Palm,et al.  Classification of color textures by Gabor filtering , 2002 .

[40]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[41]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[42]  Hanqing Lu,et al.  Face detection using improved LBP under Bayesian framework , 2004, Third International Conference on Image and Graphics (ICIG'04).

[43]  Yong Man Ro,et al.  Local Color Vector Binary Patterns From Multichannel Face Images for Face Recognition , 2012, IEEE Transactions on Image Processing.

[44]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Thomas Martinetz,et al.  Deep convolutional neural networks as generic feature extractors , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[46]  Paolo Napoletano,et al.  Combining local binary patterns and local color contrast for texture classification under varying illumination. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[47]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Gertjan J. Burghouts,et al.  Material-specific adaptation of color invariant features , 2009, Pattern Recognit. Lett..

[49]  Alice Porebski,et al.  A new benchmark image test suite for evaluating colour texture classification schemes , 2013, Multimedia Tools and Applications.

[50]  Barbara Caputo,et al.  Class-Specific Material Categorisation , 2005, ICCV.

[51]  Francesco Bianconi,et al.  A Unifying Framework for LBP and Related Methods , 2013, Local Binary Patterns.

[52]  Francesco Bianconi,et al.  Texture Classification Using Rotation Invariant LBP Based on Digital Polygons , 2015, ICIAP Workshops.

[53]  Olivier Alata,et al.  Choice of a pertinent color space for color texture characterization using parametric spectral analysis , 2011, Pattern Recognit..

[54]  Donald A. Adjeroh,et al.  Comparison of Texture Analysis Schemes Under Nonideal Conditions , 2011, IEEE Transactions on Image Processing.

[55]  Francesco Di Maria,et al.  Experimental comparison of color spaces for material classification , 2016, J. Electronic Imaging.

[56]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[57]  Enrico Puppo,et al.  New Trends in Image Analysis and Processing -- ICIAP 2015 Workshops , 2015, Lecture Notes in Computer Science.

[58]  Francesco Bianconi,et al.  An investigation on the use of local multi-resolution patterns for image classification , 2016, Inf. Sci..

[59]  Ig-Jae Kim,et al.  PSI-CNN: A Pyramid-Based Scale-Invariant CNN Architecture for Face Recognition Robust to Various Image Resolutions , 2018, Applied Sciences.

[60]  Anne Humeau-Heurtier,et al.  Texture Feature Extraction Methods: A Survey , 2019, IEEE Access.

[61]  Andrew Zisserman,et al.  A Statistical Approach to Material Classification Using Image Patch Exemplars , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Francesco Bianconi,et al.  Performance analysis of colour descriptors for parquet sorting , 2013, Expert Syst. Appl..

[63]  Jarbas Joaci de Mesquita Sá Junior,et al.  Plant leaf identification using Gabor wavelets , 2009 .

[64]  Nong Sang,et al.  Robust Illumination Invariant Texture Classification Using Gradient Local Binary Patterns , 2011, 2011 International Workshop on Multi-Platform/Multi-Sensor Remote Sensing and Mapping.

[65]  Mario Fritz,et al.  On the Significance of Real-World Conditions for Material Classification , 2004, ECCV.

[66]  Subhransu Maji,et al.  Deep filter banks for texture recognition and segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  Paolo Napoletano,et al.  Hand-Crafted vs Learned Descriptors for Color Texture Classification , 2017, CCIW.

[68]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Matti Pietikäinen,et al.  From BoW to CNN: Two Decades of Texture Representation for Texture Classification , 2018, International Journal of Computer Vision.

[70]  Matti Pietikäinen,et al.  Local binary features for texture classification: Taxonomy and experimental study , 2017, Pattern Recognit..

[71]  Markus Vincze,et al.  Texture Characterization with Semantic Attributes: Database and Algorithm , 2016 .

[72]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[73]  Gustaf Kylberg,et al.  Automatic Virus Identification using TEM : Image Segmentation and Texture Analysis ; Automatisk identifiering av virus med hjälp av transmissionselektronmikroskopi : bildsegmentering och texturanalys , 2014 .

[74]  Matti Pietikäinen,et al.  Outex - new framework for empirical evaluation of texture analysis algorithms , 2002, Object recognition supported by user interaction for service robots.