Image classification by combining local and global features

Several techniques have recently been proposed to extract the features of an image. Feature extraction is one of the most important steps in various image processing and computer vision applications such as image retrieval, image classification, matching, object recognition. Relevant feature (global or local) contains discriminating information and is able to distinguish one object from others. Global features describe the entire image, whereas local features describe the image patches (small group of pixels). In this paper, we present a novel descriptor to extract the color-texture features via two information types. Our descriptor named concatenation of local and global color features is based on the fusion of global features using wavelet transform and a modified version of local ternary pattern, whereas, for the local features, speeded-up robust feature descriptor and bag of words model were used. All the features are extracted from the three color planes. To evaluate the effectiveness of our descriptor for image classification, we carried out experiments using the challenging datasets: New-BarkTex, Outex-TC13, Outex-TC14, MIT scene, UIUC sports event, Caltech 101 and MIT indoor scene. Experimental results showed that our descriptor outperforms the existing state-of-the-art methods.

[1]  Qian Du,et al.  Fusing Local and Global Features for High-Resolution Scene Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Audrey Ledoux,et al.  Color local binary patterns: compact descriptors for texture classification , 2016, J. Electronic Imaging.

[3]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[5]  Yang Wang,et al.  MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification , 2016, IEEE Geoscience and Remote Sensing Letters.

[6]  Cécile Barat,et al.  Spatial histograms of soft pairwise similar patches to improve the bag-of-visual-words model , 2015, Comput. Vis. Image Underst..

[7]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[8]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[9]  Shih-Fu Chang,et al.  Automated binary texture feature sets for image retrieval , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[10]  Alain Trémeau,et al.  Texture Classification across Illumination Color Variations , 2013 .

[11]  Chengjun Liu,et al.  New image descriptors based on color, texture, shape, and wavelets for object and scene image classification , 2013, Neurocomputing.

[12]  KABBAI LEILA,et al.  NEW ROBUST DESCRIPTOR FOR IMAGE MATCHING , 2016 .

[13]  Ali Douik,et al.  Texture descriptor based on local combination adaptive ternary pattern , 2015, IET Image Process..

[14]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, CVPR 2004.

[15]  Ali Douik,et al.  Hybrid local and global descriptor enhanced with colour information , 2017, IET Image Process..

[16]  Li Yan,et al.  Improved Class-Specific Codebook with Two-Step Classification for Scene-Level Classification of High Resolution Remote Sensing Images , 2017, Remote. Sens..

[17]  Qian Du,et al.  Object Tracking in Satellite Videos by Fusing the Kernel Correlation Filter and the Three-Frame-Difference Algorithm , 2018, IEEE Geoscience and Remote Sensing Letters.

[18]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[19]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[20]  Alice Porebski,et al.  A new benchmark image test suite for evaluating colour texture classification schemes , 2013, Multimedia Tools and Applications.

[21]  Danni Ai,et al.  Color Independent Components Based SIFT Descriptors for Object/Scene Classification , 2010, IEICE Trans. Inf. Syst..

[22]  John See,et al.  Exploiting textures for better action recognition in low-quality videos , 2017, EURASIP J. Image Video Process..

[23]  Xiaosheng Wu,et al.  Joint-scale LBP: a new feature descriptor for texture classification , 2017, The Visual Computer.

[24]  Shanto Rahman,et al.  DTCTH: a discriminative local pattern descriptor for image classification , 2017, EURASIP Journal on Image and Video Processing.

[25]  Chengjun Liu,et al.  Cross Disciplinary Biometric Systems , 2012, Intelligent Systems Reference Library.

[26]  Johan A. K. Suykens,et al.  LS-SVMlab Toolbox User's Guide , 2010 .

[27]  Vipin Tyagi,et al.  An effective scheme for image texture classification based on binary local structure pattern , 2013, The Visual Computer.

[28]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Chunlei Yang,et al.  Multi-scale counting and difference representation for texture classification , 2017, The Visual Computer.

[30]  Qian Du,et al.  Scene classification using local and global features with collaborative representation fusion , 2016, Inf. Sci..

[31]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[32]  Fei-Fei Li,et al.  What, where and who? Classifying events by scene and object recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[33]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[34]  Xudong Jiang,et al.  LBP-Based Edge-Texture Features for Object Recognition , 2014, IEEE Transactions on Image Processing.

[35]  María Vanrell,et al.  Texton theory revisited: A bag-of-words approach to combine textons , 2012, Pattern Recognit..

[36]  Matti Pietikäinen,et al.  Outex - new framework for empirical evaluation of texture analysis algorithms , 2002, Object recognition supported by user interaction for service robots.

[37]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[38]  Michael G. Strintzis,et al.  Combining Global and Local Information for Knowledge-Assisted Image Analysis and Classification , 2007, EURASIP J. Adv. Signal Process..

[39]  Paolo Napoletano,et al.  Combining local binary patterns and local color contrast for texture classification under varying illumination. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[40]  C. Chandrasekar,et al.  Combining local and global feature for object recognition using SVM-KNN , 2012, International Conference on Pattern Recognition, Informatics and Medical Engineering (PRIME-2012).

[41]  Ali Douik,et al.  Robust color texture descriptor for material recognition , 2016, Pattern Recognit. Lett..

[42]  Moncef Gabbouj,et al.  An Ordinal Co-occurrence Matrix Framework for Texture Retrieval , 2007, EURASIP J. Image Video Process..

[43]  Carlo Gatta,et al.  Unsupervised Deep Feature Extraction for Remote Sensing Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[45]  Mohamed Abdou Berbar.  Three robust features extraction approaches for facial gender classification , 2013, The Visual Computer.

[46]  Manoj D. Chaudhary,et al.  Fusion of local and global features using Stationary Wavelet Transform for efficient Content Based Image Retrieval , 2014, 2014 IEEE Students' Conference on Electrical, Electronics and Computer Science.

[47]  Jianjun Hou,et al.  Learning two-pathway convolutional neural networks for categorizing scene images , 2017, Multimedia Tools and Applications.

[48]  Ali Douik,et al.  Content based image retrieval using local and global features descriptor , 2016, 2016 2nd International Conference on Advanced Technologies for Signal and Image Processing (ATSIP).

[49]  Hao Su,et al.  Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification , 2010, NIPS.

[50]  Subhransu Maji,et al.  Efficient Classification for Additive Kernel SVMs , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[52]  Ying Jiang,et al.  Discussion on Remote Sensing Big Data to Promote the Development of Smart City , 2020 .

[53]  Liming Chen,et al.  Image region description using orthogonal combination of local binary patterns enhanced with color information , 2013, Pattern Recognit..

[54]  Jun Wu,et al.  Fusion framework for color image retrieval based on bag-of-words model and color local Haar binary patterns , 2016, J. Electronic Imaging.

[55]  Yan Ding,et al.  Image matching with an improved descriptor based on SIFT , 2017, International Conference on Electronics and Information Engineering.

[56]  Xi Zhang,et al.  Feature integration analysis of bag-of-features model for image retrieval , 2013, Neurocomputing.

[57]  Chengjun Liu,et al.  New color GPHOG descriptors for object and scene image classification , 2014, Machine Vision and Applications.

[58]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[59]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[60]  Dieter Fox,et al.  Hierarchical Matching Pursuit for Image Classification: Architecture and Fast Algorithms , 2011, NIPS.