Scene Capture and Selected Codebook-Based Refined Fuzzy Classification of Large High-Resolution Images

Scene classification has been successfully applied to the semantic interpretation of large high-resolution images (HRIs). The bag-of-words (BOW) model has been proven to be effective but inadequate for HRIs because of the complex arrangement of the ground objects and the multiple types of land cover. How to define the scenes in HRIs is still a problem for scene classification. The previous methods involve selecting the scenes manually or with a fixed spatial distribution, leading to scenes with a mixture of objects from different categories. In this paper, to address these issues, a scene capture method using adjacent segmented images and a support vector machine classifier is proposed to generate scenes dominated by one category. The codebook in BOW is obtained from clustering features extracted from all the categories, which may lose the discrimination in some vocabularies. Thus, more discriminative visual vocabularies are selected by the introduced mutual information and the proposed intraclass variability balance in each category, to decrease the redundancy of the codebook. In addition, a refined fuzzy classification strategy is presented to avoid misclassification in similar categories. The experimental results obtained with three different types of HRI data sets confirm that the proposed method obtains classification results better than those obtained by most of the previous methods in all the large HRIs, demonstrating that the selection of representative vocabularies, the refined fuzzy classification, and the scene capture strategy are all effective in improving the performance of scene classification.

[1]  Ping Tang,et al.  Land-Use Scene Classification Using a Concentric Circle-Structured Multiscale Bag-of-Visual-Words Model , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[3]  Liangpei Zhang,et al.  Scene Classification Based on the Fully Sparse Semantic Topic Model , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Bei Zhao,et al.  Scene classification via latent Dirichlet allocation using a hybrid generative/discriminative strategy for high spatial resolution remote sensing imagery , 2013 .

[6]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[7]  Nelson H. C. Yung,et al.  Feature fusion within local region using localized maximum-margin learning for scene categorization , 2012, Pattern Recognit..

[8]  M. Pal,et al.  Random forests for land cover classification , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[9]  Peter Emerson,et al.  The original Borda count and partial voting , 2013, Soc. Choice Welf..

[10]  Mohamed Cheriet,et al.  Support Vector Machine , 2009, Encyclopedia of Biometrics.

[11]  Gong Cheng,et al.  Scene classification of high resolution remote sensing images using convolutional neural networks , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[12]  Xiangtao Zheng,et al.  Dimensionality Reduction by Spatial–Spectral Preservation in Selected Bands , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Xiangtao Zheng,et al.  Remote Sensing Scene Classification by Unsupervised Representation Learning , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[15]  Nelson H. C. Yung,et al.  Scene categorization based on local–global feature fusion and multi-scale multi-spatial resolution encoding , 2014, Signal, Image and Video Processing.

[16]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[17]  Chong-Wah Ngo,et al.  Evaluating bag-of-visual-words representations in scene classification , 2007, MIR '07.

[18]  Rahul Dev Garg,et al.  Classification of high resolution satellite images using spatial constraints-based fuzzy clustering , 2014 .

[19]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[20]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[21]  Shan Suthaharan,et al.  Support Vector Machine , 2016 .

[22]  Bo Du,et al.  Scene Classification via a Gradient Boosting Random Convolutional Network Framework , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[24]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[25]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[27]  Qian Du,et al.  Scene classification using local and global features with collaborative representation fusion , 2016, Inf. Sci..

[28]  Andrew Zisserman,et al.  Scene Classification Via pLSA , 2006, ECCV.

[29]  Thomas M. Cover,et al.  Estimation by the nearest neighbor rule , 1968, IEEE Trans. Inf. Theory.

[30]  Jefersson Alex dos Santos,et al.  Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[31]  Gui-Song Xia,et al.  Dirichlet-Derived Multiple Topic Scene Classification Model for High Spatial Resolution Remote Sensing Imagery , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[33]  Maria Scalzo-Cornacchia,et al.  Fusion of Deep Convolutional Neural Networks , 2017, 2017 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[34]  Jianya Gong,et al.  Land-Use Scene Classification in High-Resolution Remote Sensing Images Using Improved Correlatons , 2015, IEEE Geoscience and Remote Sensing Letters.

[35]  Liangpei Zhang,et al.  Scene Classification Based on the Multifeature Fusion Probabilistic Topic Model for High Spatial Resolution Remote Sensing Imagery , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[37]  Bo Du,et al.  Saliency-Guided Unsupervised Feature Learning for Scene Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Li Yan,et al.  Improved Class-Specific Codebook with Two-Step Classification for Scene-Level Classification of High Resolution Remote Sensing Images , 2017, Remote. Sens..

[39]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[41]  Antonio Criminisi,et al.  Object categorization by learned universal visual dictionary , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[42]  Nelson H. C. Yung,et al.  Scene categorization with multiscale category specific visual words , 2009, Electronic Imaging.