Scene classification of remote sensing image based on deep network grading transferring

Abstract Aiming at low precision of remote sensing image scene classification, a classification method DCNN_GT based on deep convolutional neural network (DCNN) and grading transfer (GT) is proposed. First, the DCNN model pre-trained on a large dataset (such as ImageNet) was transferred to a relatively smaller target dataset to fine-tune and the first-level classification features of images are extracted. Then, the similar images in the target dataset are clustered into several high-similarity categories, and the DCNN model of the first-level transfer is fine-tuned again on them and the second- level classification features are extracted. Then, the two-level classification features are encoded and fused, and the multi-kernel support vector machine (MKSVM) is used for scene classification. The experimental results in the common remote sensing datasets show that the average classification accuracy of the proposed method is improved by at least 1.83% compared with the one-time transfer and fine-tuning, especially the classification of confusable images is increased by at least 4%. In this paper, the DCNN is transferred gradually and is advanced to enhance the representation ability of the extracted image features, which makes the fusion features more recognizable. At the same time, the MKSVM is used to improve the generalization ability of the fusion features, so the classification result is better.

[1]  Gerald Penn,et al.  Applying Convolutional Neural Networks concepts to hybrid NN-HMM model for speech recognition , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Ye Zhang,et al.  Classification of hyperspectral image based on deep belief networks , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[3]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[4]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Stephen Balaban,et al.  Deep learning and face recognition: the state of the art , 2015, Defense + Security Symposium.

[7]  S. Sachin Kumar,et al.  Deep Model for Classification of Hyperspectral image using Restricted Boltzmann Machine , 2014, ICONIAAC '14.

[8]  Chuanhai Liu,et al.  MAXIMUM LIKELIHOOD ESTIMATION FROM INCOMPLETE DATA VIA EM-TYPE ALGORITHMS , 2003 .

[9]  Hong Sun,et al.  Unsupervised Feature Learning Via Spectral Clustering of Multidimensional Patches for Remotely Sensed Scene Classification , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[10]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[11]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Liangpei Zhang,et al.  An SVM Ensemble Approach Combining Spectral, Structural, and Semantic Features for the Classification of High-Resolution Remotely Sensed Imagery , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Chong-Wah Ngo,et al.  Evaluating bag-of-visual-words representations in scene classification , 2007, MIR '07.

[14]  William J. Emery,et al.  A neural network approach using multi-scale textural metrics from very high-resolution panchromatic imagery for urban land-use classification , 2009 .

[15]  Bin Luo,et al.  Robust Autodual Morphological Profiles for the Classification of High-Resolution Satellite Images , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Shawn D. Newsam,et al.  Spatial pyramid co-occurrence for image classification , 2011, 2011 International Conference on Computer Vision.

[17]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[18]  Junwei Han,et al.  Multi-class geospatial object detection and geographic image classification based on collection of part detectors , 2014 .

[19]  Lei Guo,et al.  Auto-encoder-based shared mid-level visual dictionary learning for scene classification using very high resolution remote sensing images , 2015, IET Comput. Vis..

[20]  Luisa Verdoliva,et al.  Land Use Classification in Remote Sensing Images by Convolutional Neural Networks , 2015, ArXiv.

[21]  Sun Cui-juan,et al.  Support Vector Machine Based K-type Kernel Function , 2006 .

[22]  Anil M. Cheriyadat,et al.  Unsupervised Feature Learning for Aerial Scene Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[24]  Jefersson Alex dos Santos,et al.  Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Bo Du,et al.  Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art , 2016, IEEE Geoscience and Remote Sensing Magazine.

[27]  Xing Zhao,et al.  Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[28]  Jianping Li,et al.  Multiple-kernel SVM based multiple-task oriented data mining system for gene expression data analysis , 2011, Expert Syst. Appl..

[29]  Lei Guo,et al.  Effective and Efficient Midlevel Visual Elements-Oriented Land-Use Classification Using VHR Remote Sensing Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[31]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.