High-Resolution Remote Sensing Image Classification Method Based on Convolutional Neural Network and Restricted Conditional Random Field

Convolutional neural networks (CNNs) can adapt to more complex data, extract deeper characteristics from images, and achieve higher classification accuracy in remote sensing image scene classification and object detection compared to traditional shallow-model methods. However, directly applying common-structure CNNs to pixel-based remote sensing image classification will lead to boundary or outline distortions of the land cover and consumes enormous computation time in the image classification stage. To solve this problem, we propose a high-resolution remote sensing image classification method based on CNN and the restricted conditional random field algorithm (CNN-RCRF). CNN-RCRF adopts CNN superpixel classification instead of pixel-based classification and uses the restricted conditional random field algorithm (RCRF) to refine the superpixel result image into a pixel-based result. The proposed method not only takes advantage of the classification ability of CNNs but can also avoid boundary or outline distortions of the land cover and greatly reduce computation time in classifying images. The effectiveness of the proposed method is tested with two high-resolution remote sensing images, and the experimental results show that the CNN-RCRF outperforms the existing traditional methods in terms of overall accuracy, and CNN-RCRF’s computation time is much less than that of traditional pixel-based deep-model methods.

[1]  Tara N. Sainath,et al.  Deep Convolutional Neural Networks for Large-scale Speech Tasks , 2015, Neural Networks.

[2]  M. Siegel,et al.  Hyperspectral classification via deep networks and superpixel segmentation , 2015 .

[3]  Jun Wang,et al.  Road network extraction: a neural-dynamic framework based on deep learning and a finite state machine , 2015 .

[4]  Ming Liu,et al.  ECG signal enhancement based on improved denoising auto-encoder , 2016, Eng. Appl. Artif. Intell..

[5]  Xin Pan,et al.  A hybrid MLP-CNN classifier for very fine resolution remotely sensed image classification , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[6]  Jie Geng,et al.  High-Resolution SAR Image Classification via Deep Convolutional Autoencoders , 2015, IEEE Geoscience and Remote Sensing Letters.

[7]  Brian Johnson,et al.  Classifying a high resolution image of an urban area using super-object information , 2013 .

[8]  Guosheng Lin,et al.  CRF Learning with CNN Features for Image Segmentation , 2015, Pattern Recognit..

[9]  Hongyi Liu,et al.  A New Pan-Sharpening Method With Deep Neural Networks , 2015, IEEE Geoscience and Remote Sensing Letters.

[10]  Aamir Saeed Malik,et al.  Scene classification for aerial images based on CNN using sparse coding technique , 2017 .

[11]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Haipeng Wang,et al.  Target Classification Using the Deep Convolutional Networks for SAR Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[13]  William J. Emery,et al.  Using active learning to adapt remote sensing image classifiers , 2011 .

[14]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[15]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[18]  Xin Pan,et al.  A variable precision rough set approach to the remote sensing land use/cover classification , 2010, Comput. Geosci..

[19]  Shihong Du,et al.  Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Dimension Reduction and Deep Learning Approach , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[21]  Jun Zhou,et al.  CRF learning with CNN features for hyperspectral image segmentation , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[22]  Chalavadi Krishna Mohan,et al.  Classification of human actions using pose-based features and stacked auto encoder , 2016, Pattern Recognit. Lett..

[23]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[24]  Yong Dou,et al.  Classification of Hyperspectral Remote Sensing Image Using Hierarchical Local-Receptive-Field-Based Extreme Learning Machine , 2016, IEEE Geoscience and Remote Sensing Letters.

[25]  Gang Fu,et al.  Classification for High Resolution Remote Sensing Imagery Using a Fully Convolutional Network , 2017, Remote. Sens..

[26]  Yu Liu,et al.  Hourglass-ShapeNetwork Based Semantic Segmentation for High Resolution Aerial Imagery , 2017, Remote. Sens..

[27]  Xin Pan,et al.  A central-point-enhanced convolutional neural network for high-resolution remote-sensing image classification , 2017 .

[28]  Uwe Stilla,et al.  Deep Learning Earth Observation Classification Using ImageNet Pretrained Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[29]  Fan Xia,et al.  Assessing object-based classification: advantages and limitations , 2009 .

[30]  Jie Geng,et al.  Spectral–Spatial Classification of Hyperspectral Image Based on Deep Auto-Encoder , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[31]  Liangpei Zhang,et al.  An Efficient and Robust Integrated Geospatial Object Detection Framework for High Spatial Resolution Remote Sensing Imagery , 2017, Remote. Sens..

[32]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[33]  Yong Dou,et al.  An efficient and effective convolutional auto-encoder extreme learning machine network for 3d feature learning , 2016, Neurocomputing.

[34]  Shihong Du,et al.  Scene classification using multi-scale deeply described visual words , 2016 .

[35]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[36]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Congcong Li,et al.  Stacked Autoencoder-based deep learning for remote-sensing image classification: a case study of African land-cover mapping , 2016 .

[38]  Lei Wang,et al.  Fuzzy AutoEncode Based Cloud Detection for Remote Sensing Imagery , 2017, Remote. Sens..

[39]  Carlo Gatta,et al.  Unsupervised Deep Feature Extraction for Remote Sensing Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.