Adaptive Multiscale Deep Fusion Residual Network for Remote Sensing Image Classification

With the development of remote sensing imaging technology, remote sensing images with high-resolution and complex structure can be acquired easily. The classification of remote sensing images is always a hot and challenging problem. In order to improve the performance of remote sensing image classification, we propose an adaptive multiscale deep fusion residual network (AMDF-ResNet). The AMDF-ResNet consists of a backbone network and a fusion network. The backbone network including several residual blocks generates multiscale hierarchy features, which contain semantic information from low to high levels. In the fusion network, the adaptive feature fusion module proposed can emphasize useful information and suppress useless information by learning the weights, which represent the importance of the features. The AMDF-ResNet can make full use of the multiscale hierarchy features and the extracted feature is discriminative. In addition, we propose a samples selection method named important samples selection strategy (ISSS). Based on superpixels segmentation result, gradient information and spatial distribution are used as two references to determine the selection numbers and select samples. Compared with the random selection strategy, training samples selected by ISSS are more representative and diverse. The experimental results on four data sets demonstrate that the AMDF-ResNet and ISSS are effective.

[1]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Jin Zhao,et al.  Superpixel-Based Multiple Local CNN for Panchromatic and Multispectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[5]  T. Herring,et al.  GPS Meteorology: Remote Sensing of Atmospheric Water Vapor Using the Global Positioning System , 1992 .

[6]  Johannes R. Sveinsson,et al.  Classification of hyperspectral data from urban areas based on extended morphological profiles , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Fan Zhang,et al.  Deep Convolutional Neural Networks for Hyperspectral Image Classification , 2015, J. Sensors.

[8]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[9]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Xin Huang,et al.  A multi-index learning approach for classification of high-resolution remotely sensed images over urban areas , 2014 .

[11]  Mahesh Pal,et al.  Random forest classifier for remote sensing classification , 2005 .

[12]  Patrick Lambert,et al.  3-D Deep Learning Approach for Remote Sensing Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Jon Atli Benediktsson,et al.  Classification of Hyperspectral Images by Exploiting Spectral–Spatial Information of Superpixel via Multiple Kernels , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Bin Wang,et al.  Deep Convolutional networks with superpixel segmentation for hyperspectral image classification , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[15]  Antonio J. Plaza,et al.  Semisupervised Hyperspectral Image Segmentation Using Multinomial Logistic Regression With Active Learning , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Yun Shi,et al.  3D Convolutional Neural Networks for Crop Classification with Multi-Temporal Remote Sensing Images , 2018, Remote. Sens..

[17]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Philippe Marthon,et al.  An optimal multiedge detector for SAR image segmentation , 1998, IEEE Trans. Geosci. Remote. Sens..

[19]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[20]  Hongxun Yao,et al.  Deep Feature Fusion for VHR Remote Sensing Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Qian Du,et al.  Multisource Remote Sensing Data Classification Based on Convolutional Neural Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Licheng Jiao,et al.  Multifeature Hyperspectral Image Classification With Local and Nonlocal Spatial Information via Markov Random Field in Semantic Space , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Xing Chen,et al.  Stacked Denoise Autoencoder Based Feature Extraction and Classification for Hyperspectral Images , 2016, J. Sensors.

[25]  韩玲 Han Ling,et al.  High Spatial Resolution Remote Sensing Image Classification Based on Deep Learning , 2016 .

[26]  Shuyuan Yang,et al.  Deep Fully Convolutional Network-Based Spatial Distribution Prediction for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Ryan S. Miller,et al.  Identifying populations potentially exposed to agricultural pesticides using remote sensing and a Geographic Information System. , 1999, Environmental health perspectives.

[28]  Johannes R. Sveinsson,et al.  Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles , 2008, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[29]  Alexandre Boulch,et al.  Benchmarking classification of earth-observation data: From learning explicit features to convolutional networks , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[30]  Xing Zhao,et al.  Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[31]  Shuicheng Yan,et al.  Dual Path Networks , 2017, NIPS.

[32]  Koreen Millard,et al.  On the Importance of Training Data Sample Selection in Random Forest Image Classification: A Case Study in Peatland Ecosystem Mapping , 2015, Remote. Sens..

[33]  Gang Wang,et al.  Spectral-spatial classification of hyperspectral image using autoencoders , 2013, 2013 9th International Conference on Information, Communications & Signal Processing.

[34]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Jin Zhao,et al.  Deep Multiple Instance Learning-Based Spatial–Spectral Classification for PAN and MS Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[36]  S. P. Hoogendoorn,et al.  Microscopic Traffic Data Collection by Remote Sensing , 2003 .

[37]  Bo Du,et al.  Unsupervised-Restricted Deconvolutional Neural Network for Very High Resolution Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Fachao Qin,et al.  Superpixel Segmentation for Polarimetric SAR Imagery Using Local Iterative Clustering , 2015, IEEE Geoscience and Remote Sensing Letters.

[39]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Liangpei Zhang,et al.  On Combining Multiple Features for Hyperspectral Remote Sensing Image Classification , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Johannes R. Sveinsson,et al.  Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[42]  Ying Li,et al.  Spectral-Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network , 2017, Remote. Sens..

[43]  Shutao Li,et al.  Hyperspectral Image Classification With Deep Feature Fusion Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Shihong Du,et al.  Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Dimension Reduction and Deep Learning Approach , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Licheng Jiao,et al.  Multiple Kernel Learning Based on Discriminative Kernel Clustering for Hyperspectral Band Selection , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[47]  K. L. Majumder,et al.  Information fusion in tree classifiers , 2001 .

[48]  Antonio J. Plaza,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 1 Spectral–Spatial Hyperspectral Image Segmentation Using S , 2022 .

[49]  J. Anthony Gualtieri,et al.  Support vector machines for hyperspectral remote sensing classification , 1999, Other Conferences.

[50]  Jon Atli Benediktsson,et al.  Classification and feature extraction for remote sensing images from urban areas based on morphological transformations , 2003, IEEE Trans. Geosci. Remote. Sens..

[51]  Jun Zhou,et al.  On the Sampling Strategy for Evaluation of Spectral-Spatial Methods in Hyperspectral Image Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[52]  James E. Fowler,et al.  Nearest Regularized Subspace for Hyperspectral Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.