Convolutional neural network for spectral–spatial classification of hyperspectral images

Hyperspectral images (HSIs) have great potential in military reconnaissance, land use, marine monitoring and many other fields. In recent years, the convolutional neural network (CNN) has been successfully used to classify hyperspectral data and achieved remarkable performance. However, the limited labeled samples of HSI lead to the small sample size problem, which remains the major challenge for CNN-based HSI classification. Besides, most CNN models have large number of parameters needed to be learned, which cause high computational cost. To address the aforementioned two issues, a novel CNN-based HSI classification method is proposed. The proposed classification method has several distinguishing characteristics. First, the proposed method can robustly extract spectral and spatial features of the HSI simultaneously. Second, in the proposed CNN architecture, all convolution layers are 1 × 1 convolution layer except the first one, which can greatly reduce the number of network parameters, thus accelerating the training and testing process. Third, a small convolution and feature reuse (SC-FR) module is developed. The SC-FR module is composed of two composite layers and each composite layer consists of two cascaded 1 × 1 convolution layers. Through cross-layer connecting, the input and output features of each composite layer are concatenated and passed to the next convolution layer, thus achieving feature reuse mechanism. Cross-layer connection increases information flow and the utilization rate of middle-level features, which enhances the generalization performance of CNN effectively. Experimental results on three benchmark HSIs demonstrate the competitive superiority of the proposed method over several state-of-the-art HSI classification methods, especially when training samples are limited.

[1]  R. Holyer,et al.  Coastal bathymetry from hyperspectral observations of water radiance , 1998 .

[2]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral Images Classification Based on Dense Convolutional Networks with Spectral-Wise Attention Mechanism , 2019, Remote. Sens..

[3]  Qingshan Liu,et al.  Cascaded Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Hui Zhou,et al.  Multi-branch fusion network for hyperspectral image classification , 2019, Knowl. Based Syst..

[6]  Xuchu Yu,et al.  A dense convolutional neural network for hyperspectral image classification , 2018, Remote Sensing Letters.

[7]  Qian Du,et al.  Optimizing extreme learning machine for hyperspectral image classification , 2015 .

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Antonio Plaza,et al.  A new deep convolutional neural network for fast hyperspectral image classification , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[10]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[11]  Trac D. Tran,et al.  Hyperspectral Image Classification via Kernel Sparse Representation , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Hui Zhou,et al.  Joint Alternate Small Convolution and Feature Reuse for Hyperspectral Image Classification , 2018, ISPRS Int. J. Geo Inf..

[13]  Shutao Li,et al.  Hyperspectral Image Classification With Squeeze Multibias Network , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Deyu Meng,et al.  Hyperspectral Image Classification With Markov Random Fields and a Convolutional Neural Network , 2017, IEEE Transactions on Image Processing.

[15]  Erik Cambria,et al.  Recent Trends in Deep Learning Based Natural Language Processing , 2017, IEEE Comput. Intell. Mag..

[16]  Jun Zhou,et al.  Hyperspectral Image Classification Based on Structured Sparse Logistic Regression and Three-Dimensional Wavelet Texture Features , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Lei Shu,et al.  Hyperspectral Image Classification With Stacking Spectral Patches and Convolutional Neural Networks , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[20]  Jiaqi Liu,et al.  Deep Recurrent Convolutional Neural Network: Improving Performance For Speech Recognition , 2016 .

[21]  Qiang Song,et al.  Spectral–Spatial Feature Learning Using Cluster-Based Group Sparse Coding for Hyperspectral Image Classification , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[22]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Zheng Wenwei The Development of Hyperspectral Remote Sensing and Its Threatening to Military Equipments , 2008 .

[24]  Sen Jia,et al.  Convolutional neural networks for hyperspectral image classification , 2017, Neurocomputing.

[25]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Zhiming Luo,et al.  Spectral–Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Qi Li,et al.  Hyperspectral Imagery Classification Using Sparse Representations of Convolutional Neural Network Features , 2016, Remote. Sens..

[28]  Heesung Kwon,et al.  Going Deeper With Contextual CNN for Hyperspectral Image Classification , 2016, IEEE Transactions on Image Processing.

[29]  Juha Suomalainen,et al.  Generation of Spectral–Temporal Response Surfaces by Combining Multispectral Satellite and Hyperspectral UAV Imagery for Precision Agriculture Applications , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[30]  Shutao Li,et al.  Hyperspectral Image Classification With Deep Feature Fusion Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Nikolaos Doulamis,et al.  Deep supervised learning for hyperspectral data classification through convolutional neural networks , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[32]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[33]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Baocai Yin,et al.  Hyperspectral Image Classification Based on Deep Deconvolution Network With Skip Architecture , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Lin Zhu,et al.  Hyperspectral Images Classification With Gabor Filtering and Convolutional Neural Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[36]  Mark A. Richardson,et al.  An introduction to hyperspectral imaging and its application for security, surveillance and target acquisition , 2010 .

[37]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.