Multi-scale spatial-spectral fusion based on multi-input fusion calculation and coordinate attention for hyperspectral image classification

Abstract Recently, the deep learning method that integrates image features has gradually become a hot development trend in hyperspectral image classification. However, these studies did not fully consider the fusion of image features, and did not remove the interference to the classification process caused by the difference in the size of the objects. These factors hinder the further improvement of the classification effect. To eliminate these drawbacks, this paper proposes a more effective fusion scheme (MSF-MIF), which realizes the fusion from the perspective of location characteristics and channel characteristics through 3D convolution and spatial feature concatenation. In view of the size discrepancy of the objects to be classified, this method extracts features from several input patches of different scales and uses the novel calculation method proposed to fuse them, which minimizes the interference caused by size differences. In addition, this research also tried to quote the coordinate attention structure for the first time that combines spatial and spectral attention features to further improve the classification performance. Experimental results on three commonly used data sets prove that this framework has achieved a breakthrough in classification accuracy.

[1]  Jun Li,et al.  Improved hyperspectral image classification by active learning using pre-designed mixed pixels , 2016, Pattern Recognit..

[2]  Li Ma,et al.  Spatial and class structure regularized sparse representation graph for semi-supervised hyperspectral image classification , 2018, Pattern Recognit..

[3]  Xuelong Li,et al.  A Hybrid Sparsity and Distance-Based Discrimination Detector for Hyperspectral Images , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[4]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[5]  Chao Tao,et al.  Semi-Supervised Variational Generative Adversarial Networks for Hyperspectral Image Classification , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[6]  Yue Wu,et al.  Double-Branch Multi-Attention Mechanism Network for Hyperspectral Image Classification , 2019, Remote. Sens..

[7]  Tianming Zhan,et al.  3-D Channel and Spatial Attention Based Multiscale Spatial–Spectral Residual Network for Hyperspectral Image Classification , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Licheng Jiao,et al.  Recursive Autoencoders-Based Unsupervised Feature Learning for Hyperspectral Image Classification , 2017, IEEE Geoscience and Remote Sensing Letters.

[9]  Yansheng Li,et al.  Unsupervised Spectral–Spatial Feature Learning With Stacked Sparse Autoencoder for Hyperspectral Imagery Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[10]  Yunsong Li,et al.  High-quality spectral-spatial reconstruction using saliency detection and deep feature enhancement , 2019, Pattern Recognit..

[11]  Xing Zhao,et al.  Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[12]  Bo Du,et al.  Spectral–Spatial Unified Networks for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Qian Du,et al.  Combined sparse and collaborative representation for hyperspectral target detection , 2015, Pattern Recognit..

[14]  Jon Atli Benediktsson,et al.  Linear Versus Nonlinear PCA for the Classification of Hyperspectral Data Based on the Extended Morphological Profiles , 2012, IEEE Geoscience and Remote Sensing Letters.

[15]  Cheng Shi,et al.  Superpixel-based 3D deep neural networks for hyperspectral image classification , 2018, Pattern Recognit..

[16]  Heesung Kwon,et al.  Going Deeper With Contextual CNN for Hyperspectral Image Classification , 2016, IEEE Transactions on Image Processing.

[17]  Hao Wu,et al.  Semi-supervised dimensionality reduction of hyperspectral imagery using pseudo-labels , 2018, Pattern Recognit..

[18]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Johannes R. Sveinsson,et al.  Classification of hyperspectral data from urban areas based on extended morphological profiles , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Wei He,et al.  Hyperspectral Image Denoising With Total Variation Regularization and Nonlocal Low-Rank Tensor Decomposition , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Bing Liu,et al.  Supervised Deep Feature Extraction for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Jun Huang,et al.  Spectral-Spatial Attention Networks for Hyperspectral Image Classification , 2019, Remote. Sens..

[23]  Xiuping Jia,et al.  Spectral constraint adversarial autoencoders approach to feature representation in hyperspectral anomaly detection , 2019, Neural Networks.

[24]  Haibo He,et al.  Self-adaptive manifold discriminant analysis for feature extraction from hyperspectral imagery , 2020, Pattern Recognit..

[25]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Zhiming Luo,et al.  Spectral–Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Shihong Du,et al.  Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Dimension Reduction and Deep Learning Approach , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Christopher E. Ndehedehe,et al.  Deep support vector machine for hyperspectral image classification , 2020, Pattern Recognit..

[29]  Li Ma,et al.  Probabilistic class structure regularized sparse representation graph for semi-supervised hyperspectral image classification , 2017, Pattern Recognit..

[30]  P. C. Mohanty,et al.  Identification of coral reef feature using hyperspectral remote sensing , 2016, Asia-Pacific Remote Sensing.

[31]  Huchuan Lu,et al.  Deep visual tracking: Review and experimental comparison , 2018, Pattern Recognit..

[32]  Zhijing Yang,et al.  MIMN-DPP: Maximum-information and minimum-noise determinantal point processes for unsupervised hyperspectral band selection , 2020, Pattern Recognit..

[33]  Qian Du,et al.  Gabor-Filtering-Based Nearest Regularized Subspace for Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.