Expansion Spectral–Spatial Attention Network for Hyperspectral Image Classification

Deep learning is increasingly used for the classification of hyperspectral images (HSI), thanks to its ability to completely utilize the rich characteristics of this type of imagery. However, at present, most classification models proposed for processing HSI data are based on standard convolution neural networks, which prefer to learn local information rather than global information, so that it is difficult to achieve ideal accuracy in the case of insufficient training samples in real applications. In this article, we propose a novel expansion spectral–spatial attention network (ESSAN) for HSI data classification in cases of insufficient training samples. First, a dual-branch network based on expansion convolution is employed as the model backbone to extract spectral and spatial information. All feature maps produced during the dual-branch process are superimposed to combine deep and shallow features by the ResNet concept. With the design philosophy of the superposition of expansion convolutional layers, the network can increase the receptive field to gather more global contextual information. Second, the model also includes a coordinate attention block, which directs the network to weight features according to their significance and suppresses those that are irrelevant. Finally, the method was tested on the four datasets from Matiwan Village, Pavia Center, Pavia University, and Shenzhen University, utilizing 1%, 1%, 5%, and 0.2% training samples, respectively. The results showed the overall accuracies, in order, 97.96%, 99.12%, 98.73%, and 99.36%. The preliminary results demonstrate the higher efficacy and accuracy of the proposed ESSAN in HSI data classification than the other state-of-the-art.

[1]  Richard M. Mariita,et al.  HybridGBN-SR: A Deep 3D/2D Genome Graph-Based Network for Hyperspectral Image Classification , 2022, Remote. Sens..

[2]  Y. Ge,et al.  Using soil library hyperspectral reflectance and machine learning to predict soil organic carbon: Assessing potential of airborne and spaceborne optical soil sensing , 2022, Remote Sensing of Environment.

[3]  Bing Liu,et al.  Small Sample Hyperspectral Image Classification Based on Cascade Fusion of Mixed Spatial-Spectral Features and Second-Order Pooling , 2022, Remote. Sens..

[4]  M. Vastaranta,et al.  Close-range Hyperspectral Spectroscopy Reveals Leaf Water Content Dynamics , 2021, Remote Sensing of Environment.

[5]  Lianru Gao,et al.  SpectralFormer: Rethinking Hyperspectral Image Classification With Transformers , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Zhijie Lin,et al.  A survey: Deep learning for hyperspectral image classification with few labeled samples , 2021, Neurocomputing.

[7]  Jiashi Feng,et al.  Coordinate Attention for Efficient Mobile Network Design , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Z. Cai,et al.  Spectral–Spatial and Superpixelwise PCA for Unsupervised Feature Extraction of Hyperspectral Imagery , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Yushi Chen,et al.  Spatial-Spectral Transformer for Hyperspectral Image Classification , 2021, Remote. Sens..

[10]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[11]  Hong Chen,et al.  Hybrid Depth-Separable Residual Networks for Hyperspectral Image Classification , 2020, Complex..

[12]  Raymond Y. K. Lau,et al.  Synergistic 2D/3D Convolutional Neural Network for Hyperspectral Image Classification , 2020, Remote. Sens..

[13]  Qian Du,et al.  Deep Latent Spectral Representation Learning-Based Hyperspectral Band Selection for Target Detection , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[14]  E. Honkavaara,et al.  Tree Species Classification of Drone Hyperspectral and RGB Imagery with Deep Learning Convolutional Neural Networks , 2020, Remote. Sens..

[15]  Yang Yang,et al.  Classification of Hyperspectral Image Based on Double-Branch Dual-Attention Mechanism Network , 2019, Remote. Sens..

[16]  Yanfang Ming,et al.  Classification of hyperspectral imagery with a 3D convolutional neural network and J-M distance , 2019, Advances in Space Research.

[17]  Hugo Van hamme,et al.  Hyperspectral image classification using Non-negative Tensor Factorization and 3D Convolutional Neural Networks , 2019, Signal Process. Image Commun..

[18]  Onur Avci,et al.  1D Convolutional Neural Networks and Applications: A Survey , 2019, Mechanical Systems and Signal Processing.

[19]  Richard Alan Peters,et al.  A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trends , 2019, Knowl. Based Syst..

[20]  Bidyut Baran Chaudhuri,et al.  HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyperspectral Image Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[21]  Lu Zhi,et al.  Convolutional neural networks and local binary patterns for hyperspectral image classification , 2019, European Journal of Remote Sensing.

[22]  Chenming Li,et al.  Convolution Neural Network Based on Two-Dimensional Spectrum for Hyperspectral Image Classification , 2018, J. Sensors.

[23]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[24]  Wenju Wang,et al.  A Fast Dense Spectral-Spatial Convolution Network Framework for Hyperspectral Images Classification , 2018, Remote. Sens..

[25]  Zhiming Luo,et al.  Spectral–Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Changzhe Jiao,et al.  Multiple Instance Hybrid Estimator for Hyperspectral Target Characterization and Sub-pixel Target Detection , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[27]  Shutao Li,et al.  PCA-Based Edge-Preserving Features for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Garrison W. Cottrell,et al.  Understanding Convolution for Semantic Segmentation , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[30]  Heesung Kwon,et al.  Going Deeper With Contextual CNN for Hyperspectral Image Classification , 2016, IEEE Transactions on Image Processing.

[31]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Yi Yang,et al.  Attention to Scale: Scale-Aware Semantic Image Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Dacheng Xiu,et al.  Principal Component Analysis of High-Frequency Data , 2015, Journal of the American Statistical Association.

[34]  Zhihao Qin,et al.  Estimation of Crop LAI using hyperspectral vegetation indices and a hybrid inversion method , 2015 .

[35]  Siyang Zhang,et al.  A novel hybrid KPCA and SVM with GA model for intrusion detection , 2014, Appl. Soft Comput..

[36]  Julien Jacques,et al.  Model-based clustering for multivariate functional data , 2013, Comput. Stat. Data Anal..

[37]  Evgeny A. Antipov,et al.  Mass Appraisal of Residential Apartments: An Application of Random Forest for Valuation and a CART-Based Approach for Model Diagnostics , 2010, Expert Syst. Appl..

[38]  Lorenzo Bruzzone,et al.  Extended profiles with morphological attribute filters for the analysis of hyperspectral data , 2010 .

[39]  Qian Du,et al.  Modified Fisher's Linear Discriminant Analysis for Hyperspectral Imagery , 2007, IEEE Geoscience and Remote Sensing Letters.

[40]  Johannes R. Sveinsson,et al.  Classification of hyperspectral data from urban areas based on extended morphological profiles , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[41]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[42]  Liguo Wang,et al.  Hyperspectral Image Classification Based on Expansion Convolution Network , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Jiangtao Peng,et al.  Mapping Coastal Wetlands Using Transformer in Transformer Deep Network on China ZY1-02D Hyperspectral Satellite Images , 2022, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[44]  Weiwei SUN,et al.  A Dual Global-local Attention Network for Hyperspectral Band Selection , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Zebin Wu,et al.  Spectral–Spatial Feature Tokenization Transformer for Hyperspectral Image Classification , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Maoguo Gong,et al.  A Spectral and Spatial Attention Network for Change Detection in Hyperspectral Images , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[47]  Weiqiang Pi,et al.  3D-CNN based UAV hyperspectral imagery for grassland degradation indicator ground object classification research , 2021, Ecol. Informatics.

[48]  A. Rogach,et al.  Synergistic 2D/3D Convolutional Neural Network for Hyperspectral Image Classification , 2020 .

[49]  Hui Zhang,et al.  Spatial-Spectral Joint Classification of Hyperspectral Image With Locality and Edge Preserving , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[50]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .