A Pixel Cluster CNN and Spectral-Spatial Fusion Algorithm for Hyperspectral Image Classification With Small-Size Training Samples

Convolutional neural networks (CNNs) can automatically learn features from the hyperspectral image (HSI) data, avoiding the difficulty of manually extracting features. However, the number of training samples for the classification of HSIs is always limited, making it difficult for CNN to obtain effective features and resulting in low classification accuracy. To solve this problem, a pixel cluster CNN and spectral-spatial fusion (SSF) algorithm for hyperspectral image classification with small-size training samples is proposed in this article. First, spatial information is extracted by the gray level co-occurrence matrix. Then, spatial information and spectral information are fused by means of bands superposition, forming spectral-spatial features. To expand the number of training samples, the pixels after SSF are combined into pixel clusters according to a certain rule. Finally, a CNN framework is utilized to extract effective features from the pixel clusters. Experiments based on three standard HSIs demonstrate that the proposed algorithm can get better performance than the conventional CNN and also outperforms other studied algorithms in the case of small training set.

[1]  Yiran Li,et al.  Hyperspectral Image Classification Based on Unsupervised Heterogeneous Domain Adaptation CycleGan , 2020 .

[2]  Zeshui Xu,et al.  Kernel Low-Rank Entropic Component Analysis for Hyperspectral Image Classification , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  Daniel Jiwoong Im,et al.  Semisupervised Hyperspectral Image Classification via Neighborhood Graph Learning , 2015, IEEE Geoscience and Remote Sensing Letters.

[4]  Yanning Zhang,et al.  Hyperspectral Image Classification With CapsNet and Markov Random Fields , 2020, IEEE Access.

[5]  Yicong Zhou,et al.  Region-Kernel-Based Support Vector Machines for Hyperspectral Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Brajesh Kumar,et al.  Hyperspectral image classification using three-dimensional geometric moments , 2020, IET Image Process..

[7]  Qian Du,et al.  Hyperspectral Image Classification Using Deep Pixel-Pair Features , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Jian Yu,et al.  Texture Image Segmentation Based on Gaussian Mixture Models and Gray Level Co-occurrence Matrix , 2010, 2010 Third International Symposium on Information Science and Engineering.

[9]  Cheng Shi,et al.  Multiscale Superpixel-Based Hyperspectral Image Classification Using Recurrent Neural Networks With Stacked Autoencoders , 2020, IEEE Transactions on Multimedia.

[10]  Lorenzo Bruzzone,et al.  Semisupervised Classification of Hyperspectral Images by SVMs Optimized in the Primal , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Rui Li,et al.  A Convolutional Neural Network With Mapping Layers for Hyperspectral Image Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Peng Fu,et al.  Hyperspectral Image Classification Method Based on 2D–3D CNN and Multibranch Feature Fusion , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[13]  Jon Atli Benediktsson,et al.  Automatic Design of Convolutional Neural Network for Hyperspectral Image Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Xiangtao Zheng,et al.  Spectral–Spatial Attention Network for Hyperspectral Image Classification , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Bing Liu,et al.  Supervised Deep Feature Extraction for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Meiping Song,et al.  Hyperspectral Image Classification Method Based on CNN Architecture Embedding With Hashing Semantic Feature , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Ying Wang,et al.  Deep Learning Ensemble for Hyperspectral Image Classification , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[18]  Hong Huang,et al.  Adaptive Residual Convolutional Neural Network for Hyperspectral Image Classification , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[19]  Lianru Gao,et al.  Graph Convolutional Networks for Hyperspectral Image Classification , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Wei Feng,et al.  New margin-based subsampling iterative technique in modified random forests for classification , 2019, Knowl. Based Syst..

[21]  Wei Feng,et al.  Imbalanced Hyperspectral Image Classification With an Adaptive Ensemble Method Based on SMOTE and Rotation Forest With Differentiated Sampling Rates , 2019, IEEE Geoscience and Remote Sensing Letters.

[22]  Sean M. Kennedy,et al.  Cluster-Based Spectral-Spatial Segmentation of Hyperspectral Imagery , 2019, 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[23]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[24]  Fan Zhang,et al.  Deep Convolutional Neural Networks for Hyperspectral Image Classification , 2015, J. Sensors.

[25]  Liang Zou,et al.  Spectral–Spatial Exploration for Hyperspectral Image Classification via the Fusion of Fully Convolutional Networks , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[26]  Jon Atli Benediktsson,et al.  Sensitivity of Support Vector Machines to Random Feature Selection in Classification of Hyperspectral Data , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[27]  C. L. Philip Chen,et al.  Hyperspectral Image Classification Based on Domain Adaptation Broad Learning , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[28]  Yingying Dong,et al.  Ensemble Margin Based Semi-Supervised Random Forest for the Classification of Hyperspectral Image with Limited Training Data , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[29]  Mingli Ding,et al.  Deep Feature Fusion via Two-Stream Convolutional Neural Network for Hyperspectral Image Classification , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Jun Li,et al.  Recent Advances on Spectral–Spatial Hyperspectral Image Classification: An Overview and New Guidelines , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Leiquan Wang,et al.  An Encoder–Decoder Convolution Network With Fine-Grained Spatial Information for Hyperspectral Images Classification , 2020, IEEE Access.

[32]  Nikolaos Doulamis,et al.  Deep supervised learning for hyperspectral data classification through convolutional neural networks , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[33]  Farid Melgani,et al.  Nearest Neighbor Classification of Remote Sensing Images With the Maximal Margin Principle , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Jinping Wang,et al.  Ensemble EMD-Based Spectral-Spatial Feature Extraction for Hyperspectral Image Classification , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Jakub Nalepa,et al.  Hyperspectral Band Selection Using Attention-Based Convolutional Neural Networks , 2020, IEEE Access.

[36]  Jon Atli Benediktsson,et al.  SVM- and MRF-Based Method for Accurate Classification of Hyperspectral Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[37]  Hongying Liu,et al.  An improved ISODATA algorithm for hyperspectral image classification , 2014, 2014 7th International Congress on Image and Signal Processing.

[38]  Eligius M. T. Hendrix,et al.  Rotation Equivariant Convolutional Neural Networks for Hyperspectral Image Classification , 2020, IEEE Access.

[39]  Deyu Meng,et al.  Hyperspectral Image Classification With Markov Random Fields and a Convolutional Neural Network , 2017, IEEE Transactions on Image Processing.

[40]  Tianming Zhan,et al.  3-D Channel and Spatial Attention Based Multiscale Spatial–Spectral Residual Network for Hyperspectral Image Classification , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[41]  G. Arturo Sanchez-Azofeifa,et al.  Mapping Tropical Dry Forest Succession With CHRIS/PROBA Hyperspectral Images Using Nonparametric Decision Trees , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[42]  Chein-I Chang,et al.  Iterative Random Training Sampling Spectral Spatial Classification for Hyperspectral Images , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[43]  W. Marsden.  I and J , 2012 .

[44]  Mingquan Wu,et al.  Dynamic Synthetic Minority Over-Sampling Technique-Based Rotation Forest for the Classification of Imbalanced Hyperspectral Data , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[45]  Richard J. Murphy,et al.  Pretraining for Hyperspectral Convolutional Neural Network Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Meiping Song,et al.  A Simplified 2D-3D CNN Architecture for Hyperspectral Image Classification Based on Spatial–Spectral Fusion , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Yi Ma,et al.  Hyperspectral Classification Using Deep Belief Networks Based on Conjugate Gradient Update and Pixel-Centric Spectral Block Features , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.