Fusion of Dual Spatial Information for Hyperspectral Image Classification

The inclusion of spatial information into spectral classifiers for fine-resolution hyperspectral imagery has led to significant improvements in terms of classification performance. The task of spectral-spatial hyperspectral image classification has remained challenging because of high intraclass spectrum variability and low interclass spectral variability. This fact has made the extraction of spatial information highly active. In this work, a novel hyperspectral image classification framework using the fusion of dual spatial information is proposed, in which the dual spatial information is built by both exploiting pre-processing feature extraction and post-processing spatial optimization. In the feature extraction stage, an adaptive texture smoothing method is proposed to construct the structural profile (SP), which makes it possible to precisely extract discriminative features from hyperspectral images. The SP extraction method is used here for the first time in the remote sensing community. Then, the extracted SP is fed into a spectral classifier. In the spatial optimization stage, a pixel-level classifier is used to obtain the class probability followed by an extended random walker-based spatial optimization technique. Finally, a decision fusion rule is utilized to fuse the class probabilities obtained by the two different stages. Experiments performed on three data sets from different scenes illustrate that the proposed method can outperform other state-of-the-art classification techniques. In addition, the proposed feature extraction method, i.e., SP, can effectively improve the discrimination between different land covers.

[1]  Antonio J. Plaza,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 1 Spectral–Spatial Hyperspectral Image Segmentation Using S , 2022 .

[2]  Licheng Jiao,et al.  Classification of Hyperspectral Images Based on Multiclass Spatial–Spectral Generative Adversarial Networks , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Hao Wu,et al.  Semi-Supervised Deep Learning Using Pseudo Labels for Hyperspectral Image Classification , 2018, IEEE Transactions on Image Processing.

[4]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Stephen Marshall,et al.  Effective Feature Extraction and Data Reduction in Remote Sensing Using Hyperspectral Imaging [Applications Corner] , 2014, IEEE Signal Processing Magazine.

[6]  Hui Lin,et al.  Classification of Hyperspectral Images by Gabor Filtering Based Deep Network , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[7]  Naoto Yokoya,et al.  Invariant Attribute Profiles: A Spatial-Frequency Joint Feature Extractor for Hyperspectral Image Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Liang Xiao,et al.  Supervised Spectral–Spatial Hyperspectral Image Classification With Weighted Markov Random Fields , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Shutao Li,et al.  PCA-Based Edge-Preserving Features for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Xia Xu,et al.  Hierarchical Guidance Filtering-Based Ensemble Classification for Hyperspectral Images , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Jon Atli Benediktsson,et al.  Intrinsic Image Decomposition for Feature Extraction of Hyperspectral Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Jon Atli Benediktsson,et al.  Automatic Framework for Spectral–Spatial Classification Based on Supervised Feature Extraction and Morphological Attribute Profiles , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[13]  Xiao Xiang Zhu,et al.  Robust Object-Based Multipass InSAR Deformation Reconstruction , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Johannes R. Sveinsson,et al.  Hyperspectral Feature Extraction Using Total Variation Component Analysis , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Naoto Yokoya,et al.  Random Forest Ensembles and Extended Multiextinction Profiles for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Jon Atli Benediktsson,et al.  Feature Extraction of Hyperspectral Images With Image Fusion and Recursive Filtering , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Xiangtao Zheng,et al.  Remote Sensing Scene Classification by Unsupervised Representation Learning , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Min Zhou,et al.  Collaborative Sparse Hyperspectral Unmixing Using $l_{0}$ Norm , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Jon Atli Benediktsson,et al.  Classification of Hyperspectral Images by Using Extended Morphological Attribute Profiles and Independent Component Analysis , 2011, IEEE Geoscience and Remote Sensing Letters.

[20]  Naoto Yokoya,et al.  X-ModalNet: A semi-supervised deep cross-modal network for classification of remote sensing data , 2020, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[21]  Xuelong Li,et al.  Semi-Supervised Multitask Learning for Scene Recognition , 2015, IEEE Transactions on Cybernetics.

[22]  Jon Atli Benediktsson,et al.  Feature Extraction for Hyperspectral Imagery: The Evolution From Shallow to Deep: Overview and Toolbox , 2020, IEEE Geoscience and Remote Sensing Magazine.

[23]  Jon Atli Benediktsson,et al.  Fusion of Multiple Edge-Preserving Operations for Hyperspectral Image Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Jon Atli Benediktsson,et al.  Deep Learning for Hyperspectral Image Classification: An Overview , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Peijun Du,et al.  Random Subspace Ensembles for Hyperspectral Image Classification With Extended Morphological Attribute Profiles , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Jon Atli Benediktsson,et al.  Multiple Feature Learning for Hyperspectral Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Chunyang He,et al.  Environmental degradation in the urban areas of China: Evidence from multi-source remote sensing data , 2017 .

[28]  Qi Wang,et al.  Salient Band Selection for Hyperspectral Image Classification via Manifold Ranking , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Jon Atli Benediktsson,et al.  Classification of Hyperspectral Images by Exploiting Spectral–Spatial Information of Superpixel via Multiple Kernels , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Pedram Ghamisi,et al.  Component Decomposition-Based Hyperspectral Resolution Enhancement for Mineral Mapping , 2020, Remote. Sens..

[31]  Jun Li,et al.  Active Learning With Convolutional Neural Networks for Hyperspectral Image Classification Using a New Bayesian Approach , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Jun Li,et al.  Advanced Spectral Classifiers for Hyperspectral Images: A review , 2017, IEEE Geoscience and Remote Sensing Magazine.

[33]  Pedram Ghamisi,et al.  Noise-Robust Hyperspectral Image Classification via Multi-Scale Total Variation , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[34]  Pedram Ghamisi,et al.  A Machine Learning Framework for Drill-Core Mineral Mapping Using Hyperspectral and High-Resolution Mineralogical Data Fusion , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Lin Zhu,et al.  Generative Adversarial Networks for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Jon Atli Benediktsson,et al.  Nonlinear Multiple Kernel Learning With Multiple-Structure-Element Extended Morphological Profiles for Hyperspectral Image Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Zhenwei Shi,et al.  MugNet: Deep learning for hyperspectral image classification using limited samples , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[39]  Yanfeng Gu,et al.  Superpixel-Based Intrinsic Image Decomposition of Hyperspectral Images , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Bo Du,et al.  Hyperspectral image classification via a random patches network , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[42]  Sukho Lee,et al.  A Framework for Moving Least Squares Method with Total Variation Minimizing Regularization , 2013, Journal of Mathematical Imaging and Vision.

[43]  Shutao Li,et al.  Decolorization-Based Hyperspectral Image Visualization , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Pedram Ghamisi,et al.  Texture-aware total variation-based removal of sun glint in hyperspectral images , 2020 .

[45]  Naoto Yokoya,et al.  Learning to propagate labels on graphs: An iterative multitask regression framework for semi-supervised hyperspectral dimensionality reduction , 2019, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[46]  Ying Wang,et al.  Deep Learning Ensemble for Hyperspectral Image Classification , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Johannes R. Sveinsson,et al.  Classification of hyperspectral data from urban areas based on extended morphological profiles , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Jon Atli Benediktsson,et al.  Extinction Profiles for the Classification of Remote Sensing Data , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[49]  Hong Yan,et al.  Generalized Tensor Regression for Hyperspectral Image Classification , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Mahdi Khodadadzadeh,et al.  Multi-Sensor Spectral Imaging of Geological Samples: A Data Fusion Approach Using Spatio-Spectral Feature Extraction , 2019, Sensors.

[51]  Xuelong Li,et al.  Locality and Structure Regularized Low Rank Representation for Hyperspectral Image Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[53]  Lorenzo Bruzzone,et al.  Classification of Hyperspectral Images With Regularized Linear Discriminant Analysis , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[54]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[55]  Xudong Kang,et al.  Dual-Path Network-Based Hyperspectral Image Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[56]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[57]  Qingshan Liu,et al.  Cascaded Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.