Multiscale Dynamic Graph Convolutional Network for Hyperspectral Image Classification

Convolutional Neural Network (CNN) has demonstrated impressive ability to represent hyperspectral images and to achieve promising results in hyperspectral image classification. However, traditional CNN models can only operate convolution on regular square image regions with fixed size and weights, so they cannot universally adapt to the distinct local regions with various object distributions and geometric appearances. Therefore, their classification performances are still to be improved, especially in class boundaries. To alleviate this shortcoming, we consider employing the recently proposed Graph Convolutional Network (GCN) for hyperspectral image classification, as it can conduct the convolution on arbitrarily structured non-Euclidean data and is applicable to the irregular image regions represented by graph topological information. Different from the commonly used GCN models which work on a fixed graph, we enable the graph to be dynamically updated along with the graph convolution process, so that these two steps can be benefited from each other to gradually produce the discriminative embedded features as well as a refined graph. Moreover, to comprehensively deploy the multi-scale information inherited by hyperspectral images, we establish multiple input graphs with different neighborhood scales to extensively exploit the diversified spectral-spatial correlations at multiple scales. Therefore, our method is termed 'Multi-scale Dynamic Graph Convolutional Network' (MDGCN). The experimental results on three typical benchmark datasets firmly demonstrate the superiority of the proposed MDGCN to other state-of-the-art methods in both qualitative and quantitative aspects.

[1]  Xiao Xiang Zhu,et al.  A Self-Improving Convolution Neural Network for the Classification of Hyperspectral Data , 2016, IEEE Geoscience and Remote Sensing Letters.

[2]  Le Song,et al.  Discriminative Embeddings of Latent Variable Models for Structured Data , 2016, ICML.

[3]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[4]  Jun Li,et al.  Discriminative Low-Rank Gabor Filtering for Spectral–Spatial Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[6]  Shutao Li,et al.  Spectral-spatial classification of hyperspectral images via multiscale superpixels based sparse representation , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[7]  Bo Li,et al.  Multi-scale 3D deep convolutional neural network for hyperspectral image classification , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[8]  Jun Li,et al.  Active Learning With Convolutional Neural Networks for Hyperspectral Image Classification Using a New Bayesian Approach , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Sanja Fidler,et al.  3D Graph Neural Networks for RGBD Semantic Segmentation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  Jon Atli Benediktsson,et al.  Advances in Spectral-Spatial Classification of Hyperspectral Images , 2013, Proceedings of the IEEE.

[11]  Shiming Xiang,et al.  Discriminant Tensor Spectral–Spatial Feature Extraction for Hyperspectral Image Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[12]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[13]  Qian Du,et al.  Local Binary Patterns and Extreme Learning Machine for Hyperspectral Imagery Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Bo Du,et al.  Saliency-Guided Unsupervised Feature Learning for Scene Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Yanpeng Li,et al.  Improving deep neural networks using softplus units , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[16]  Qingshan Liu,et al.  Matrix-Based Discriminant Subspace Ensemble for Hyperspectral Image Spatial–Spectral Feature Fusion , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Li Ma,et al.  Local Manifold Learning-Based $k$ -Nearest-Neighbor for Hyperspectral Image Classification , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[19]  Gustavo Camps-Valls,et al.  Composite kernels for hyperspectral image classification , 2006, IEEE Geoscience and Remote Sensing Letters.

[20]  Xiuping Jia,et al.  Simplified Conditional Random Fields With Class Boundary Constraint for Spectral-Spatial Based Remote Sensing Image Classification , 2012, IEEE Geoscience and Remote Sensing Letters.

[21]  Yuan Yan Tang,et al.  Spectral–Spatial Graph Convolutional Networks for Semisupervised Hyperspectral Image Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[22]  Shihong Du,et al.  Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Dimension Reduction and Deep Learning Approach , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Haokui Zhang,et al.  Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network , 2017 .

[24]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[25]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[26]  Xia Xu,et al.  Hierarchical Guidance Filtering-Based Ensemble Classification for Hyperspectral Images , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Ping Zhong,et al.  Multiple Instance Learning for Multiple Diverse Hyperspectral Target Characterizations , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Nikolaos Doulamis,et al.  Deep supervised learning for hyperspectral data classification through convolutional neural networks , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[29]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[30]  Ah Chung Tsoi,et al.  Computational Capabilities of Graph Neural Networks , 2009, IEEE Transactions on Neural Networks.

[31]  Xavier Bresson,et al.  Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks , 2017, NIPS.

[32]  Qingshan Liu,et al.  Cascaded Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Yang Wang,et al.  A Dynamic Hidden Markov Random Field Model for Foreground and Shadow Segmentation , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[34]  Huchuan Lu,et al.  Hyperspectral Image Classification via JCR and SVM Models With Decision Fusion , 2016, IEEE Geoscience and Remote Sensing Letters.

[35]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[36]  Yi Shen,et al.  Convolutional neural network based classification for hyperspectral data , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[37]  Trac D. Tran,et al.  Hyperspectral Image Classification Using Dictionary-Based Sparse Representation , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Yuan Yan Tang,et al.  Simultaneous Spectral-Spatial Feature Selection and Extraction for Hyperspectral Images , 2019, IEEE Transactions on Cybernetics.

[40]  G. Mercier,et al.  Support vector machines for hyperspectral image classification with spectral-based kernels , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[41]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Lei Shi,et al.  Supervised Graph Embedding for Polarimetric SAR Image Classification , 2013, IEEE Geoscience and Remote Sensing Letters.

[43]  Fan Zhang,et al.  Deep Convolutional Neural Networks for Hyperspectral Image Classification , 2015, J. Sensors.

[44]  Zhengyang Wang,et al.  Large-Scale Learnable Graph Convolutional Networks , 2018, KDD.

[45]  Lin He,et al.  A three-dimensional filtering method for spectral-spatial hyperspectral image classification , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[46]  Wei Li,et al.  Diverse Region-Based CNN for Hyperspectral Image Classification , 2018, IEEE Transactions on Image Processing.

[47]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral image classification using two-channel deep convolutional neural network , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[48]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[49]  Shutao Li,et al.  Learning to Diversify Deep Belief Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Zhou Guo,et al.  On combining multiscale deep learning features for the classification of hyperspectral remote sensing imagery , 2015 .

[51]  Xing Zhao,et al.  Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[52]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[53]  Cheng Shi,et al.  Multi-scale hierarchical recurrent neural networks for hyperspectral image classification , 2018, Neurocomputing.

[54]  Ye Zhang,et al.  Classification of hyperspectral image based on deep belief networks , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[55]  Johannes R. Sveinsson,et al.  Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles , 2008, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[56]  Bor-Chen Kuo,et al.  Spatial information based support vector machine for hyperspectral image classification , 2010, 2010 IEEE International Geoscience and Remote Sensing Symposium.

[57]  Liangpei Zhang,et al.  A Support Vector Conditional Random Fields Classifier With a Mahalanobis Distance Boundary Constraint for High Spatial Resolution Remote Sensing Imagery , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[58]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  Qian Du,et al.  Hyperspectral Image Classification Using Deep Pixel-Pair Features , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[60]  Lorenzo Bruzzone,et al.  Two-Stream Deep Architecture for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[61]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[62]  Raymond Y. K. Lau,et al.  Hyperspectral Image Classification With Deep Learning Models , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[63]  Shanjun Mao,et al.  Spectral–spatial classification of hyperspectral images using deep convolutional neural networks , 2015 .

[64]  J. Benediktsson,et al.  Remotely Sensed Image Classification Using Sparse Representations of Morphological Attribute Profiles , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[65]  Subhransu Maji,et al.  Classification using intersection kernel support vector machines is efficient , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[66]  Heesung Kwon,et al.  Going Deeper With Contextual CNN for Hyperspectral Image Classification , 2016, IEEE Transactions on Image Processing.

[67]  Bo Wang,et al.  Unsupervised metric fusion by cross diffusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[68]  Jie Geng,et al.  Hyperspectral image classification via contextual deep learning , 2015, EURASIP Journal on Image and Video Processing.

[69]  Antonio J. Plaza,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 1 Spectral–Spatial Hyperspectral Image Segmentation Using S , 2022 .