Learning Hierarchical Spectral–Spatial Features for Hyperspectral Image Classification

This paper proposes a spectral-spatial feature learning (SSFL) method to obtain robust features of hyperspectral images (HSIs). It combines the spectral feature learning and spatial feature learning in a hierarchical fashion. Stacking a set of SSFL units, a deep hierarchical model called the spectral-spatial networks (SSN) is further proposed for HSI classification. SSN can exploit both discriminative spectral and spatial information simultaneously. Specifically, SSN learns useful high-level features by alternating between spectral and spatial feature learning operations. Then, kernel-based extreme learning machine (KELM), a shallow neural network, is embedded in SSN to classify image pixels. Extensive experiments are performed on two benchmark HSI datasets to verify the effectiveness of SSN. Compared with state-of-the-art methods, SSN with a deep hierarchical architecture obtains higher classification accuracy in terms of the overall accuracy, average accuracy, and kappa (κ) coefficient of agreement, especially when the number of the training samples is small.

[1]  Geoffrey E. Hinton,et al.  Deep, Narrow Sigmoid Belief Networks Are Universal Approximators , 2008, Neural Computation.

[2]  Yuan Yan Tang,et al.  Hierarchical kernel-based rotation and scale invariant similarity , 2014, Pattern Recognition.

[3]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[4]  Jon Atli Benediktsson,et al.  Advances in Spectral-Spatial Classification of Hyperspectral Images , 2013, Proceedings of the IEEE.

[5]  Hamid R. Rabiee,et al.  Spatial-Aware Dictionary Learning for Hyperspectral Image Classification , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Bin Wang,et al.  A Novel Spatial–Spectral Similarity Measure for Dimensionality Reduction and Classification of Hyperspectral Imagery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Jon Atli Benediktsson,et al.  Spectral–Spatial Classification of Multispectral Images Using Kernel Feature Space Representation , 2014, IEEE Geoscience and Remote Sensing Letters.

[8]  Shiming Xiang,et al.  Discriminant Tensor Spectral–Spatial Feature Extraction for Hyperspectral Image Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[9]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[10]  Qian Du,et al.  Local Binary Patterns and Extreme Learning Machine for Hyperspectral Imagery Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Xuelong Li,et al.  Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Zhuowen Tu,et al.  Robust $L_{2}E$ Estimation of Transformation for Non-Rigid Registration , 2015, IEEE Transactions on Signal Processing.

[13]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Pedram Ghamisi,et al.  Spectral–Spatial Classification of Hyperspectral Images Based on Hidden Markov Random Fields , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Dacheng Tao,et al.  Fick’s Law Assisted Propagation for Semisupervised Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Xuelong Li,et al.  Spectral-Spatial Constraint Hyperspectral Image Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Filiberto Pla,et al.  Spectral–Spatial Pixel Characterization Using Gabor Filters for Hyperspectral Image Classification , 2013, IEEE Geoscience and Remote Sensing Letters.

[19]  C. L. Philip Chen,et al.  A rapid learning and dynamic stepwise updating algorithm for flat neural networks and the application to time-series prediction , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[20]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[21]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[22]  C. L. Philip Chen,et al.  An incremental adaptive implementation of functional-link processing for function approximation, time-series prediction, and system identification , 1998, Neurocomputing.

[23]  Hong Li,et al.  Hyperspectral Image Classification Using Functional Data Analysis , 2014, IEEE Transactions on Cybernetics.

[24]  Farid Melgani,et al.  Toward an Optimal SVM Classification System for Hyperspectral Remote Sensing Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Peijun Du,et al.  Spectral–Spatial Classification for Hyperspectral Data Using Rotation Forests With Local Feature Extraction and Markov Random Fields , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Joshua B. Tenenbaum,et al.  Learning with Hierarchical-Deep Models , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Jungho Im,et al.  Support vector machines in remote sensing: A review , 2011 .

[29]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[30]  Antonio J. Plaza,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 1 Spectral–Spatial Classification of Hyperspectral Data Usi , 2022 .

[31]  Yoshua Bengio,et al.  On the Expressive Power of Deep Architectures , 2011, ALT.

[32]  Yicong Zhou,et al.  Extreme Learning Machine With Composite Kernels for Hyperspectral Image Classification , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[33]  C. L. Philip Chen,et al.  Hierarchical Feature Extraction With Local Neural Response for Image Recognition , 2013, IEEE Transactions on Cybernetics.

[34]  Chen Chen,et al.  Spectral–Spatial Preprocessing Using Multihypothesis Prediction for Noise-Robust Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Jon Atli Benediktsson,et al.  Spectral–Spatial Hyperspectral Image Classification With Edge-Preserving Filtering , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Hongming Zhou,et al.  Stacked Extreme Learning Machines , 2015, IEEE Transactions on Cybernetics.

[37]  Dejan J. Sobajic,et al.  Learning and generalization characteristics of the random vector Functional-link net , 1994, Neurocomputing.

[38]  Liang Xiao,et al.  Supervised Spectral–Spatial Hyperspectral Image Classification With Weighted Markov Random Fields , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[40]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Yicong Zhou,et al.  Extreme learning machine for ranking: Generalization analysis and applications , 2014, Neural Networks.

[42]  Jon Atli Benediktsson,et al.  Spectral–Spatial Hyperspectral Image Classification via Multiscale Adaptive Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Francisco Argüello,et al.  Spectral–Spatial Classification of Hyperspectral Images Using Wavelets and Extended Morphological Profiles , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[44]  Zhuowen Tu,et al.  Regularized vector field learning with sparse approximation for mismatch removal , 2013, Pattern Recognit..

[45]  Zongben Xu,et al.  Dynamic Extreme Learning Machine and Its Approximation Capability , 2013, IEEE Transactions on Cybernetics.

[46]  Qian Du,et al.  Gabor-Filtering-Based Nearest Regularized Subspace for Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[48]  Jon Atli Benediktsson,et al.  Multiple Feature Learning for Hyperspectral Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[49]  Nicolas Le Roux,et al.  Deep Belief Networks Are Compact Universal Approximators , 2010, Neural Computation.

[50]  Zhaohui Xue,et al.  Spectral–Spatial Classification of Hyperspectral Data via Morphological Component Analysis-Based Image Separation , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[51]  Jon Atli Benediktsson,et al.  Generalized Composite Kernel Framework for Hyperspectral Image Classification , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Johannes R. Sveinsson,et al.  Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles , 2008, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[53]  Tian Han,et al.  Nonlinear feature extraction of hyperspectral data based on locally linear embedding (LLE) , 2005, Proceedings. 2005 IEEE International Geoscience and Remote Sensing Symposium, 2005. IGARSS '05..

[54]  Bo Du,et al.  Semisupervised Discriminative Locally Enhanced Alignment for Hyperspectral Image Classification , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[55]  Ying-Nong Chen,et al.  Hyperspectral Image Classification Using Nearest Feature Line Embedding Approach , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[56]  Stephen Lin,et al.  Marginal Fisher Analysis and Its Variants for Human Gait Recognition and Content- Based Image Retrieval , 2007, IEEE Transactions on Image Processing.

[57]  Jon Atli Benediktsson,et al.  A Survey on Spectral–Spatial Classification Techniques Based on Attribute Profiles , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[58]  Timothy A. Warner,et al.  Kernel-based extreme learning machine for remote-sensing image classification , 2013 .

[59]  Yicong Zhou,et al.  Dimension Reduction Using Spatial and Spectral Regularized Local Discriminant Embedding for Hyperspectral Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[60]  Chen Chen,et al.  Spectral-Spatial Classification of Hyperspectral Image Based on Kernel Extreme Learning Machine , 2014, Remote. Sens..

[61]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[62]  Ji Zhao,et al.  Non-rigid visible and infrared face registration via regularized Gaussian fields criterion , 2015, Pattern Recognit..

[63]  James E. Fowler,et al.  Locality-Preserving Dimensionality Reduction and Classification for Hyperspectral Image Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.