Deep Metric Learning-Based Feature Embedding for Hyperspectral Image Classification

Learning from a limited number of labeled samples (pixels) remains a key challenge in the hyperspectral image (HSI) classification. To address this issue, we propose a deep metric learning-based feature embedding model, which can meet the tasks both for same- and cross-scene HSI classifications. In the first task, when only a few labeled samples are available, we employ ideas from metric learning based on deep embedding features and make a similarity learning between pairs of samples. In this case, the proposed model can learn well to compare whether two samples belong to the same class. In another task, when an HSI image (target scene) that needs to be classified is not labeled at all, the embedding model can learn from another similar HSI image (source scene) with sufficient labeled samples and then transfer to the target model by using an unsupervised domain adaptation technique, which not only employs the adversarial approach to make the embedding features from the source and target samples indistinguishable but also encourages the target scene’s embeddings to form similar clusters with the source scene one. After the domain adaptation between the HSIs of the two scenes is finished, any traditional HSI classifier can be used. In a simple manner, the nearest neighbor (NN) algorithm is selected as the classifier for the classification tasks throughout this article. The experimental results from a series of popular HSIs demonstrate the advantages of the proposed model both in the same- and cross-scene classification tasks.

[1]  Jon Atli Benediktsson,et al.  Neural Network Approaches Versus Statistical Methods in Classification of Multisource Remote Sensing Data , 1989, 12th Canadian Symposium on Remote Sensing Geoscience and Remote Sensing Symposium,.

[2]  Qingquan Li,et al.  A 3-D Gabor Phase-Based Coding and Matching Framework for Hyperspectral Imagery Classification , 2018, IEEE Transactions on Cybernetics.

[3]  Yuan Yan Tang,et al.  Dictionary Learning-Based Feature-Level Domain Adaptation for Cross-Scene Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[4]  William J. Emery,et al.  Active Learning Methods for Remote Sensing Image Classification , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Xiang Yu,et al.  Deep Metric Learning via Lifted Structured Feature Embedding , 2016 .

[6]  Xueqi Ma,et al.  $p$ -Laplacian Regularization for Scene Recognition , 2019, IEEE Transactions on Cybernetics.

[7]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Bing Liu,et al.  A semi-supervised convolutional neural network for hyperspectral image classification , 2017 .

[9]  Rong Jin,et al.  Fine-grained visual categorization via multi-stage metric learning , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Lorenzo Bruzzone,et al.  Extended profiles with morphological attribute filters for the analysis of hyperspectral data , 2010 .

[11]  Patrick Lambert,et al.  3-D Deep Learning Approach for Remote Sensing Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Lorenzo Bruzzone,et al.  A Novel Transductive SVM for Semisupervised Classification of Remote-Sensing Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Naoto Yokoya,et al.  Advances in Hyperspectral Image and Signal Processing: A Comprehensive Overview of the State of the Art , 2017, IEEE Geoscience and Remote Sensing Magazine.

[14]  Lorenzo Bruzzone,et al.  Unsupervised retraining of a maximum-likelihood classifier for the analysis of multitemporal remote sensing images , 1999, Remote Sensing.

[15]  Jiwen Lu,et al.  PCANet: A Simple Deep Learning Baseline for Image Classification? , 2014, IEEE Transactions on Image Processing.

[16]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[17]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Joydeep Ghosh,et al.  An Active Learning Approach to Hyperspectral Data Classification , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Luis Gómez-Chova,et al.  Graph Matching for Adaptation in Remote Sensing , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Johannes R. Sveinsson,et al.  Classification of hyperspectral data from urban areas based on extended morphological profiles , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Xia Xu,et al.  R-VCANet: A New Deep-Learning-Based Hyperspectral Image Classification Method , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[22]  Fan Zhang,et al.  Deep Convolutional Neural Networks for Hyperspectral Image Classification , 2015, J. Sensors.

[23]  Johannes R. Sveinsson,et al.  Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles , 2008, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[24]  Ying Li,et al.  Spectral-Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network , 2017, Remote. Sens..

[25]  Gregory R. Koch,et al.  Siamese Neural Networks for One-Shot Image Recognition , 2015 .

[26]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Antonio J. Plaza,et al.  Multiple Morphological Component Analysis Based Decomposition for Remote Sensing Image Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Jun Yu,et al.  Semantic preserving distance metric learning and applications , 2014, Inf. Sci..

[29]  Xiao Xiang Zhu,et al.  Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources , 2017, IEEE Geoscience and Remote Sensing Magazine.

[30]  Melba M. Crawford,et al.  Adaptive Classification for Hyperspectral Image Data Using Manifold Regularization Kernel Machines , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Edoardo Pasolli,et al.  An Active Learning Framework for Hyperspectral Image Classification Using Hierarchical Segmentation , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[32]  Zhenwei Shi,et al.  MugNet: Deep learning for hyperspectral image classification using limited samples , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[33]  Jon Atli Benediktsson,et al.  Nonlinear Multiple Kernel Learning With Multiple-Structure-Element Extended Morphological Profiles for Hyperspectral Image Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Jun Li,et al.  Advanced Spectral Classifiers for Hyperspectral Images: A review , 2017, IEEE Geoscience and Remote Sensing Magazine.

[35]  Lizhe Wang,et al.  SuperPCA: A Superpixelwise PCA Approach for Unsupervised Feature Extraction of Hyperspectral Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Mahesh Pal Extreme‐learning‐machine‐based land cover classification , 2008, ArXiv.

[37]  Jon Atli Benediktsson,et al.  A Survey on Spectral–Spatial Classification Techniques Based on Attribute Profiles , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[38]  John A. Richards,et al.  Remote Sensing Digital Image Analysis: An Introduction , 1999 .

[39]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[40]  Qingquan Li,et al.  Local Binary Pattern-Based Hyperspectral Image Classification With Superpixel Guidance , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Sen Jia,et al.  Convolutional neural networks for hyperspectral image classification , 2017, Neurocomputing.

[42]  Lorenzo Bruzzone,et al.  Semisupervised Classification of Hyperspectral Images by SVMs Optimized in the Primal , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[43]  M. Pal,et al.  Random forests for land cover classification , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[44]  Pedro H. O. Pinheiro,et al.  Unsupervised Domain Adaptation with Similarity Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[46]  Wei Li,et al.  Diverse Region-Based CNN for Hyperspectral Image Classification , 2018, IEEE Transactions on Image Processing.

[47]  Jiasong Zhu,et al.  Discriminative Gabor Feature Selection for Hyperspectral Image Classification , 2013, IEEE Geoscience and Remote Sensing Letters.

[48]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[49]  Qian Du,et al.  Hyperspectral Image Classification Using Deep Pixel-Pair Features , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Melba M. Crawford,et al.  Active Learning: Any Value for Classification of Remotely Sensed Data? , 2013, Proceedings of the IEEE.

[51]  Hanyun Wang,et al.  Learn Multiple-Kernel SVMs for Domain Adaptation in Hyperspectral Data , 2013, IEEE Geoscience and Remote Sensing Letters.

[52]  Xueqi Ma,et al.  Hypergraph $p$ -Laplacian Regularization for Remotely Sensed Image Recognition , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[53]  Hao Wu,et al.  Semi-Supervised Deep Learning Using Pseudo Labels for Hyperspectral Image Classification , 2018, IEEE Transactions on Image Processing.

[54]  Fei Gao,et al.  Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking , 2017, IEEE Transactions on Cybernetics.

[55]  Tao Xiang,et al.  Learning to Compare: Relation Network for Few-Shot Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[56]  Gustavo Camps-Valls,et al.  Semi-Supervised Graph-Based Hyperspectral Image Classification , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[57]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[58]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[60]  Nikolaos Doulamis,et al.  Deep supervised learning for hyperspectral data classification through convolutional neural networks , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[61]  Qingshan Liu,et al.  Cascaded Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.