Caps-TripleGAN: GAN-Assisted CapsNet for Hyperspectral Image Classification

The increase in the spectral and spatial information of hyperspectral imagery poses challenges in classification due to the fact that spectral bands are highly correlated, training samples may be limited, and high resolution may increase intraclass difference and interclass similarity. In this paper, in order to better handle these problems, a Caps-TripleGAN framework is proposed by exploring the 1-D structure triple generative adversarial network (TripleGAN) for sample generation and integrating CapsNet for hyperspectral image classification. Moreover, spatial information is utilized to verify the learning capacity and discriminative ability of the Caps-TripleGAN framework. The experimental results obtained with three real hyperspectral data sets confirm that the proposed method outperforms most of the state-of-the-art methods.

[1]  Yang Wang,et al.  MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification , 2016, IEEE Geoscience and Remote Sensing Letters.

[2]  Lorenzo Bruzzone,et al.  A Composite Semisupervised SVM for Classification of Hyperspectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[3]  Rupert Müller,et al.  On the possibility of conditional adversarial networks for multi-sensor image matching , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[4]  Volodymyr I. Ponomaryov,et al.  Feature extraction scheme for a textural hyperspectral image classification using gray-scaled HSV and NDVI image features vectors fusion , 2016, 2016 International Conference on Electronics, Communications and Computers (CONIELECOMP).

[5]  Zhetao Li,et al.  Generative Adversarial Networks for Change Detection in Multispectral Imagery , 2017, IEEE Geoscience and Remote Sensing Letters.

[6]  Qingshan Liu,et al.  Cascaded Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Jing Zhang,et al.  An Efficient Hyperspectral Image Retrieval Method: Deep Spectral-Spatial Feature Extraction with DCGAN and Dimensionality Reduction Using t-SNE-Based NM Hashing , 2018, Remote. Sens..

[8]  Peng Yan-bin.  Hyperspectral Data Classification with Spectral and Texture Features by Co-training Algorithm , 2012 .

[9]  Filiberto Pla,et al.  Capsule Networks for Hyperspectral Image Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Dan Hu,et al.  Semisupervised Hyperspectral Image Classification Based on Generative Adversarial Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[11]  Xiaodong Mu,et al.  Remote sensing image scene classification based on generative adversarial networks , 2018 .

[12]  Naoto Yokoya,et al.  IMG2DSM: Height Simulation From Single Imagery Using Conditional Generative Adversarial Net , 2018, IEEE Geoscience and Remote Sensing Letters.

[13]  Xiang Bai,et al.  An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  Qian Du,et al.  Local Binary Patterns and Extreme Learning Machine for Hyperspectral Imagery Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[17]  Ujjwal Maulik,et al.  Learning with transductive SVM for semisupervised pixel classification of remote sensing imagery , 2013 .

[18]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Ying Li,et al.  Spectral-spatial classification of hyperspectral imagery based on deep convolutional network , 2016, 2016 International Conference on Orange Technologies (ICOT).

[20]  Adrian J. Brown.  Spectral curve fitting for automatic hyperspectral data analysis , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Lin Zhu,et al.  Generative Adversarial Networks for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Gustavo Camps-Valls,et al.  Semi-Supervised Graph-Based Hyperspectral Image Classification , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Simon J. Hook,et al.  HYDROTHERMAL FORMATION OF CLAY-CARBONATE ALTERATION ASSEMBLAGES IN THE , 2010, 1402.1150.

[24]  Jun Zhu,et al.  Triple Generative Adversarial Nets , 2017, NIPS.

[25]  Jon Atli Benediktsson,et al.  Spectral–Spatial Hyperspectral Image Classification With Edge-Preserving Filtering , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Hojjat Adeli,et al.  Supervised Deep Restricted Boltzmann Machine for Estimation of Concrete , 2017 .

[27]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[28]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[29]  Janne Heikkilä,et al.  Towards Virtual H&E Staining of Hyperspectral Lung Histology Images Using Conditional Generative Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[30]  S. Dunagan,et al.  The MARTE VNIR imaging spectrometer experiment: design and analysis. , 2008, Astrobiology.

[31]  Ting Yuan,et al.  Hyperspectral Image Classification with Capsule Network Using Limited Training Samples , 2018, Sensors.

[32]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[33]  Shutao Li,et al.  Hyperspectral Image Classification With Deep Feature Fusion Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Jun Li,et al.  Kernel Low-Rank Multitask Learning in Variational Mode Decomposition Domain for Multi-/Hyperspectral Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Yang Wang,et al.  Synthesizing remote sensing images by conditional adversarial networks , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[36]  Lei Zhang,et al.  CatGAN: Coupled Adversarial Transfer for Domain Generation , 2017, ArXiv.

[37]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[38]  Xing Zhao,et al.  Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[39]  Pengfei Liu,et al.  Spatial-Hessian-Feature-Guided Variational Model for Pan-Sharpening , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Dong Ni,et al.  Multispectral Image Alignment With Nonlinear Scale-Invariant Keypoint and Enhanced Local Feature Matrix , 2015, IEEE Geoscience and Remote Sensing Letters.

[41]  Bo Li,et al.  Multi-scale 3D deep convolutional neural network for hyperspectral image classification , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[42]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[43]  Wei Liu,et al.  Semi-Supervised Classification of Hyperspectral Data Based on Generative Adversarial Networks and Neighborhood Majority Voting , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[44]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[45]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[47]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[48]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[49]  Masashi Matsuoka,et al.  Filmy Cloud Removal on Satellite Imagery with Multispectral Conditional Generative Adversarial Nets , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[50]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[51]  Thomas Burger,et al.  PerTurbo Manifold Learning Algorithm for Weakly Labeled Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.