Hyperspectral Image Classification Based on Parameter-Optimized 3D-CNNs Combined with Transfer Learning and Virtual Samples

Recent research has shown that spatial-spectral information can help to improve the classification of hyperspectral images (HSIs). Therefore, three-dimensional convolutional neural networks (3D-CNNs) have been applied to HSI classification. However, a lack of HSI training samples restricts the performance of 3D-CNNs. To solve this problem and improve the classification, an improved method based on 3D-CNNs combined with parameter optimization, transfer learning, and virtual samples is proposed in this paper. Firstly, to optimize the network performance, the parameters of the 3D-CNN of the HSI to be classified (target data) are adjusted according to the single variable principle. Secondly, in order to relieve the problem caused by insufficient samples, the weights in the bottom layers of the parameter-optimized 3D-CNN of the target data can be transferred from another well trained 3D-CNN by a HSI (source data) with enough samples and the same feature space as the target data. Then, some virtual samples can be generated from the original samples of the target data to further alleviate the lack of HSI training samples. Finally, the parameter-optimized 3D-CNN with transfer learning can be trained by the training samples consisting of the virtual and the original samples. Experimental results on real-world hyperspectral satellite images have shown that the proposed method has great potential prospects in HSI classification.

[1]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[2]  Guizhong Liu,et al.  An efficient macroblock-based diverse and flexible prediction modes selection for hyperspectral images coding , 2010, Signal Process. Image Commun..

[3]  Meng Wang,et al.  Active learning in multimedia annotation and retrieval: A survey , 2011, TIST.

[4]  Gwanggil Jeon,et al.  Hyperspectral image compression based on lapped transform and Tucker decomposition , 2015, Signal Process. Image Commun..

[5]  D. B. Megherbi,et al.  A robust multi-stage information-theoretic approach for registration of partially overlapped hyperspectral aerial imagery and evaluation in the presence of system noise , 2017, Signal Process. Image Commun..

[6]  Shuying Li,et al.  Structure Preserving Transfer Learning for Unsupervised Hyperspectral Image Classification , 2017, IEEE Geoscience and Remote Sensing Letters.

[7]  M. Bauer,et al.  Airborne hyperspectral remote sensing to assess spatial distribution of water quality characteristics in large rivers: the Mississippi River and its tributaries in Minnesota. , 2013 .

[8]  Jon Atli Benediktsson,et al.  A spatial-spectral kernel-based approach for the classification of remote-sensing images , 2012, Pattern Recognit..

[9]  Weifeng Liu,et al.  Multiview Canonical Correlation Analysis Networks for Remote Sensing Image Recognition , 2017, IEEE Geoscience and Remote Sensing Letters.

[10]  Qingquan Li,et al.  Three-dimensional local binary patterns for hyperspectral imagery classification , 2017, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[11]  Qian Du,et al.  Hyperspectral Classification Based on Texture Feature Enhancement and Deep Belief Networks , 2018, Remote. Sens..

[12]  Ying Li,et al.  Spectral-Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network , 2017, Remote. Sens..

[13]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Gangyao Kuang,et al.  SAR Target Recognition via Local Sparse Representation of Multi-Manifold Regularized Low-Rank Approximation , 2018, Remote. Sens..

[15]  Juha Suomalainen,et al.  Generation of Spectral–Temporal Response Surfaces by Combining Multispectral Satellite and Hyperspectral UAV Imagery for Precision Agriculture Applications , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Qingshan Liu,et al.  Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Xiangtao Zheng,et al.  Spectral–Spatial Kernel Regularized for Hyperspectral Image Denoising , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Jonathan Cheung-Wai Chan,et al.  Learning and Transferring Deep Joint Spectral–Spatial Features for Hyperspectral Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Zhiming Luo,et al.  Spectral–Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Meng Wang,et al.  Unified Video Annotation via Multigraph Learning , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Jun Li,et al.  3D-Gabor Inspired Multiview Active Learning for Spectral-Spatial Hyperspectral Image Classification , 2018, Remote. Sens..

[23]  Weifeng Liu,et al.  Multiview dimension reduction via Hessian multiset canonical correlations , 2018, Inf. Fusion.

[24]  Bin Wang,et al.  Graph-based deep Convolutional networks for Hyperspectral image classification , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[25]  Bo Du,et al.  Slow Feature Analysis for Change Detection in Multispectral Imagery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[27]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[28]  Caroline Fossati,et al.  Reduction of Signal-Dependent Noise From Hyperspectral Images for Target Detection , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Dapeng Tao,et al.  Manifold regularized kernel logistic regression for web image annotation , 2013, Neurocomputing.

[30]  Qian Du,et al.  Hyperspectral Image Spatial Super-Resolution via 3D Full Convolutional Neural Network , 2017, Remote. Sens..

[31]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Stefan Winkler,et al.  Ground-based image analysis: A tutorial on machine-learning techniques and applications , 2016, IEEE Geoscience and Remote Sensing Magazine.

[33]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[34]  Barnali M. Dixon,et al.  Application of Support Vector Machines for Landuse Classification Using High-Resolution RapidEye Images: A Sensitivity Analysis , 2015 .

[35]  Zongxu Pan,et al.  Transfer Learning with Deep Convolutional Neural Network for SAR Target Classification with Limited Labeled Data , 2017, Remote. Sens..

[36]  Antonio J. Plaza,et al.  Robust Matrix Discriminative Analysis for Feature Extraction From Hyperspectral Images , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[37]  Wen Liu,et al.  Learning Change from Synthetic Aperture Radar Images: Performance Evaluation of a Support Vector Machine to Detect Earthquake and Tsunami-Induced Changes , 2016, Remote. Sens..

[38]  Alessandro Chiuso,et al.  The harmonic analysis of kernel functions , 2017, Autom..

[39]  Xing Zhao,et al.  Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[40]  Naoto Yokoya,et al.  Potential of Resolution-Enhanced Hyperspectral Data for Mineral Mapping Using Simulated EnMAP and Sentinel-2 Images , 2016, Remote. Sens..

[41]  A-Li Luo,et al.  Restricted Boltzmann machine: a non-linear substitute for PCA in spectral processing , 2015 .

[42]  Lorenzo Bruzzone,et al.  Domain Adaptation for the Classification of Remote Sensing Data: An Overview of Recent Advances , 2016, IEEE Geoscience and Remote Sensing Magazine.

[43]  Caroline Fossati,et al.  Denoising of Hyperspectral Images Using the PARAFAC Model and Statistical Performance Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[45]  Pabitra Mitra,et al.  BASS Net: Band-Adaptive Spectral-Spatial Feature Learning Neural Network for Hyperspectral Image Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Francesca Bovolo,et al.  Updating Land-Cover Maps by Classification of Image Time Series: A Novel Change-Detection-Driven Transfer Learning Approach , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[47]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[48]  D. Anguita,et al.  K-fold generalization capability assessment for support vector classifiers , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[49]  Sasa Nikolic,et al.  Multi-channel descriptors and ensemble of Extreme Learning Machines for classification of remote sensing images , 2015, Signal Process. Image Commun..

[50]  Gang Wang,et al.  Learning Contextual Dependencies with Convolutional Hierarchical Recurrent Neural Networks , 2015, ArXiv.

[51]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[52]  Ke Lu,et al.  $p$-Laplacian Regularized Sparse Coding for Human Activity Recognition , 2016, IEEE Transactions on Industrial Electronics.

[53]  Xiao Xiang Zhu,et al.  A Self-Improving Convolution Neural Network for the Classification of Hyperspectral Data , 2016, IEEE Geoscience and Remote Sensing Letters.

[54]  R. Valentini,et al.  Biodiversity Mapping in a Tropical West African Forest with Airborne Hyperspectral Data , 2014, PloS one.

[55]  Qingshan Liu,et al.  Matrix-Based Discriminant Subspace Ensemble for Hyperspectral Image Spatial–Spectral Feature Fusion , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[56]  J R Fielding,et al.  Spiral CT in the evaluation of flank pain: overall accuracy and feature analysis. , 1997, Journal of computer assisted tomography.

[57]  Matthew D. Zeiler ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[58]  F. M. Lacar,et al.  Use of hyperspectral imagery for mapping grape varieties in the Barossa Valley, South Australia , 2001, IGARSS 2001. Scanning the Present and Resolving the Future. Proceedings. IEEE 2001 International Geoscience and Remote Sensing Symposium (Cat. No.01CH37217).

[59]  Geunseop Lee,et al.  Fast computation of the compressive hyperspectral imaging by using alternating least squares methods , 2018, Signal Process. Image Commun..

[60]  Bo Du,et al.  Spectral–Spatial Unified Networks for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[61]  Gui-Song Xia,et al.  Learning High-level Features for Satellite Image Classification With Limited Labeled Samples , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[62]  Weizhou Zhao,et al.  Secure Fingerprint Recognition Based on Frobenius Norm , 2012, 2012 International Conference on Computer Science and Electronics Engineering.

[63]  Yi Shen,et al.  3D gray-gradient-gradient tensor field feature for hyperspectral image classification , 2015, 2015 10th International Conference on Communications and Networking in China (ChinaCom).