Effective Training of Deep Convolutional Neural Networks for Hyperspectral Image Classification through Artificial Labeling

Hyperspectral imaging is a rich source of data, allowing for a multitude of effective applications. However, such imaging remains challenging because of large data dimension and, typically, a small pool of available training examples. While deep learning approaches have been shown to be successful in providing effective classification solutions, especially for high dimensional problems, unfortunately they work best with a lot of labelled examples available. The transfer learning approach can be used to alleviate the second requirement for a particular dataset: first the network is pre-trained on some dataset with large amount of training labels available, then the actual dataset is used to fine-tune the network. This strategy is not straightforward to apply with hyperspectral images, as it is often the case that only one particular image of some type or characteristic is available. In this paper, we propose and investigate a simple and effective strategy of transfer learning that uses unsupervised pre-training step without label information. This approach can be applied to many of the hyperspectral classification problems. The performed experiments show that it is very effective at improving the classification accuracy without being restricted to a particular image type or neural network architecture. The experiments were carried out on several deep neural network architectures and various sizes of labeled training sets. The greatest improvement in overall accuracy on the Indian Pines and Pavia University datasets is over 21 and 13 percentage points, respectively. An additional advantage of the proposed approach is the unsupervised nature of the pre-training step, which can be done immediately after image acquisition, without the need of the potentially costly expert’s time.

[1]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[2]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Przemysław Głomb,et al.  Semi-supervised hyperspectral classification from a small number of training samples using a co-training approach , 2016 .

[4]  Sebastian Ruder,et al.  Universal Language Model Fine-tuning for Text Classification , 2018, ACL.

[5]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[6]  Geoffrey E. Hinton Deep belief networks , 2009, Scholarpedia.

[7]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[8]  Jianjun Lei,et al.  Joint spatial-spectral hyperspectral image classification based on convolutional neural network , 2020, Pattern Recognit. Lett..

[9]  Richard J. Murphy,et al.  Pretraining for Hyperspectral Convolutional Neural Network Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[11]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[12]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[13]  Stefano Ermon,et al.  Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping , 2015, AAAI.

[14]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[15]  Qian Du,et al.  An efficient semi-supervised classification approach for hyperspectral imagery , 2014 .

[16]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[17]  Sen Jia,et al.  Convolutional neural networks for hyperspectral image classification , 2017, Neurocomputing.

[18]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[19]  Bo Du,et al.  Unsupervised transfer learning for target detection from hyperspectral images , 2013, Neurocomputing.

[20]  Stefan Winkler,et al.  Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning , 2015, ICMI.

[21]  Quoc V. Le,et al.  Semi-supervised Sequence Learning , 2015, NIPS.

[22]  Chu Zhang,et al.  Cost-sensitive stacked sparse auto-encoder models to detect striped stem borer infestation on rice based on hyperspectral imaging , 2019, Knowl. Based Syst..

[23]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[24]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Jon Atli Benediktsson,et al.  Segmentation and Classification of Hyperspectral Images Using Minimum Spanning Forest Grown From Automatically Selected Markers , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  A. Plaza,et al.  Semi-supervised classification of urban hyperspectral data using spectral unmixing concepts , 2013, Joint Urban Remote Sensing Event 2013.

[27]  Heesung Kwon,et al.  Going Deeper With Contextual CNN for Hyperspectral Image Classification , 2016, IEEE Transactions on Image Processing.

[28]  Hui Zhou,et al.  Multi-branch fusion network for hyperspectral image classification , 2019, Knowl. Based Syst..

[29]  Bing Liu,et al.  Supervised Deep Feature Extraction for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Z. Jane Wang,et al.  Deep Transfer Learning for Hyperspectral Image Classification , 2018, 2018 IEEE 20th International Workshop on Multimedia Signal Processing (MMSP).

[31]  Xintao Hu,et al.  Weakly supervised target detection in remote sensing images based on transferred deep features and negative bootstrapping , 2016, Multidimens. Syst. Signal Process..

[32]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[33]  Przemysław Głomb,et al.  A Spatial-Spectral Disagreement-Based Sample Selection With an Application to Hyperspectral Data Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[34]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[35]  Jun Li,et al.  Advanced Spectral Classifiers for Hyperspectral Images: A review , 2017, IEEE Geoscience and Remote Sensing Magazine.

[36]  Hao Wu,et al.  Convolutional Recurrent Neural Networks forHyperspectral Data Classification , 2017, Remote. Sens..

[37]  Hui Lu,et al.  A deep information based transfer learning method to detect annual urban dynamics of Beijing and Newyork from 1984–2016 , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[38]  Lichao Mou,et al.  Learning a Transferable Change Rule from a Recurrent Neural Network for Land Cover Change Detection , 2016, Remote. Sens..

[39]  Jon Atli Benediktsson,et al.  Recent Advances in Techniques for Hyperspectral Image Processing , 2009 .

[40]  Cheng Shi,et al.  Multi-scale hierarchical recurrent neural networks for hyperspectral image classification , 2018, Neurocomputing.

[41]  Zhenwei Shi,et al.  MugNet: Deep learning for hyperspectral image classification using limited samples , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[42]  Przemysław Głomb,et al.  Automatic pigment identification from hyperspectral data , 2018 .

[43]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[44]  Bing Tu,et al.  Multiple convolutional layers fusion framework for hyperspectral image classification , 2019, Neurocomputing.

[45]  Ying Wang,et al.  Semi-supervised classification for hyperspectral imagery based on spatial-spectral Label Propagation , 2014 .

[46]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[47]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[48]  Xiangtao Zheng,et al.  Hyperspectral Image Superresolution by Transfer Learning , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[49]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[50]  Chein-I. Chang Hyperspectral Imaging: Techniques for Spectral Detection and Classification , 2003 .

[51]  Fathi M. Salem,et al.  Gate-variants of Gated Recurrent Unit (GRU) neural networks , 2017, 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS).

[52]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[53]  Jonathan Cheung-Wai Chan,et al.  Semi-Supervised Deep Learning Classification for Hyperspectral Image Based on Dual-Strategy Sample Selection , 2018, Remote. Sens..

[54]  Hao Wu,et al.  Semi-Supervised Deep Learning Using Pseudo Labels for Hyperspectral Image Classification , 2018, IEEE Transactions on Image Processing.

[55]  Nir Shavit,et al.  Deep Learning is Robust to Massive Label Noise , 2017, ArXiv.

[56]  Lei Zhang,et al.  A deep manifold learning approach for spatial-spectral classification with limited labeled training samples , 2019, Neurocomputing.

[57]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[58]  Bo Du,et al.  Hyperspectral image classification via a random patches network , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[59]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[60]  Zhi-Hua Zhou,et al.  A New Analysis of Co-Training , 2010, ICML.

[61]  Wei Li,et al.  Transferred Deep Learning for Anomaly Detection in Hyperspectral Imagery , 2017, IEEE Geoscience and Remote Sensing Letters.

[62]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[64]  Yu Zhang,et al.  Guided filter based Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IIKI.

[65]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[66]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.