Compressive Spectral Image Classification using 3D Coded Neural Network

Hyperspectral image classification (HIC) is an active research topic in remote sensing. However, the huge volume of three-dimensional (3D) hyperspectral images poses big challenges in data acquisition, storage, transmission and processing. To overcome these limitations, this paper develops a novel deep learning HIC approach based on the compressive measurements of coded-aperture snapshot spectral imaging (CASSI) system, without reconstructing the complete hyperspectral data cube. A new kind of deep learning strategy, namely 3D coded convolutional neural network (3D-CCNN) is proposed to efficiently solve for the HIC problem, where the hardware-based coded aperture is regarded as a pixel-wise connected network layer. An end-to-end training method is developed to jointly optimize the network parameters and the coded aperture pattern with periodic structure. The accuracy of HIC approach is effectively improved by involving the degrees of optimization freedom from the coded aperture. The superiority of the proposed method is assessed on some public hyperspectral datasets over the state-of-the-art HIC methods.

[1]  David J. Brady,et al.  Multiframe image estimation for coded aperture snapshot spectral imagers. , 2010, Applied optics.

[2]  Qian Du,et al.  High Performance Computing for Hyperspectral Remote Sensing , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  Trac D. Tran,et al.  Hyperspectral Image Classification Using Dictionary-Based Sparse Representation , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Henry Arguello,et al.  Spectral Image Classification From Optimal Coded-Aperture Compressive Measurements , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Luis Samaniego,et al.  Supervised Classification of Remotely Sensed Imagery Using a Modified $k$-NN Technique , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Henry Arguello,et al.  Spatiotemporal blue noise coded aperture design for multi-shot compressive spectral imaging. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[7]  Mercedes Eugenia Paoletti,et al.  Deep learning classifiers for hyperspectral imaging: A review , 2019 .

[8]  Antonio J. Plaza,et al.  Semisupervised Hyperspectral Image Segmentation Using Multinomial Logistic Regression With Active Learning , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Fan Zhang,et al.  Deep Convolutional Neural Networks for Hyperspectral Image Classification , 2015, J. Sensors.

[12]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[13]  Nahum Gat,et al.  Imaging spectroscopy using tunable filters: a review , 2000, SPIE Defense + Commercial Sensing.

[14]  Alistair Gorman,et al.  Generalization of the Lyot filter and its application to snapshot spectral imaging. , 2010, Optics express.

[15]  Trac D. Tran,et al.  Hyperspectral Image Classification via Kernel Sparse Representation , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Bidyut Baran Chaudhuri,et al.  HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyperspectral Image Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[17]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[18]  Patrick Lambert,et al.  3-D Deep Learning Approach for Remote Sensing Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[20]  Ashwin A. Wagadarikar,et al.  Single disperser design for coded aperture snapshot spectral imaging. , 2008, Applied optics.

[21]  M E Gehm,et al.  Single-shot compressive spectral imaging with a dual-disperser architecture. , 2007, Optics express.

[22]  Henry Arguello,et al.  Rank Minimization Code Aperture Design for Spectrally Selective Compressive Imaging , 2013, IEEE Transactions on Image Processing.

[23]  Siamak Khorram,et al.  Hierarchical maximum-likelihood classification for improved accuracies , 1997, IEEE Trans. Geosci. Remote. Sens..

[24]  Gonzalo R Arce,et al.  Multi-spectral compressive snapshot imaging using RGB image sensors. , 2015, Optics express.

[25]  Daniel L. Lau,et al.  Compressive Spectral Imaging Based on Hexagonal Blue Noise Coded Apertures , 2019, IEEE Transactions on Computational Imaging.

[26]  Dennis W Prather,et al.  Development of a digital-micromirror-device-based multishot snapshot spectral imaging system. , 2011, Optics letters.

[27]  Bo Li,et al.  Multi-scale 3D deep convolutional neural network for hyperspectral image classification , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[28]  Henry Arguello,et al.  Compressive Coded Aperture Spectral Imaging: An Introduction , 2014, IEEE Signal Processing Magazine.