Unsupervised Spatial-spectral Network Learning for Hyperspectral Compressive Snapshot Reconstruction

Hyperspectral compressive imaging takes advantage of compressive sensing theory to achieve coded aperture snapshot measurement without temporal scanning, and the entire three-dimensional spatial-spectral data is captured by a twodimensional projection during a single integration period. Its core issue is how to reconstruct the underlying hyperspectral image using compressive sensing reconstruction algorithms. Due to the diversity in the spectral response characteristics and wavelength range of different spectral imaging devices, previous works are often inadequate to capture complex spectral variations or lack the adaptive capacity to new hyperspectral imagers. In order to address these issues, we propose an unsupervised spatial-spectral network to reconstruct hyperspectral images only from the compressive snapshot measurement. The proposed network acts as a conditional generative model conditioned on the snapshot measurement, and it exploits the spatial-spectral attention module to capture the joint spatial-spectral correlation of hyperspectral images. The network parameters are optimized to make sure that the network output can closely match the given snapshot measurement according to the imaging model, thus the proposed network can adapt to different imaging settings, which can inherently enhance the applicability of the network. Extensive experiments upon multiple datasets demonstrate that our network can achieve better reconstruction results than the state-of-the-art methods.

[1]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[2]  Stephen Lin,et al.  A Prism-Mask System for Multispectral Video Acquisition. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[3]  Shree K. Nayar,et al.  Generalized Mosaicing: Wide Field of View Multispectral Imaging , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[5]  David J. Brady,et al.  Multiframe image estimation for coded aperture snapshot spectral imagers. , 2010, Applied optics.

[6]  M. Attas,et al.  Near-infrared spectroscopic imaging in art conservation: investigation of drawing constituents , 2003 .

[7]  Deyu Meng,et al.  Hyperspectral Image Classification With Markov Random Fields and a Convolutional Neural Network , 2017, IEEE Transactions on Image Processing.

[8]  Ashwin A. Wagadarikar,et al.  Single disperser design for coded aperture snapshot spectral imaging. , 2008, Applied optics.

[9]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Giljoo Nam,et al.  High-quality hyperspectral reconstruction using a spectral prior , 2017, ACM Trans. Graph..

[11]  Antonio J. Plaza,et al.  Anomaly Detection in Hyperspectral Images Based on Low-Rank and Sparse Representation , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Tao Zhang,et al.  Hyperspectral Image Reconstruction Using Deep External and Internal Learning , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Pavan K. Turaga,et al.  ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Gonzalo R. Arce,et al.  A Maximum Likelihood Approach to Least Absolute Deviation Regression , 2004, EURASIP J. Adv. Signal Process..

[15]  Xin Yuan,et al.  Generalized alternating projection based total variation minimization for compressive sensing , 2015, 2016 IEEE International Conference on Image Processing (ICIP).

[16]  M. Borengasser,et al.  Hyperspectral Remote Sensing: Principles and Applications , 2007 .

[17]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Vassilis Athitsos,et al.  lambda-Net: Reconstruct Hyperspectral Images From a Snapshot Measurement , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[20]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.

[21]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[22]  Boaz Arad,et al.  Sparse Recovery of Hyperspectral Signal from Natural RGB Images , 2016, ECCV.

[23]  Qionghai Dai,et al.  Rank Minimization for Snapshot Compressive Imaging , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[25]  Wallace M. Porter,et al.  A System Overview Of The Airborne Visible/Infrared Imaging Spectrometer (Aviris) , 1987, Optics & Photonics.

[26]  Guodong Guo,et al.  Learning Non-Locally Regularized Compressed Sensing Network With Half-Quadratic Splitting , 2020, IEEE Transactions on Multimedia.

[27]  Stephen Lin,et al.  Computational Snapshot Multispectral Cameras: Toward dynamic capture of the spectral world , 2016, IEEE Signal Processing Magazine.

[28]  Alfred S. McEwen,et al.  Spectral evidence for hydrated salts in recurring slope lineae on Mars , 2015 .

[29]  P. Mouroulis,et al.  Pushbroom imaging spectrometer with high spectroscopic data fidelity: experimental demonstration , 2000 .

[30]  M E Gehm,et al.  Single-shot compressive spectral imaging with a dual-disperser architecture. , 2007, Optics express.

[31]  Nahum Gat,et al.  Imaging spectroscopy using tunable filters: a review , 2000, SPIE Defense + Commercial Sensing.

[32]  Xin Yuan,et al.  Snapshot Compressed Sensing: Performance Bounds and Algorithms , 2018, IEEE Transactions on Information Theory.

[33]  Zongben Xu,et al.  ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[35]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[36]  Qionghai Dai,et al.  Supplementary Document : Spatial-spectral Encoded Compressive Hyperspectral Imaging , 2014 .

[37]  Tao Zhang,et al.  HyperReconNet: Joint Coded Aperture Optimization and Image Reconstruction for Compressive Hyperspectral Imaging , 2019, IEEE Transactions on Image Processing.

[38]  Xin Yuan,et al.  Compressive Hyperspectral Imaging With Side Information , 2015, IEEE Journal of Selected Topics in Signal Processing.

[39]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[40]  Ayan Chakrabarti,et al.  Statistics of real-world hyperspectral images , 2011, CVPR 2011.

[41]  Min H. Kim,et al.  Compact single-shot hyperspectral imaging using a prism , 2017, ACM Trans. Graph..

[42]  José M. Bioucas-Dias,et al.  HYCA: A New Technique for Hyperspectral Compressive Sensing , 2015, IEEE Trans. Geosci. Remote. Sens..

[43]  Qingshan Liu,et al.  Learning image compressed sensing with sub-pixel convolutional generative adversarial network , 2020, Pattern Recognit..

[44]  Dong Liu,et al.  HSCNN: CNN-Based Hyperspectral Image Recovery from Spectrally Undersampled Projections , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[45]  Michael W. Kudenov,et al.  Review of snapshot spectral imaging technologies , 2013, Optics and Precision Engineering.