Dual-Domain Feature Learning and Memory-Enhanced Unfolding Network for Spectral Compressive Imaging

In the coded aperture snapshot spectral compressive imaging (CASSI) system, hyperspectral image (HSI) reconstruction methods are employed to recover 3D signals from 2D compressive measurements. Among these methods, although the deep unfolding networks show the advantages of interpretability and high efficiency, they still have the limitations of insufficient feature utilization and low efficiency of information interaction between stages. To solve these problems, in this paper two well-designed techniques, dubbed as dual-domain feature learning and feature memory-enhanced module, are introduced into a deep unfolding network. The former is proposed to enhance the representation ability of deep networks, while the latter is designed to efficiently promote the cross-stage information interaction. Extensive experimental results validate the efficiency and effectiveness of the proposed method.

[1]  L. Gool,et al.  Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction , 2022, ECCV.

[2]  L. Gool,et al.  HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Yongbing Zhang,et al.  HerosNet: Hyperspectral Explicable Reconstruction and Optimal Sampling Deep Network for Snapshot Compressive Imaging , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  L. Gool,et al.  Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Wenbin Zou,et al.  SDWNet: A Straight Dilated Network with Wavelet Transformation for image Deblurring , 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[6]  Victor Lempitsky,et al.  Resolution-robust Large Mask Inpainting with Fourier Convolutions , 2021, 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[7]  Jiwen Lu,et al.  Global Filter Networks for Image Classification , 2021, NeurIPS.

[8]  Guangming Shi,et al.  Deep Gaussian Scale Mixture Prior for Spectral Compressive Imaging , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Shirin Jalali,et al.  GAP-net for Snapshot Compressive Imaging , 2020, 2012.08364.

[10]  Shensheng Han,et al.  Deep plug-and-play priors for spectral snapshot compressive imaging , 2020, Photonics Research.

[11]  Xin Yuan,et al.  End-to-End Low Cost Compressive Spectral Imaging with Spatial-Spectral Self-Attention , 2020, ECCV.

[12]  Hua Huang,et al.  DNU: Deep Non-Local Unrolling for Computational Spectral Imaging , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Stefano Soatto,et al.  FDA: Fourier Domain Adaptation for Semantic Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  V. Athitsos,et al.  lambda-Net: Reconstruct Hyperspectral Images From a Snapshot Measurement , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[15]  Ying Fu,et al.  Hyperspectral Image Reconstruction Using a Deep Spatial-Spectral Prior , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Qionghai Dai,et al.  Rank Minimization for Snapshot Compressive Imaging , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Chang-Su Kim,et al.  Single-Image Depth Estimation Based on Fourier Domain Analysis , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Guangming Shi,et al.  Adaptive Nonlocal Sparse Representation for Dual-Camera Compressive Hyperspectral Imaging , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Heikki Huttunen,et al.  Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[20]  Xin Yuan,et al.  Generalized alternating projection based total variation minimization for compressive sensing , 2015, 2016 IEEE International Conference on Image Processing (ICIP).

[21]  Gonzalo R. Arce,et al.  Compressive Hyperspectral Imaging via Approximate Message Passing , 2015, IEEE Journal of Selected Topics in Signal Processing.

[22]  E. Todeva.  Networks , 2007 .

[23]  Xin Yuan,et al.  Supplementary Material for “Self-supervised Neural Networks for Spectral Snapshot Compressive Imaging” , 2021 .

[24]  Yan Wang,et al.  Deep Residual Fourier Transformation for Single Image Deblurring , 2021, ArXiv.

[25]  Yadong Mu,et al.  Fast Fourier Convolution , 2020, NeurIPS.