A Simple Adaptive Unfolding Network for Hyperspectral Image Reconstruction

We present a simple, efficient, and scalable unfolding network, SAUNet , to simplify the network design with an adaptive alternate optimization framework for hyperspectral image ( HSI ) reconstruction. SAUNet customizes a Residual Adaptive ADMM Framework ( R2ADMM ) to connect each stage of the network via a group of learnable parameters to promote the usage of mask prior, which greatly stabilizes training and solves the accuracy degradation issue. Additionally, we introduce a simple convolutional modulation block ( CMB ), which leads to efficient training, easy scale-up, and less computation. Coupling these two designs, SAUNet can be scaled to non-trivial 13 stages with continuous improvement. Without bells and whistles, SAUNet improves both performance and speed compared with the previous state-of-the-art counterparts, which makes it feasible for practical high-resolution HSI reconstruction scenarios. We set new records on CAVE and KAIST HSI reconstruction benchmarks. Code and models are available at https://github.com/hustvl/SAUNet.

[1]  Hao Zhang,et al.  Recurrent Neural Networks for Snapshot Compressive Imaging , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Cheng Lu,et al.  Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition , 2022, IEEE transactions on pattern analysis and machine intelligence.

[3]  L. Gool,et al.  Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging , 2022, NeurIPS.

[4]  Jianfeng Gao,et al.  Focal Modulation Networks , 2022, NeurIPS.

[5]  Jian Sun,et al.  Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  L. Gool,et al.  Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction , 2022, ECCV.

[7]  L. Gool,et al.  HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Trevor Darrell,et al.  A ConvNet for the 2020s , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  L. Gool,et al.  Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Ross Wightman,et al.  ResNet strikes back: An improved training procedure in timm , 2021, ArXiv.

[11]  Guangming Shi,et al.  Deep Gaussian Scale Mixture Prior for Spectral Compressive Imaging , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[13]  Yonina C. Eldar,et al.  Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing , 2019, IEEE Signal Processing Magazine.

[14]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[15]  Shirin Jalali,et al.  GAP-net for Snapshot Compressive Imaging , 2020, 2012.08364.

[16]  Xin Yuan,et al.  End-to-End Low Cost Compressive Spectral Imaging with Spatial-Spectral Self-Attention , 2020, ECCV.

[17]  Yuhong He,et al.  Recent Advances of Hyperspectral Imaging Technology and Applications in Agriculture , 2020, Remote. Sens..

[18]  Zhenming Yu,et al.  Snapshot multispectral endomicroscopy. , 2020, Optics letters.

[19]  Hua Huang,et al.  DNU: Deep Non-Local Unrolling for Computational Spectral Imaging , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Xin Yuan,et al.  Deep learning for video compressive sensing , 2020, APL Photonics.

[21]  Zheng Shou,et al.  Deep Tensor ADMM-Net for Snapshot Compressive Imaging , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[22]  Vassilis Athitsos,et al.  lambda-Net: Reconstruct Hyperspectral Images From a Snapshot Measurement , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[23]  Qionghai Dai,et al.  Rank Minimization for Snapshot Compressive Imaging , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Qinghua Liu,et al.  Linearized ADMM for Nonconvex Nonsmooth Optimization With Convergence Analysis , 2017, IEEE Access.

[25]  Masatoshi Okutomi,et al.  Remote Heart Rate Measurement from RGB-NIR Video Based on Spatial and Spectral Face Patch Selection , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[26]  Xiangyu Wang,et al.  Proximal Alternating Direction Network: A Globally Converged Deep Unrolling Framework , 2017, AAAI.

[27]  Giljoo Nam,et al.  High-quality hyperspectral reconstruction using a spectral prior , 2017, ACM Trans. Graph..

[28]  Zheng Xu,et al.  Adaptive Relaxed ADMM: Convergence Theory and Practical Implementation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Zheng Xu,et al.  Adaptive ADMM with Spectral Penalty Parameter Selection , 2016, AISTATS.

[30]  Gordon Wetzstein,et al.  ProxImaL , 2016, ACM Trans. Graph..

[31]  Tianqi Chen,et al.  Training Deep Nets with Sublinear Memory Cost , 2016, ArXiv.

[32]  Antonio J. Plaza,et al.  Anomaly Detection in Hyperspectral Images Based on Low-Rank and Sparse Representation , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Xin Yuan,et al.  Generalized alternating projection based total variation minimization for compressive sensing , 2015, 2016 IEEE International Conference on Image Processing (ICIP).

[34]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[35]  Tieniu Tan,et al.  Half-Quadratic-Based Iterative Minimization for Robust Sparse Representation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[37]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[38]  Min H. Kim,et al.  3D imaging spectroscopy for measuring hyperspectral patterns on solid objects , 2012, ACM Trans. Graph..

[39]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[40]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[41]  David J. Brady,et al.  Multiframe image estimation for coded aperture snapshot spectral imagers. , 2010, Applied optics.

[42]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[43]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[44]  Ashwin A. Wagadarikar,et al.  Single disperser design for coded aperture snapshot spectral imaging. , 2008, Applied optics.

[45]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.