SWDGAN: GAN-based sampling and whole image denoising network for compressed sensing image reconstruction

Abstract. We propose a framework named sampling and whole image denoising network based on generative adversarial network (GAN) for compressed sensing image reconstruction (SWDGAN) to reconstruct natural image from compressed sensing (CS) measurements. This work is devoted to balancing the performance of reconstruction quality, practicability, and running efficiency. Different from the recent deep learning reconstruction networks, we further enhance the feature representation and remove the blocking artifacts by introducing a whole image dense residual denoising module without affecting running efficiency. To improve the flexibility of sampling process and the practicability of our algorithm, a fully connected network without bias is applied in the sampling process, whose weight can be extracted separately and used as a measurement matrix. In this way, the measurements can be obtained by matrix multiplication in multiple running environments, not limited to deep learning framework. Besides, the sampling network can also improve the reconstruction quality even at low sampling ratios. Moreover, we remove batch normalization (BN) layer of reconstruction network to avoid the influence of BN artifact on reconstruction image. The experimental results illustrate that our method outperforms the most advanced traditional methods and deep learning-based methods in terms of reconstruction quality and running time.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Yao Zhao,et al.  FilterNet: Adaptive Information Filtering Network for Accurate and Fast Image Super-Resolution , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Jun Jiang,et al.  Rapid compressed sensing reconstruction: A semi-tensor product approach , 2020, Inf. Sci..

[5]  Anhong Wang,et al.  DCSN-Cast: Deep compressed sensing network for wireless video multicast , 2019, Signal Process. Image Commun..

[6]  Tian Zhou,et al.  Image super resolution via nonlocal and second-order feature fusion network , 2020, J. Electronic Imaging.

[7]  Feng Jiang,et al.  Scalable Convolutional Neural Network for Image Compressed Sensing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Pavan K. Turaga,et al.  ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Pavan Turaga,et al.  Convolutional Neural Networks for Noniterative Reconstruction of Compressively Sensed Images , 2017, IEEE Transactions on Computational Imaging.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Wen Gao,et al.  Group-Based Sparse Representation for Image Restoration , 2014, IEEE Transactions on Image Processing.

[12]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[13]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[14]  James E. Fowler,et al.  Block Compressed Sensing of Images Using Directional Transforms , 2010, 2010 Data Compression Conference.

[15]  Yali Qin,et al.  Fractional-order total variation algorithm with nonlocal self-similarity for image reconstruction , 2021, J. Electronic Imaging.

[16]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.

[17]  Huihui Song,et al.  Conditional generative adversarial network with densely-connected residual learning for single image super-resolution , 2020, Multim. Tools Appl..

[18]  Xinbo Gao,et al.  Soft Semantic Representation for Cross-Domain Face Recognition , 2021, IEEE Transactions on Information Forensics and Security.

[19]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[20]  En Zhang,et al.  Subdata image encryption scheme based on compressive sensing and vector quantization , 2020, Neural Computing and Applications.

[21]  Yangyang Li,et al.  Hyperspectral image compressive reconstruction with low-rank tensor constraint , 2020, J. Electronic Imaging.

[22]  Wei Chen,et al.  An image compressed sensing algorithm based on adaptive nonlinear network , 2020 .

[23]  Sridha Sridharan,et al.  Semantic Consistency and Identity Mapping Multi-Component Generative Adversarial Network for Person Re-Identification , 2020, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[24]  Mohamed Abdel-Nasser,et al.  Compressive sensing MRI reconstruction using empirical wavelet transform and grey wolf optimizer , 2018, Neural Computing and Applications.

[25]  Lu Gan Block Compressed Sensing of Natural Images , 2007, 2007 15th International Conference on Digital Signal Processing.

[26]  Yongdong Zhang,et al.  DR2-Net: Deep Residual Reconstruction Network for Image Compressive Sensing , 2017, Neurocomputing.

[27]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[28]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[29]  Qingshan Liu,et al.  Learning image compressed sensing with sub-pixel convolutional generative adversarial network , 2020, Pattern Recognit..

[30]  Xiaoming Yuan,et al.  Alternating algorithms for total variation image reconstruction from random projections , 2012 .

[31]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[33]  Yiran Chen,et al.  An efficient chaos-based image compression and encryption scheme using block compressive sensing and elementary cellular automata , 2018, Neural Computing and Applications.

[34]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Chen Chen,et al.  Compressed-sensing recovery of images and video using multihypothesis predictions , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[36]  Shuicheng Yan,et al.  Image Classification With Tailored Fine-Grained Dictionaries , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Lei Zhang,et al.  Image reconstruction with locally adaptive sparsity and nonlocal robust regularization , 2012, Signal Process. Image Commun..

[38]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[39]  Rui-Sheng Jia,et al.  Underwater image enhancement method based on the generative adversarial network , 2021, J. Electronic Imaging.

[40]  Bernard Ghanem,et al.  ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[43]  Guangming Shi,et al.  ROI-CSNet: Compressive sensing network for ROI-aware image recovery , 2019, Signal Process. Image Commun..

[44]  Piotr Indyk,et al.  A fast approximation algorithm for tree-sparse recovery , 2014, 2014 IEEE International Symposium on Information Theory.