Data-Driven-Based Terahertz Image Restoration

Terahertz (THz) has been widely used in radar, remote sensing, atmospheric and environmental monitoring, real-time biological information extraction, and medical diagnosis. However, the long-wavelength property of THz leads to low imaging resolution. Due to the expensive nature of experimental equipment and the difficulty of obtaining THz sources, most scientific institutions have difficulty in obtaining large amounts of the THz dataset. In addition, it is difficult for convolution neural networks (CNNs) to recover low-resolution complex images because of the phase information contained in the images. In this article, we build a THz imaging system and propose a progressive multiscale augmented neural network (PMANN) to address this problem. PMANN integrates self-attention and channel attention for image contextual feature extraction by the proposed progressive network. Compared with the complex CNN (CCNN) in private cartoon datasets, PMANN improves the quantitative metrics on peak signal-to-noise ratio (PSNR) by about 18.6%. It also obtains better visual quality than CCNN. Moreover, we publicly synthesize the first grayscale image MinistTHz dataset. Compared with SwinTransformer, PMANN improves the PSNR by 28.2%. The experimental results on the THz dataset demonstrate that the proposed model achieves better performances than the state-of-the-art (SOTA) models. Our synthetic THz dataset is open at https://github.com/szpxmu/PMANN.

[1]  Q. Yuan,et al.  Generating a long-term (2003-2020) hourly 0.25° global PM2.5 dataset via spatiotemporal downscaling of CAMS with deep learning (DeepCAMS). , 2022, The Science of the total environment.

[2]  Zhengjun Zha,et al.  Image De-raining Transformer. , 2022, IEEE transactions on pattern analysis and machine intelligence.

[3]  Mandi Luo,et al.  Memory Uncertainty Learning for Real-World Single Image Deraining , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Q. Yuan,et al.  Space-time super-resolution for satellite video: A joint framework based on multi-scale spatial-temporal transformer , 2022, Int. J. Appl. Earth Obs. Geoinformation.

[5]  Angeliki Alexiou,et al.  Directional Terahertz Communication Systems for 6G: Fact Check: A Quantitative Look , 2021, IEEE Vehicular Technology Magazine.

[6]  Syed Waqas Zamir,et al.  Restormer: Efficient Transformer for High-Resolution Image Restoration , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Z. Xiong,et al.  Multi-Scale Hybrid Fusion Network for Single Image Deraining , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Qiangqiang Yuan,et al.  Satellite Video Super-Resolution via Multiscale Deformable Convolution Alignment and Temporal Grouping Projection , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Zhongyuan Wang,et al.  Rain-Free and Residue Hand-in-Hand: A Progressive Coupled Network for Real-Time Image Deraining , 2021, IEEE Transactions on Image Processing.

[10]  Tao Mei,et al.  Contextual Transformer Networks for Visual Recognition , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jinkuan Wang,et al.  Terahertz image super-resolution based on a complex convolutional neural network. , 2021, Optics letters.

[12]  Jianmin Bao,et al.  Uformer: A General U-Shaped Transformer for Image Restoration , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Zheng-Jun Zha,et al.  Rain Streak Removal via Dual Graph Convolutional Network , 2021, AAAI.

[14]  J. Zhang,et al.  HINet: Half Instance Normalization Network for Image Restoration , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Xiao-Ping Zhang,et al.  Non-local Channel Aggregation Network for Single Image Rain Removal , 2021, Neurocomputing.

[16]  Ling Shao,et al.  Multi-Stage Progressive Image Restoration , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Chi-Wing Fu,et al.  Single-Image Real-Time Rain Removal Based on Depth-Guided Non-Local Features , 2021, IEEE Transactions on Image Processing.

[18]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[19]  S. Ravishankar,et al.  Exploiting Non-Local Priors via Self-Convolution for Highly-Efficient Image Restoration , 2020, IEEE Transactions on Image Processing.

[20]  Bin Song,et al.  A Novel Symmetry Driven Siamese Network for THz Concealed Object Verification , 2020, IEEE Transactions on Image Processing.

[21]  Pengfei Zhang,et al.  High Performance Depthwise and Pointwise Convolutions on Mobile Devices , 2020, AAAI.

[22]  Ximing Ren,et al.  Learning Non-Local Spatial Correlations To Restore Sparse 3D Single-Photon Data , 2019, IEEE Transactions on Image Processing.

[23]  Jinkuan Wang,et al.  Resolution Enhancement in Terahertz Imaging via Deconvolution , 2019, IEEE Access.

[24]  Tianyi Wang,et al.  Terahertz image super-resolution based on a deep convolutional neural network. , 2019, Applied optics.

[25]  Junjun Jiang,et al.  Edge-Enhanced GAN for Remote Sensing Image Superresolution , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Qinghua Hu,et al.  Progressive Image Deraining Networks: A Better and Simpler Baseline , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Yunchao Wei,et al.  CCNet: Criss-Cross Attention for Semantic Segmentation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  Yang Xie,et al.  Vehicle logo recognition based on overlapping enhanced patterns of oriented edge magnitudes , 2018, Comput. Electr. Eng..

[29]  Yu Qiao,et al.  ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks , 2018, ECCV Workshops.

[30]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[31]  Jaakko Lehtinen,et al.  Noise2Noise: Learning Image Restoration without Clean Data , 2018, ICML.

[32]  Nassir Navab,et al.  Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks , 2018, MICCAI.

[33]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[34]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Coloma Ballester,et al.  Affine Non-Local Means Image Denoising , 2017, IEEE Transactions on Image Processing.

[36]  Frank Hutter,et al.  SGDR: Stochastic Gradient Descent with Warm Restarts , 2016, ICLR.

[37]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[39]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Haoxiang Wang,et al.  MMW and THz images denoising based on adaptive CBM3D , 2014, Digital Image Processing.

[41]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[43]  Hairong Qi,et al.  A new perspective on terahertz image reconstruction based on linear spectral unmixing , 2008, 2008 15th IEEE International Conference on Image Processing.

[44]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[45]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[46]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .