Real-Time Ultrasound Image Despeckling Using Mixed-Attention Mechanism Based Residual UNet

Ultrasound imaging has been widely used for clinical diagnosis. However, the inherent speckle noise will degrade the quality of ultrasound images. Existing despeckling methods cannot deliver sufficient speckle reduction and preserve image details well at high noise corruption and they cannot realize real-time ultrasound image denoising. With the popularity of deep learning, supervised learning for image denoising has recently attracted considerable attention. In this paper, we have proposed a novel residual UNet using mixed-attention mechanism (MARU) for real-time ultrasound image despeckling. In view of the signal-dependent characteristics of speckle noise, we have designed an encoder-decoder network to reconstruct the despeckled image by extracting features from the noisy image. Furthermore, a lightweight mixed-attention block is proposed to effectively enhance the image features and suppress some speckle noise during the encoding phase by using separation and re-fusion strategy for channel and spatial attention. Besides, we have graded the speckle noise levels with a certain interval and designed an algorithm to estimate the noise levels for despeckling real ultrasound images. Experiments have been done on the natural images, the synthetic image, the image simulated using Field II and the real ultrasound images. Compared with existing despeckling methods, the proposed network has achieved the state-of-the-art despeckling performance in terms of subjective human vision and such quantitative indexes as peak signal to noise ratio (PSNR), structural similarity (SSIM), equivalent number of looks (ENL) and contrast-to-noise ratio (CNR).

[1]  Yaxing Wang,et al.  S-UNet: A Bridge-Style U-Net Framework With a Saliency Mechanism for Retinal Vessel Segmentation , 2019, IEEE Access.

[2]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[3]  Shilian Zheng,et al.  Fusion Methods for CNN-Based Automatic Modulation Classification , 2019, IEEE Access.

[4]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[5]  Olaf Ronneberger,et al.  Invited Talk: U-Net Convolutional Networks for Biomedical Image Segmentation , 2017, Bildverarbeitung für die Medizin.

[6]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[7]  Basit Raza,et al.  Multi-View Feature Fusion Based Four Views Model for Mammogram Classification Using Convolutional Neural Network , 2019, IEEE Access.

[8]  Vishal M. Patel,et al.  SAR Image Despeckling Using a Convolutional Neural Network , 2017, IEEE Signal Processing Letters.

[9]  Yang Zhao,et al.  Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000 , 2018 .

[10]  C. Lindsay Anderson,et al.  A New Wavelet Denoising Method for Selecting Decomposition Levels and Noise Thresholds , 2016, IEEE Access.

[11]  Tong Fang,et al.  Ultrasound-Specific Segmentation via Decorrelation and Statistical Region-Based Active Contours , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[12]  Danni Ai,et al.  Local statistics and non-local mean filter for speckle noise reduction in medical ultrasound image , 2016, Neurocomputing.

[13]  Alexander A. Sawchuk,et al.  Adaptive restoration of images with speckle , 1987, IEEE Trans. Acoust. Speech Signal Process..

[14]  Maciej Wojtkowski,et al.  Efficient Reduction of Speckle Noise in Optical Coherence Tomography References and Links , 2022 .

[15]  Davide Cozzolino,et al.  SAR image despeckling through convolutional neural networks , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[16]  Nelson D. A. Mascarenhas,et al.  Ultrasound Image Despeckling Using Stochastic Distance-Based BM3D , 2017, IEEE Transactions on Image Processing.

[17]  Pierrick Coupé,et al.  Nonlocal Means-Based Speckle Filtering for Ultrasound Images , 2009, IEEE Transactions on Image Processing.

[18]  Victor S. Frost,et al.  A Model for Radar Images and Its Application to Adaptive Digital Filtering of Multiplicative Noise , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Xinjian Chen,et al.  DeSpecNet: a CNN-based method for speckle reduction in retinal optical coherence tomography images , 2019, Physics in medicine and biology.

[20]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[21]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Honglak Lee,et al.  Unsupervised feature learning for audio classification using convolutional deep belief networks , 2009, NIPS.

[23]  Larry S. Davis,et al.  Stacked U-Nets: A No-Frills Approach to Natural Image Segmentation , 2018, ArXiv.

[24]  Francisco Javier Díaz Pernas,et al.  Wavelet‐Based Denoising for Traffic Volume Time Series Forecasting with Self‐Organizing Neural Networks , 2010, Comput. Aided Civ. Infrastructure Eng..

[25]  Hongyu Wang,et al.  End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks , 2016, IEEE Access.

[26]  Francesco Visin,et al.  A guide to convolution arithmetic for deep learning , 2016, ArXiv.

[27]  Stephen Lin,et al.  GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[28]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Bingliang Hu,et al.  Attention Mask R-CNN for Ship Detection and Segmentation From Remote Sensing Images , 2020, IEEE Access.

[31]  Scott T. Acton,et al.  Ultrasound Despeckling for Contrast Enhancement , 2010, IEEE Transactions on Image Processing.

[32]  Haoran Zhang,et al.  Cascaded Conditional Generative Adversarial Networks With Multi-Scale Attention Fusion for Automated Bi-Ventricle Segmentation in Cardiac MRI , 2019, IEEE Access.

[33]  Xin Luo,et al.  Wavelet Denoising Algorithm Based on NDOA Compressed Sensing for Fluorescence Image of Microarray , 2019, IEEE Access.

[34]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[35]  Mingyue Ding,et al.  PCANet based nonlocal means method for speckle noise removal in ultrasound images , 2018, PloS one.

[36]  Yoshua Bengio,et al.  How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation , 2014, ArXiv.

[37]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[38]  M. Grasserbauer,et al.  Wavelet denoising of Gaussian peaks: A comparative study , 1996 .

[39]  Hemant D. Tagare,et al.  Evaluation of Four Probability Distribution Models for Speckle in Clinical Cardiac Ultrasound Images , 2006, IEEE Transactions on Medical Imaging.

[40]  Wei Shi,et al.  Dilated convolution neural network with LeakyReLU for environmental sound classification , 2017, 2017 22nd International Conference on Digital Signal Processing (DSP).

[41]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[42]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Scott T. Acton,et al.  Speckle reducing anisotropic diffusion , 2002, IEEE Trans. Image Process..