UltraGAN: Ultrasound Enhancement Through Adversarial Generation

Ultrasound images are used for a wide variety of medical purposes because of their capacity to study moving structures in real time. However, the quality of ultrasound images is significantly affected by external factors limiting interpretability. We present UltraGAN, a novel method for ultrasound enhancement that transfers quality details while preserving structural information. UltraGAN incorporates frequency loss functions and an anatomical coherence constraint to perform quality enhancement. We show improvement in image quality without sacrificing anatomical consistency. We validate UltraGAN on a publicly available dataset for echocardiography segmentation and demonstrate that our quality-enhanced images are able to improve downstream tasks. To ensure reproducibility we provide our source code and training models.

[1]  Ghassan Hamarneh,et al.  Mask2Lesion: Mask-Constrained Adversarial Skin Lesion Image Synthesis , 2019, SASHIMI@MICCAI.

[2]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[3]  Aditya Deshpande,et al.  Learning Diverse Image Colorization , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Purang Abolmaesumi,et al.  Echocardiography Segmentation by Quality Translation Using Anatomically Constrained CycleGAN , 2019, MICCAI.

[5]  Frederic Cervenansky,et al.  Deep Learning for Segmentation Using an Open Large-Scale Dataset in 2D Echocardiography , 2019, IEEE Transactions on Medical Imaging.

[6]  J. Gardin,et al.  American Society of Echocardiography recommendations for use of echocardiography in clinical trials. , 2004, Journal of the American Society of Echocardiography : official publication of the American Society of Echocardiography.

[7]  Luc Van Gool,et al.  SMIT: Stochastic Multi-Label Image-to-Image Translation , 2018, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[8]  Fatimah Khalid,et al.  Echocardiography Image Segmentation: A Survey , 2013, 2013 International Conference on Advanced Computer Science Applications and Technologies.

[9]  Chen Change Loy,et al.  EDVR: Video Restoration With Enhanced Deformable Convolutional Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[11]  Andrés Eduardo Castro-Ospina,et al.  Speckle Noise Reduction in Ultrasound Images for Improving the Metrological Evaluation of Biomedical Applications: An Overview , 2020, IEEE Access.

[12]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  A. Ng,et al.  Resolution in ultrasound imaging , 2011 .

[14]  Aaron Carass,et al.  Unpaired Brain MR-to-CT Synthesis Using a Structure-Constrained CycleGAN , 2018, DLMIA/ML-CDS@MICCAI.

[15]  Martin D. Fox,et al.  Ultrasound image enhancement: A review , 2012, Biomed. Signal Process. Control..

[16]  Purang Abolmaesumi,et al.  Cardiac point-of-care to cart-based ultrasound translation using constrained CycleGAN , 2020, International Journal of Computer Assisted Radiology and Surgery.

[17]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[18]  Purang Abolmaesumi,et al.  GAN-enhanced Conditional Echocardiogram Generation , 2019, ArXiv.

[19]  Sidney Fels,et al.  A Study into Echocardiography View Conversion , 2019, ArXiv.

[20]  Harvey Feigenbaum,et al.  Recommendations for a standardized report for adult transthoracic echocardiography: a report from the American Society of Echocardiography's Nomenclature and Standards Committee and Task Force for a Standardized Echocardiography Report. , 2002, Journal of the American Society of Echocardiography : official publication of the American Society of Echocardiography.

[21]  William Stewart,et al.  American Society of Echocardiography and Society of Cardiovascular Anesthesiologists Task Force Guidelines for Training in Perioperative Echocardiography , 2002, Anesthesia and analgesia.

[22]  Antonio Torralba,et al.  To appear in the ACM SIGGRAPH conference proceedings Hybrid images , 2006 .

[23]  Purang Abolmaesumi,et al.  Semi-Supervised Learning For Cardiac Left Ventricle Segmentation Using Conditional Deep Generative Models as Prior , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[24]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[25]  Radu Timofte,et al.  Frequency Separation for Real-World Super-Resolution , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[26]  Jean-Michel Rouet,et al.  Spectral CT Based Training Dataset Generation and Augmentation for Conventional CT Vascular Segmentation , 2019, MICCAI.

[27]  Yung-Yu Chuang,et al.  Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Daniel Rueckert,et al.  Intelligent image synthesis to attack a segmentation CNN using adversarial learning , 2019, SASHIMI@MICCAI.

[29]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[30]  Pamela J Schreiner,et al.  Quality Control and Reproducibility in M‐Mode, Two‐Dimensional, and Speckle Tracking Echocardiography Acquisition and Analysis: The CARDIA Study, Year 25 Examination Experience , 2015, Echocardiography.

[31]  Purang Abolmaesumi,et al.  Echocardiography View Classification Using Quality Transfer Star Generative Adversarial Networks , 2019, MICCAI.