Echo from noise: synthetic ultrasound image generation using diffusion models for real image segmentation

We propose a novel pipeline for the generation of synthetic ultrasound images via Denoising Diffusion Probabilistic Models (DDPMs) guided by cardiac semantic label maps. We show that these synthetic images can serve as a viable substitute for real data in the training of deep-learning models for ultrasound image analysis tasks such as cardiac segmentation. To demonstrate the effectiveness of this approach, we generated synthetic 2D echocardiograms and trained a neural network for segmenting the left ventricle and left atrium. The performance of the network trained on exclusively synthetic images was evaluated on an unseen dataset of real images and yielded mean Dice scores of 88.6 $\pm 4.91$ , 91.9 $\pm 4.22$, 85.2 $\pm 4.83$ \% for left ventricular endocardium, epicardium and left atrial segmentation respectively. This represents a relative increase of $9.2$, $3.3$ and $13.9$ \% in Dice scores compared to the previous state-of-the-art. The proposed pipeline has potential for application to a wide range of other tasks across various medical imaging modalities.

[1]  Jakob Nikolas Kather,et al.  A multimodal comparison of latent denoising diffusion probabilistic models and generative adversarial networks for medical image synthesis , 2022, Scientific reports.

[2]  Qing Lyu,et al.  Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models , 2022, ArXiv.

[3]  S. Ourselin,et al.  Brain Imaging Generation with Latent Diffusion Models , 2022, DGM4MICCAI@MICCAI.

[4]  P. Lamata,et al.  Efficient Pix2Vox++ for 3D Cardiac Reconstruction from 2D echo views , 2022, ASMUS@MICCAI.

[5]  Dong Chen,et al.  Semantic Image Synthesis via Diffusion Models , 2022, ArXiv.

[6]  Christoph M. Augustin,et al.  Linking statistical shape models and simulated function in the healthy adult human heart , 2021, PLoS Comput. Biol..

[7]  Prafulla Dhariwal,et al.  Improved Denoising Diffusion Probabilistic Models , 2021, ICML.

[8]  Eigil Samset,et al.  Generating Synthetic Labeled Data From Existing Anatomical Models: An Example With Echocardiography Segmentation , 2021, IEEE Transactions on Medical Imaging.

[9]  Curtis P. Langlotz,et al.  Video-based AI for beat-to-beat assessment of cardiac function , 2020, Nature.

[10]  S. Ourselin,et al.  TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning , 2020, Comput. Methods Programs Biomed..

[11]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[12]  Bin Yang,et al.  Unsupervised Medical Image Translation Using Cycle-MedGAN , 2019, 2019 27th European Signal Processing Conference (EUSIPCO).

[13]  Frederic Cervenansky,et al.  Deep Learning for Segmentation Using an Open Large-Scale Dataset in 2D Echocardiography , 2019, IEEE Transactions on Medical Imaging.

[14]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Pamela J Schreiner,et al.  Quality Control and Reproducibility in M‐Mode, Two‐Dimensional, and Speckle Tracking Echocardiography Acquisition and Analysis: The CARDIA Study, Year 25 Examination Experience , 2015, Echocardiography.

[16]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[17]  Surya Ganguli,et al.  Deep Unsupervised Learning using Nonequilibrium Thermodynamics , 2015, ICML.

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Sébastien Marcel,et al.  Torchvision the machine-vision package of torch , 2010, ACM Multimedia.

[20]  Deli Zhao,et al.  Uncertainty Principles of Encoding GANs , 2021, ICML.