Deep Generative Models to Simulate 2D Patient-Specific Ultrasound Images in Real Time

We present a computational method for real-time, patient-specific simulation of 2D ultrasound (US) images. The method uses a large number of tracked ultrasound images to learn a function that maps position and orientation of the transducer to ultrasound images. This is a first step towards realistic patient-specific simulations that will enable improved training and retrospective examination of complex cases. Our models can simulate a 2D image in under 4ms (well within real-time constraints), and produce simulated images that preserve the content (anatomical structures and artefacts) of real ultrasound images.

[1]  Malin Premaratne,et al.  New Structural Similarity Measure for Image Comparison , 2012, ICIC.

[2]  Vincent Dumoulin,et al.  Deconvolution and Checkerboard Artifacts , 2016 .

[3]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[4]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[5]  Nassir Navab,et al.  Real-Time Simulation of Medical Ultrasound from CT Images , 2008, MICCAI.

[6]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[7]  William Albert Arkhurst.  Ein interaktiver Atlas für die Sonographie und Anatomie des Säuglingsgehirns , 2005 .

[8]  Bo Sun,et al.  Medical Student Evaluation using Virtual Pathology Echocardiography (VPE) for Augmented Standardized Patients , 2008, MMVR.

[9]  J. Arendt.  Paper presented at the 10th Nordic-Baltic Conference on Biomedical Imaging: Field: A Program for Simulating Ultrasound Systems , 1996 .

[10]  H H Ehricke.  SONOSim3D: a multimedia system for sonography simulation and education with an extensible case database. , 1998, European journal of ultrasound : official journal of the European Federation of Societies for Ultrasound in Medicine and Biology.

[11]  Daniel Cohen-Or,et al.  Real-Time Ultrasound Imaging Simulation , 1998, Real Time Imaging.

[12]  Thomas Berlage,et al.  Supporting ultrasound diagnosis using an animated 3D model of the heart , 1996, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems.

[13]  Lucile Vadcard,et al.  BiopSym: A Simulator for Enhanced Learning of Ultrasound-Guided Prostate Biopsy , 2009, MMVR.

[14]  Andreas Rieger,et al.  A Review of Computer-Based Simulators for Ultrasound Training , 2013, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[15]  Dean C. Barratt,et al.  Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks , 2017, CMMI/RAMBO/SWITCH@MICCAI.

[16]  K. Boone,et al.  Effect of skin impedance on image quality and variability in electrical impedance tomography: a model study , 1996, Medical and Biological Engineering and Computing.

[17]  Pascal Vincent,et al.  Higher Order Contractive Auto-Encoder , 2011, ECML/PKDD.

[18]  Thomas Brox,et al.  Learning to generate chairs with convolutional neural networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jürgen Hesser,et al.  Simulation of Dynamic Ultrasound Based on CT Models for Medical Education , 2008, MMVR.

[20]  Reinhard Männer,et al.  Real-time simulator for intravascular ultrasound (IVUS) , 2007, SPIE Medical Imaging.

[21]  Pascal Vincent,et al.  Contractive Auto-Encoders: Explicit Invariance During Feature Extraction , 2011, ICML.