Deep Learning Cross-Phase Style Transfer for Motion Artifact Correction in Coronary Computed Tomography Angiography

Motion artifacts may occur in coronary computed tomography angiography (CCTA) due to the heartbeat and impede the clinician’s diagnosis of coronary arterial diseases. Thus, motion artifact correction of the coronary artery is required to quantify the risk of disease more accurately. We present a novel method based on deep learning for motion artifact correction in CCTA. Because the image of the coronary artery without motion (the ground-truth data required in supervised deep learning) is medically unattainable, we apply a style transfer method to 2D image patches cropped from full-phase 4D computed tomography (CT) to synthesize these images. We then train a convolutional neural network (CNN) for motion artifact correction using this synthetic ground-truth (SynGT). During testing, the output motion-corrected 2D image patches of the trained network are reinserted into the 3D CT volume with volumetric interpolation. The proposed method is evaluated using both phantom and clinical data. A phantom study demonstrates comparable results to other methods in quantitative performance and outperforms those methods in computation time. For clinical data, a quantitative analysis based on metric measurements is presented that confirms the correction of motion artifacts. Moreover, an observer study finds that by applying the proposed method, motion artifacts are markedly reduced, and boundaries of the coronary artery are much sharper, with a strong inter-observer agreement ( $\kappa = 0.78$ ). Finally, evaluations using commercial software on the original and resulting CT volumes of the proposed method reveal a considerable increase in tracked coronary artery length.

[1]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  J. Tardif,et al.  Heart Rate and Use of Beta-Blockers in Stable Outpatients with Coronary Artery Disease , 2012, PloS one.

[5]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[6]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Joachim Hornegger,et al.  C-arm CT: Reconstruction of dynamic high contrast objects applied to the coronary sinus , 2008, 2008 IEEE Nuclear Science Symposium Conference Record.

[8]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[9]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[10]  Zhenhua Wang,et al.  Synthesizing Training Images for Boosting Human 3D Pose Estimation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[11]  Nassir Navab,et al.  Dynamic Cone-Beam Reconstruction Using a Variational Level Set Formulation , 2009 .

[12]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Hongfeng Ma,et al.  Evaluation of motion artifact metrics for coronary CT angiography , 2018, Medical physics.

[14]  N. Paul,et al.  Perioperative β-Blockers : Use With Caution Perioperative β Blockers in Patients Having Non-Cardiac Surgery : A Meta-Analysis , 2010 .

[15]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[16]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[17]  Jeroen J. Bax,et al.  Automatic centerline extraction of coronary arteries in coronary computed tomographic angiography , 2011, The International Journal of Cardiovascular Imaging.

[18]  Yeonggul Jang,et al.  Deep Learning Based Coronary Artery Motion Artifact Compensation Using Style-Transfer Synthesis in CT Images , 2018, SASHIMI@MICCAI.

[19]  Antonio Pepe,et al.  PET-Train: Automatic Ground Truth Generation from PET Acquisitions for Urinary Bladder Segmentation in CT Images using Deep Learning , 2018, 2018 11th Biomedical Engineering International Conference (BMEiCON).

[20]  Christopher Rohkohl,et al.  Improving best-phase image quality in cardiac CT by motion correction with MAM optimization. , 2013, Medical physics.

[21]  Guanglei Xiong,et al.  Heart-rate dependent improvement in image quality and diagnostic accuracy of coronary computed tomographic angiography by novel intracycle motion correction algorithm. , 2015, Clinical imaging.

[22]  Katsuyuki Taguchi,et al.  A fully four-dimensional, iterative motion estimation and compensation method for cardiac CT. , 2012, Medical physics.

[23]  Fabian Bamberg,et al.  Efficacy of pre-scan beta-blockade and impact of heart rate on image quality in patients undergoing coronary multidetector computed tomography angiography. , 2008, European journal of radiology.

[24]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[25]  Jong Beom Ra,et al.  Cardiac motion correction based on partial angle reconstructed images in x-ray CT. , 2015, Medical physics.

[26]  Maria Iatrou,et al.  Nonrigid registration-based coronary artery motion correction for cardiac computed tomography. , 2012, Medical physics.

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  Melonie P. Heron Deaths: Leading Causes for 2012. , 2015, National vital statistics reports : from the Centers for Disease Control and Prevention, National Center for Health Statistics, National Vital Statistics System.

[29]  Günter Lauritsch,et al.  CAVAREV—an open platform for evaluating 3D and 4D cardiac vasculature reconstruction , 2010, Physics in medicine and biology.

[30]  Stephan Achenbach,et al.  Spatio-temporally regularized 4-D cardiovascular C-arm CT reconstruction using a proximal algorithm , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[31]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[32]  Jana Kosecka,et al.  Synthesizing Training Data for Object Detection in Indoor Scenes , 2017, Robotics: Science and Systems.

[33]  Yeonggul Jang,et al.  Maximum a posteriori estimation method for aorta localization and coronary seed identification , 2017, Pattern Recognit..

[34]  Michael Grass,et al.  Fully automatic nonrigid registration-based local motion estimation for motion-corrected iterative cardiac CT reconstruction. , 2010, Medical physics.

[35]  C Schwemmer,et al.  Residual motion compensation in ECG-gated interventional cardiac vasculature reconstruction , 2013, Physics in medicine and biology.

[36]  Marc Kachelrieß,et al.  Motion compensation in the region of the coronary arteries based on partial angle reconstructions from short‐scan CT data , 2017, Medical physics.

[37]  Günter Lauritsch,et al.  Opening Windows – Increasing Window Size in Motion-Compensated ECG-gated Cardiac Vasculature Reconstruction , 2013 .

[38]  Leon Sixt,et al.  RenderGAN: Generating Realistic Labeled Data , 2016, Front. Robot. AI.

[39]  Tomas Pfister,et al.  Learning from Simulated and Unsupervised Images through Adversarial Training , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Lovedeep Gondara,et al.  Medical Image Denoising Using Convolutional Denoising Autoencoders , 2016, 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW).