Stacked Bidirectional Convolutional LSTMs for 3D Non-contrast CT Reconstruction from Spatiotemporal 4D CT

The imaging workup in acute stroke can be simplified by reconstructing the noncontrast CT (NCCT) from CT perfusion (CTP) images, resulting in reduced workup time and radiation dose. This work presents a stacked bidirectional convolutional LSTM (C-LSTM) network to predict 3D volumes from 4D spatiotemporal data. Several parameterizations of the C-LSTM network were trained on a set of 17 CTP-NCCT pairs to learn to reconstruct NCCT from CTP and were subsequently quantitatively evaluated on a separate cohort of 16 cases. The results show that C-LSTM network clearly outperforms basic reconstruction methods and provides a promising general deep learning approach for handling high-dimensional spatiotemporal medical data.

[1]  Christopher S. Coffey,et al.  Association Focused Update of the 2013 Guidelines for the Early Management of Patients With Acute Ischemic Stroke Regarding Endovascular Treatment , 2015 .

[2]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[3]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[4]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[5]  Bram van Ginneken,et al.  Robust cranial cavity segmentation in CT and CT perfusion images of trauma and suspected stroke patients , 2017, Medical Image Anal..

[6]  Christopher S Coffey,et al.  2015 American Heart Association/American Stroke Association Focused Update of the 2013 Guidelines for the Early Management of Patients With Acute Ischemic Stroke Regarding Endovascular Treatment: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association , 2015, Stroke.

[7]  Sander Dieleman,et al.  Beyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video , 2015, International Journal of Computer Vision.

[8]  Bram van Ginneken,et al.  Timing-Invariant Imaging of Collateral Vessels in Acute Ischemic Stroke , 2013, Stroke.

[9]  Yaozong Gao,et al.  Estimating CT Image from MRI Data Using 3D Fully Convolutional Networks , 2016, LABELS/DLMIA@MICCAI.

[10]  Wojciech Zaremba,et al.  An Empirical Exploration of Recurrent Network Architectures , 2015, ICML.

[11]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[12]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[14]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[15]  Hu Chen,et al.  Low-dose CT via convolutional neural network. , 2017, Biomedical optics express.

[16]  John Salvatier,et al.  Theano: A Python framework for fast computation of mathematical expressions , 2016, ArXiv.

[17]  Anelia Angelova,et al.  Geometry-based next frame prediction from monocular video , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[18]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[19]  Jianbo Liu,et al.  LSTM Pose Machines , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Liang Wang,et al.  Bidirectional Recurrent Convolutional Networks for Multi-Frame Super-Resolution , 2015, NIPS.

[21]  Ce Zhang,et al.  Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limit , 2017, ArXiv.

[22]  Yong Haur Tay,et al.  Abnormal Event Detection in Videos using Spatiotemporal Autoencoder , 2017, ISNN.

[23]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[24]  Dinggang Shen,et al.  Convolutional Neural Network for Reconstruction of 7T-like Images from 3T MRI Using Appearance and Anatomical Features , 2016, LABELS/DLMIA@MICCAI.

[25]  Jong Chul Ye,et al.  A deep convolutional neural network using directional wavelets for low‐dose X‐ray CT reconstruction , 2016, Medical physics.

[26]  Bram Platel,et al.  White Matter and Gray Matter Segmentation in 4D Computed Tomography , 2017, Scientific Reports.

[27]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.