Weekly supervised convolutional long short-term memory neural networks for MR-TRUS registration

We propose an approach based on a weekly supervised method for MR-TRUS image registration. Inspired by the viscous fluid physical model, we made the first attempt at combining convolutional neural network (CNN) and long short-term memory (LSTM) Neural Network to perform deep learning-based dense deformation field prediction. Through the integration of convolutional long short-term memory (ConvLSTM) Neural Network and weakly supervised approach, we achieved accurate results in terms of Dice similarity coefficient (DSC) and target registration error (TRE) without using conventional intensity-based image similarity measures. Thirty-six sets of patient data were used in the study. Experimental results showed that our proposed ConvLSTM neural network produced a mean TRE of 2.85±1.72 mm and a mean Dice of 0.89.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Yang Lei,et al.  Synthetic MRI-aided multi-organ segmentation on male pelvic CT using cycle consistent deep attention network. , 2019, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[3]  Tian Liu,et al.  A MR-TRUS registration method for ultrasound-guided prostate interventions , 2015, Medical Imaging.

[4]  Yang Lei,et al.  Ultrasound prostate segmentation based on multidirectional deeply supervised V-Net. , 2019, Medical physics.

[5]  Yang Lei,et al.  Automatic MRI prostate segmentation using 3D deeply supervised FCN with concatenated atrous convolution , 2019, Medical Imaging.

[6]  Cheng Wang,et al.  A learning-based automatic segmentation method on left ventricle in SPECT imaging , 2019, Medical Imaging.

[7]  Andrew Y. Ng,et al.  End-to-End People Detection in Crowded Scenes , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Tian Liu,et al.  MRI-based treatment planning for brain stereotactic radiosurgery: Dosimetric validation of a learning-based pseudo-CT generation method. , 2019, Medical dosimetry : official journal of the American Association of Medical Dosimetrists.

[9]  Tian Liu,et al.  MRI-based synthetic CT generation using semantic random forest with iterative refinement , 2019, Physics in medicine and biology.

[10]  Baowei Fei,et al.  3D non-rigid registration using surface and local salient features for transrectal ultrasound image-guided prostate biopsy , 2011, Medical Imaging.

[11]  Jun Zhou,et al.  Learning-based automatic segmentation of arteriovenous malformations on contrast CT images in brain stereotactic radiosurgery. , 2019, Medical physics.

[12]  Tian Liu,et al.  MRI-based Treatment Planning for Proton Radiotherapy: Dosimetric Validation of a Deep Learning-based Liver Synthetic CT Generation Method , 2019, Physics in medicine and biology.

[13]  Marc Modat,et al.  Label-driven weakly-supervised learning for multimodal deformarle image registration , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[14]  Yang Lei,et al.  Automated prostate segmentation of volumetric CT images using 3D deeply supervised dilated FCN , 2019, Medical Imaging: Image Processing.

[15]  Zeike A. Taylor,et al.  MR to ultrasound registration for image-guided prostate interventions , 2012, Medical Image Anal..

[16]  Yang Lei,et al.  4D-CT Deformable Image Registration Using an Unsupervised Deep Convolutional Neural Network , 2019, AIRT@MICCAI.

[17]  Shuohang Wang,et al.  Learning Natural Language Inference with LSTM , 2015, NAACL.

[18]  David Vandyke,et al.  Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems , 2015, EMNLP.

[19]  Tian Liu,et al.  Automatic multiorgan segmentation in thorax CT images using U-net-GAN. , 2019, Medical physics.

[20]  Sébastien Ourselin,et al.  Weakly-supervised convolutional neural networks for multimodal image registration , 2018, Medical Image Anal..

[21]  Tian Liu,et al.  Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic MRI prostate segmentation , 2019, Medical physics.

[22]  Yang Lei,et al.  CBCT-Based Synthetic MRI Generation for CBCT-Guided Adaptive Radiotherapy , 2019, AIRT@MICCAI.

[23]  Yang Lei,et al.  MRI-based treatment planning for liver stereotactic body radiotherapy: validation of a deep learning-based synthetic CT generation method. , 2019, The British journal of radiology.

[24]  Tian Liu,et al.  3D transrectal ultrasound (TRUS) prostate segmentation based on optimal feature learning framework , 2016, SPIE Medical Imaging.

[25]  Yang Lei,et al.  MRI-based attenuation correction for brain PET/MRI based on anatomic signature and machine learning , 2019, Physics in medicine and biology.

[26]  Russell H. Taylor,et al.  A Combined Statistical and Biomechanical Model for Estimation of Intra-operative Prostate Deformation , 2002, MICCAI.

[27]  Yang Lei,et al.  Multiparametric MRI-guided dose boost to dominant intraprostatic lesions in CT-based High-dose-rate prostate brachytherapy. , 2019, The British journal of radiology.

[28]  Ge Cui,et al.  Machine-learning-based classification of Glioblastoma using MRI-based radiomic features , 2019, Medical Imaging.

[29]  Yang Lei,et al.  Male pelvic multi-organ segmentation aided by CBCT-based synthetic MRI , 2019, Physics in medicine and biology.

[30]  Sibo Tian,et al.  A planning study of focal dose escalations to multiparametric MRI-defined dominant intraprostatic lesions in prostate proton radiation therapy. , 2020, British Journal of Radiology.

[31]  Yang Lei,et al.  Dose evaluation of MRI-based synthetic CT generated using a machine learning method for prostate cancer radiotherapy. , 2019, Medical dosimetry : official journal of the American Association of Medical Dosimetrists.

[32]  Yang Lei,et al.  Ultrasound prostate segmentation based on 3D V-Net with deep supervision , 2019, Medical Imaging.

[33]  Yang Lei,et al.  MRI-based synthetic CT generation using deep convolutional neural network , 2019, Medical Imaging: Image Processing.

[34]  Ming Xu,et al.  Towards Personalized Statistical Deformable Model and Hybrid Point Matching for Robust MR-TRUS Registration , 2016, IEEE Transactions on Medical Imaging.

[35]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[36]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Yang Lei,et al.  Brain MRI classification based on machine learning framework with auto-context model , 2019, Medical Imaging.

[38]  David J. Hawkes,et al.  Modelling Prostate Motion for Data Fusion During Image-Guided Interventions , 2011, IEEE Transactions on Medical Imaging.

[39]  Yang Lei,et al.  MRI-Based Proton Treatment Planning for Base of Skull Tumors. , 2019, International journal of particle therapy.

[40]  Yang Lei,et al.  CT Prostate Segmentation Based on Synthetic MRI-aided Deep Attention Fully Convolution Network. , 2019, Medical physics.

[41]  Yabo Fu,et al.  Deep Learning in Medical Image Registration: A Review , 2020, Physics in medicine and biology.

[42]  Yang Lei,et al.  A learning-based automatic segmentation and quantification method on left ventricle in gated myocardial perfusion SPECT imaging: A feasibility study , 2019, Journal of Nuclear Cardiology.

[43]  Yang Lei,et al.  MRI-only based synthetic CT generation using dense cycle consistent generative adversarial networks. , 2019, Medical physics.

[44]  Tian Liu,et al.  MRI-based pseudo CT synthesis using anatomical signature and alternating random forest with iterative refinement model , 2018, Journal of medical imaging.

[45]  Yang Lei,et al.  Learning-based automatic segmentation on arteriovenous malformations from contract-enhanced CT images , 2019, Medical Imaging.

[46]  Yang Lei,et al.  MRI-based pseudo CT generation using classification and regression random forest , 2019, Medical Imaging.

[47]  W. Curran,et al.  Evaluation of a deep learning-based pelvic synthetic CT generation technique for MRI-based prostate proton treatment planning , 2019, Physics in medicine and biology.