Fast contour propagation for MR‐guided prostate radiotherapy using convolutional neural networks

Purpose To quickly and automatically propagate organ contours from pretreatment to fraction images in magnetic resonance (MR)‐guided prostate external‐beam radiotherapy. Methods Five prostate cancer patients underwent 20 fractions of image‐guided external‐beam radiotherapy on a 1.5 T MR‐Linac system. For each patient, a pretreatment T2‐weighted three‐dimensional (3D) MR imaging (MRI) scan was used to delineate the clinical target volume (CTV) contours. The same scan was repeated during each fraction, with the CTV contour being manually adapted if necessary. A convolutional neural network (CNN) was trained for combined image registration and contour propagation. The network estimated the propagated contour and a deformation field between the two input images. The training set consisted of a synthetically generated ground truth of randomly deformed images and prostate segmentations. We performed a leave‐one‐out cross‐validation on the five patients and propagated the prostate segmentations from the pretreatment to the fraction scans. Three variants of the CNN, aimed at investigating supervision based on optimizing segmentation overlap, optimizing the registration, and a combination of the two were compared to results of the open‐source deformable registration software package Elastix. Results The neural networks trained on segmentation overlap or the combined objective achieved significantly better Hausdorff distances between predicted and ground truth contours than Elastix, at the much faster registration speed of 0.5 s. The CNN variant trained to optimize both the prostate overlap and deformation field, and the variant trained to only maximize the prostate overlap, produced the best propagation results. Conclusions A CNN trained on maximizing prostate overlap and minimizing registration errors provides a fast and accurate method for deformable contour propagation for prostate MR‐guided radiotherapy.

[1]  V.R.S Mani,et al.  Survey of Medical Image Registration , 2013 .

[2]  Rob H.N. Tijssen,et al.  Feasibility of stereotactic radiotherapy using a 1.5 T MR-linac: Multi-fraction treatment of pelvic lymph node oligometastases. , 2019, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[3]  M van Herk,et al.  MRI-guided prostate adaptive radiotherapy - A systematic review. , 2016, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[4]  Josien P. W. Pluim,et al.  Progressively growing convolutional networks for end-to-end deformable image registration , 2019, Medical Imaging: Image Processing.

[5]  Qian Wang,et al.  Deformable Image Registration Based on Similarity-Steered CNN Regression , 2017, MICCAI.

[6]  L G W Kerkmeijer,et al.  Magnetic Resonance Imaging only Workflow for Radiotherapy Simulation and Planning in Prostate Cancer. , 2018, Clinical oncology (Royal College of Radiologists (Great Britain)).

[7]  Radhe Mohan,et al.  A deformable image registration method to handle distended rectums in prostate cancer radiotherapy. , 2006, Medical physics.

[8]  David A Jaffray,et al.  Accuracy and sensitivity of finite element model-based deformable registration of the prostate. , 2008, Medical physics.

[9]  J G M Kok,et al.  Integrating a 1.5 T MRI scanner with a 6 MV accelerator: proof of concept , 2009, Physics in medicine and biology.

[10]  Sébastien Ourselin,et al.  Weakly-supervised convolutional neural networks for multimodal image registration , 2018, Medical Image Anal..

[11]  Stefan Klein,et al.  Automatic segmentation of the prostate in 3D MR images by atlas matching using localized mutual information. , 2008, Medical physics.

[12]  Tian Liu,et al.  Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic MRI prostate segmentation , 2019, Medical physics.

[13]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[15]  Maxime Sermesant,et al.  SVF-Net: Learning Deformable Image Registration Using Shape Matching , 2017, MICCAI.

[16]  Josien P W Pluim,et al.  Progressively Trained Convolutional Neural Networks for Deformable Image Registration , 2019, IEEE Transactions on Medical Imaging.

[17]  Max A. Viergever,et al.  A survey of medical image registration , 1998, Medical Image Anal..

[18]  Hao Chen,et al.  Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images , 2017, AAAI.

[19]  David Gillatt,et al.  10-Year Outcomes after Monitoring, Surgery, or Radiotherapy for Localized Prostate Cancer. , 2017, The New England journal of medicine.

[20]  Maria A Schmidt,et al.  Radiotherapy planning using MRI , 2015, Physics in medicine and biology.

[21]  Max A. Viergever,et al.  End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network , 2017, DLMIA/ML-CDS@MICCAI.

[22]  A N T J Kotte,et al.  First patients treated with a 1.5 T MRI-Linac: clinical proof of concept of a high-precision, high-field MRI guided radiotherapy treatment , 2017, Physics in Medicine and Biology.

[23]  Jin Tae Kwak,et al.  Deep dense multi-path neural network for prostate segmentation in magnetic resonance imaging , 2018, International Journal of Computer Assisted Radiology and Surgery.

[24]  Zhiqiang Tian,et al.  PSNet: prostate segmentation on MRI based on a convolutional neural network , 2018, Journal of medical imaging.

[25]  Colin Studholme,et al.  Deformation-based mapping of volume change from serial brain MRI in the presence of local tissue contrast change , 2006, IEEE Transactions on Medical Imaging.

[26]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[27]  Josien P. W. Pluim,et al.  Pulmonary CT Registration Through Supervised Learning With Convolutional Neural Networks , 2019, IEEE Transactions on Medical Imaging.

[28]  Sasa Mutic,et al.  The ViewRay system: magnetic resonance-guided and controlled radiotherapy. , 2014, Seminars in radiation oncology.

[29]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[30]  Mitko Veta,et al.  Deformable image registration using convolutional neural networks , 2018, Medical Imaging.

[31]  Dinggang Shen,et al.  Deformable Image Registration Using a Cue-Aware Deep Regression Network , 2018, IEEE Transactions on Biomedical Engineering.

[32]  Jan J W Lagendijk,et al.  Soft-tissue prostate intrafraction motion tracking in 3D cine-MR for MR-guided radiotherapy , 2019, Physics in medicine and biology.

[33]  Nikos Paragios,et al.  Linear and Deformable Image Registration with 3D Convolutional Neural Networks , 2018, RAMBO+BIA+TIA@MICCAI.

[34]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[35]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[36]  Indrin J Chetty,et al.  An adaptive MR-CT registration method for MRI-guided prostate cancer radiotherapy , 2015, Physics in medicine and biology.

[37]  Mert R. Sabuncu,et al.  Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration , 2018, MICCAI.

[38]  Olivier D. Faugeras,et al.  Variational Methods for Multimodal Image Matching , 2002, International Journal of Computer Vision.

[39]  Max A. Viergever,et al.  A deep learning framework for unsupervised affine and deformable image registration , 2018, Medical Image Anal..

[40]  Nathan Lay,et al.  Automatic magnetic resonance prostate segmentation by deep learning with holistically nested networks , 2017, Journal of medical imaging.

[41]  A. Kishan,et al.  Stereotactic Body Radiation Therapy for Localized Prostate Cancer: A Systematic Review and Meta-Analysis of Over 6,000 Patients Treated On Prospective Studies. , 2019, International journal of radiation oncology, biology, physics.

[42]  S. Senan,et al.  A prospective single-arm phase II study of stereotactic magnetic-resonance-guided adaptive radiotherapy for prostate cancer: Early toxicity results. , 2019, International journal of radiation oncology, biology, physics.

[43]  Stefan Heldmann,et al.  Unsupervised learning for large motion thoracic CT follow-up registration , 2019, Medical Imaging: Image Processing.

[44]  Runze Han,et al.  Effect of statistical mismatch between training and test images for CNN-based deformable registration , 2019, Medical Imaging: Image Processing.

[45]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.