A review on 3D deformable image registration and its application in dose warping

Abstract Deformable image registration (DIR) has been well explored in recent decades, and it is widely utilized in clinical tasks, especially dose warping. Nowadays, as deep learning (DL) develops rapidly, many DL-based methods were also applied in DIR. This paper reviews DL-based DIR methods in recent years and the application of DIR in dose warping. We collected and categorized the latest DL-based DIR studies. A thorough review of each category was presented, in which studies were discussed based on their supervision, advantage, and challenges. Then, we reviewed DIR-based dose warping and discussed its rationale, feasibility, successes, and difficulties. Lastly, we summarized the review on both parts and discussed their future development trend.

[1]  Hervé Delingette,et al.  Robust Non-rigid Registration Through Agent-Based Action Learning , 2017, MICCAI.

[2]  Ali R. Khan,et al.  Computing an average anatomical atlas using LDDMM and geodesic shooting , 2006, 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006..

[3]  Heinz Handels,et al.  Training CNNs for Image Registration from Few Samples with Model-based Data Augmentation , 2017, MICCAI.

[4]  Max A. Viergever,et al.  A deep learning framework for unsupervised affine and deformable image registration , 2018, Medical Image Anal..

[5]  Wei Lu,et al.  Tracking lung tissue motion and expansion/compression with inverse consistent image registration and spirometry. , 2007, Medical physics.

[6]  Kari Tanderup,et al.  Simple DVH parameter addition as compared to deformable registration for bladder dose accumulation in cervix cancer brachytherapy. , 2013, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[7]  Junyi Xia,et al.  High performance computing for deformable image registration: Towards a new paradigm in adaptive radiotherapy. , 2008, Medical physics.

[8]  J. Dimopoulos,et al.  Dose-volume histogram parameters and late side effects in magnetic resonance image-guided adaptive cervical cancer brachytherapy. , 2011, International journal of radiation oncology, biology, physics.

[9]  Pingge Jiang,et al.  CNN Driven Sparse Multi-level B-Spline Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Hualiang Zhong,et al.  Monte Carlo dose mapping on deforming anatomy , 2009, Physics in medicine and biology.

[11]  Mitko Veta,et al.  Deformable image registration using convolutional neural networks , 2018, Medical Imaging.

[12]  Dinggang Shen,et al.  Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning Based Registration , 2018, MICCAI.

[13]  Gilmer Valdes,et al.  An unsupervised convolutional neural network-based algorithm for deformable image registration , 2018, Physics in medicine and biology.

[14]  Raúl San José Estépar,et al.  Diffeomorphic Lung Registration Using Deep CNNs and Reinforced Learning , 2018, RAMBO+BIA+TIA@MICCAI.

[15]  Indrin J Chetty,et al.  Deformable Registration for Dose Accumulation. , 2019, Seminars in radiation oncology.

[16]  Gary E. Christensen,et al.  Consistent image registration , 2001, IEEE Transactions on Medical Imaging.

[17]  Dinggang Shen,et al.  Deep Learning based Inter-Modality Image Registration Supervised by Intra-Modality Similarity , 2018, MLMI@MICCAI.

[18]  Jan Seuntjens,et al.  A direct voxel tracking method for four-dimensional Monte Carlo dose calculations in deforming anatomy. , 2006, Medical physics.

[19]  Xiaohuan Cao,et al.  Adversarial learning for mono- or multi-modal registration , 2019, Medical Image Anal..

[20]  L. Xing,et al.  Four-dimensional image registration for image-guided radiotherapy. , 2008, International journal of radiation oncology, biology, physics.

[21]  Lara P Bonner Millar,et al.  Assessment of cumulative external beam and intracavitary brachytherapy organ doses in gynecologic cancers using deformable dose summation. , 2015, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[22]  Yang Lei,et al.  4D-CT deformable image registration using multiscale unsupervised deep learning , 2020, Physics in medicine and biology.

[23]  J. Pouliot,et al.  A three-dimensional head-and-neck phantom for validation of multimodality deformable image registration for adaptive radiotherapy. , 2014, Medical physics.

[24]  Snehashis Roy,et al.  MR to CT registration of brains using image synthesis , 2014, Medical Imaging.

[25]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[26]  Meritxell Arenas,et al.  Dose accumulation during vaginal cuff brachytherapy based on rigid/deformable registration vs. single plan addition. , 2014, Brachytherapy.

[27]  Sébastien Ourselin,et al.  Toward adaptive radiotherapy for head and neck patients: Uncertainties in dose warping due to the choice of deformable registration algorithm. , 2015, Medical physics.

[28]  D. G. Swain.  Computer aided diagnosis of acute abdominal pain , 1986 .

[29]  Xiangrong Zhou,et al.  Learning 3D non-rigid deformation based on an unsupervised deep learning for PET/CT image registration , 2019, Medical Imaging.

[30]  Michael Velec,et al.  A novel technique to enable experimental validation of deformable dose accumulation. , 2012, Medical physics.

[31]  Michael B Sharpe,et al.  Image-guided radiotherapy: rationale, benefits, and limitations. , 2006, The Lancet. Oncology.

[32]  Snehashis Roy,et al.  Cross contrast multi‐channel image registration using image synthesis for MR brain images , 2017, Medical Image Anal..

[33]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[34]  L. Pickup,et al.  Importance of deformable image registration and biological dose summation in planning of radiotherapy retreatments. , 2017, Medical dosimetry : official journal of the American Association of Medical Dosimetrists.

[35]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[36]  Steve B. Jiang,et al.  A deformable head and neck phantom with in-vivo dosimetry for adaptive radiotherapy quality assurance. , 2015, Medical physics.

[37]  Geoffrey G. Zhang,et al.  Generation of composite dose and biological effective dose (BED) over multiple treatment modalities and multistage planning using deformable image registration. , 2010, Medical dosimetry : official journal of the American Association of Medical Dosimetrists.

[38]  Guido Gerig,et al.  Morphometry of anatomical shape complexes with dense deformations and sparse parameters , 2014, NeuroImage.

[39]  Raymond Y Huang,et al.  Residual Convolutional Neural Network for the Determination of IDH Status in Low- and High-Grade Gliomas from MR Imaging , 2017, Clinical Cancer Research.

[40]  Dean C. Barratt,et al.  Adversarial Deformation Regularization for Training Image Registration Neural Networks , 2018, MICCAI.

[41]  Lei Dong,et al.  Adaptive radiotherapy for head-and-neck cancer: initial clinical outcomes from a prospective trial. , 2012, International journal of radiation oncology, biology, physics.

[42]  René Werner,et al.  GDL-FIRE ^\text 4D : Deep Learning-Based Fast 4D CT Image Registration , 2018, MICCAI.

[43]  Hervé Delingette,et al.  Unsupervised Probabilistic Deformation Modeling for Robust Diffeomorphic Registration , 2018, DLMIA/ML-CDS@MICCAI.

[44]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[45]  Lei Dong,et al.  Automatic segmentation of whole breast using atlas approach and deformable image registration. , 2009, International journal of radiation oncology, biology, physics.

[46]  Jong Chul Ye,et al.  Unsupervised Deformable Image Registration Using Cycle-Consistent CNN , 2019, MICCAI.

[47]  D. Hill,et al.  Non-rigid image registration: theory and practice. , 2004, The British journal of radiology.

[48]  Nikos Paragios,et al.  Linear and Deformable Image Registration with 3D Convolutional Neural Networks , 2018, RAMBO+BIA+TIA@MICCAI.

[49]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[50]  Dinggang Shen,et al.  BIRNet: Brain image registration using dual‐supervised fully convolutional networks , 2018, Medical Image Anal..

[51]  R. Louwe,et al.  Quantifying the dose accumulation uncertainty after deformable image registration in head-and-neck radiotherapy. , 2020, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[52]  Yabo Fu,et al.  Deep Learning in Medical Image Registration: A Review , 2020, Physics in medicine and biology.

[53]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.

[54]  I. Chetty,et al.  Caution Must Be Exercised When Performing Deformable Dose Accumulation for Tumors Undergoing Mass Changes During Fractionated Radiation Therapy. , 2016, International journal of radiation oncology, biology, physics.

[55]  Dimos Baltas,et al.  One-Shot Learning for Deformable Medical Image Registration and Periodic Motion Tracking , 2019, IEEE Transactions on Medical Imaging.

[56]  Colin G Orton,et al.  Point/counterpoint: it is not appropriate to "deform" dose along with deformable image registration in adaptive radiotherapy. , 2012, Medical physics.

[57]  Yue Zhang,et al.  Segmentation of lumen and outer wall of abdominal aortic aneurysms from 3D black‐blood MRI with a registration based geodesic active contour model , 2017, Medical Image Anal..

[58]  J. Dimopoulos,et al.  Recommendations from gynaecological (GYN) GEC ESTRO working group (II): concepts and terms in 3D image-based treatment planning in cervix cancer brachytherapy-3D dose volume parameters and aspects of 3D image-based anatomy, radiation physics, radiobiology. , 2006, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[59]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[60]  Alexander H. Waibel,et al.  Modular Construction of Time-Delay Neural Networks for Speech Recognition , 1989, Neural Computation.

[61]  Siyong Kim,et al.  Deformable image registration in radiation therapy , 2017, Radiation oncology journal.

[62]  Jun Zhang,et al.  Inverse-Consistent Deep Networks for Unsupervised Deformable Image Registration , 2018, ArXiv.

[63]  Marius Staring,et al.  3D Convolutional Neural Networks Image Registration Based on Efficient Supervised Learning from Artificial Deformations , 2019, ArXiv.

[64]  Matthias Guckenberger,et al.  A multi-institution evaluation of deformable image registration algorithms for automatic organ delineation in adaptive head and neck radiotherapy , 2012, Radiation oncology.

[65]  Joseph O. Deasy,et al.  DIRART – A Software Suite for Deformable Image Registration and Adaptive Radiotherapy Research , 2009 .

[66]  Hualiang Zhong,et al.  An energy transfer method for 4D Monte Carlo dose calculation. , 2008, Medical physics.

[67]  Carri Glide-Hurst,et al.  Direct dose mapping versus energy/mass transfer mapping for 4D dose accumulation: fundamental differences and dosimetric consequences , 2014, Physics in medicine and biology.

[68]  Karsten O. Noe,et al.  Bladder dose accumulation based on a biomechanical deformable image registration algorithm in volumetric modulated arc therapy for prostate cancer , 2012, Physics in medicine and biology.

[69]  Simon K. Warfield,et al.  Motion‐robust parameter estimation in abdominal diffusion‐weighted MRI by simultaneous image registration and model estimation , 2017, Medical Image Anal..

[70]  Qian Wang,et al.  Deformable Image Registration Based on Similarity-Steered CNN Regression , 2017, MICCAI.

[71]  Maxime Sermesant,et al.  SVF-Net: Learning Deformable Image Registration Using Shape Matching , 2017, MICCAI.

[72]  R. Jeraj,et al.  Automatic registration of megavoltage to kilovoltage CT images in helical tomotherapy: an evaluation of the setup verification process for the special case of a rigid head phantom. , 2006, Medical physics.

[73]  E. D. Geijsen,et al.  Online adaptive radiotherapy compared to plan selection for rectal cancer: quantifying the benefit , 2020, Radiation oncology.

[74]  Tobias Knopp,et al.  Influence of deformable image registration on 4D dose simulation for extracranial SBRT: A multi-registration framework study. , 2018, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[75]  Tom Vercauteren,et al.  Diffeomorphic demons: Efficient non-parametric image registration , 2009, NeuroImage.

[76]  W. Tomé,et al.  On the dosimetric effect and reduction of inverse consistency and transitivity errors in deformable image registration for dose accumulation. , 2011, Medical physics.

[77]  Holly Ning,et al.  Comparison of intensity-modulated radiotherapy, adaptive radiotherapy, proton radiotherapy, and adaptive proton radiotherapy for treatment of locally advanced head and neck cancer. , 2011, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[78]  Keechul Jung,et al.  GPU implementation of neural networks , 2004, Pattern Recognit..

[79]  J. Fowler,et al.  Image guidance for precise conformal radiotherapy. , 2003, International journal of radiation oncology, biology, physics.

[80]  Josien P. W. Pluim,et al.  Pulmonary CT Registration Through Supervised Learning With Convolutional Neural Networks , 2019, IEEE Transactions on Medical Imaging.

[81]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[82]  Joseph O. Deasy,et al.  Technical Note: DIRART – A software suite for deformable image registration and adaptive radiotherapy research , 2010 .

[83]  D. Yan,et al.  Adaptive radiation therapy , 1997, Physics in medicine and biology.

[84]  Dirk Verellen,et al.  A (short) history of image-guided radiotherapy. , 2008, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[85]  Pingkun Yan,et al.  Deep learning in medical image registration: a survey , 2020, Machine Vision and Applications.

[86]  Fang-Fang Yin,et al.  A multi-scale framework with unsupervised joint training of convolutional neural networks for pulmonary deformable image registration , 2019, Physics in medicine and biology.

[87]  Martin F Fast,et al.  Real-time energy/mass transfer mapping for online 4D dose reconstruction , 2018, Scientific Reports.

[88]  Tom Vercauteren,et al.  Evaluation of deformable image coregistration in adaptive dose painting by numbers for head-and-neck cancer. , 2012, International journal of radiation oncology, biology, physics.

[89]  Jeffrey N. Adams,et al.  Development of a deformable dosimetric phantom to verify dose accumulation algorithms for adaptive radiotherapy , 2016, Journal of medical physics.

[90]  Li Sun,et al.  Deformable MRI-Ultrasound Registration Using 3D Convolutional Neural Network , 2018, POCUS/BIVPCS/CuRIOUS/CPM@MICCAI.

[91]  Yong Fan,et al.  Non-rigid image registration using self-supervised fully convolutional networks without training data , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[92]  K. Brock,et al.  Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132 , 2017, Medical physics.

[93]  Osamu Abe,et al.  Deep neural network‐based computer‐assisted detection of cerebral aneurysms in MR angiography , 2018, Journal of magnetic resonance imaging : JMRI.

[94]  Yang Lei,et al.  LungRegNet: an unsupervised deformable image registration method for 4D-CT lung. , 2020, Medical physics.

[95]  J. Dimopoulos,et al.  Dose effect relationship for late side effects of the rectum and urinary bladder in magnetic resonance image-guided adaptive cervix cancer brachytherapy. , 2012, International journal of radiation oncology, biology, physics.

[96]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[97]  Dongyang Kuang,et al.  Cycle-Consistent Training for Reducing Negative Jacobian Determinant in Deep Registration Networks , 2019, SASHIMI@MICCAI.

[98]  Arno Klein,et al.  A reproducible evaluation of ANTs similarity metric performance in brain image registration , 2011, NeuroImage.