Dual‐core steered non‐rigid registration for multi‐modal images via bi‐directional image synthesis

HighlightsA bi‐directional image synthesis based multi‐modal registration method is proposed.Using complementary details from both modalities to guide non‐rigid registration.We tackle the challenging problem of MRI synthesis from single CT modality.Dual‐core deformation fusion framework is proposed to guide accurate registration Abstract In prostate cancer radiotherapy, computed tomography (CT) is widely used for dose planning purposes. However, because CT has low soft tissue contrast, it makes manual contouring difficult for major pelvic organs. In contrast, magnetic resonance imaging (MRI) provides high soft tissue contrast, which makes it ideal for accurate manual contouring. Therefore, the contouring accuracy on CT can be significantly improved if the contours in MRI can be mapped to CT domain by registering MRI with CT of the same subject, which would eventually lead to high treatment efficacy. In this paper, we propose a bi‐directional image synthesis based approach for MRI‐to‐CT pelvic image registration. First, we use patch‐wise random forest with auto‐context model to learn the appearance mapping from CT to MRI domain, and then vice versa. Consequently, we can synthesize a pseudo‐MRI whose anatomical structures are exactly same with CT but with MRI‐like appearance, and a pseudo‐CT as well. Then, our MRI‐to‐CT registration can be steered in a dual manner, by simultaneously estimating two deformation pathways: 1) one from the pseudo‐CT to the actual CT and 2) another from actual MRI to the pseudo‐MRI. Next, a dual‐core deformation fusion framework is developed to iteratively and effectively combine these two registration pathways by using complementary information from both modalities. Experiments on a dataset with real pelvic CT and MRI have shown improved registration performance of the proposed method by comparing it to the conventional registration methods, thus indicating its high potential of translation to the routine radiation therapy. Graphical abstract No Caption available.

[1]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[2]  Yaozong Gao,et al.  Learning Appearance and Shape Evolution for Infant Image Registration in the First Year of Life , 2016, MLMI@MICCAI.

[3]  D. Chan,et al.  The use of prostate specific antigen, clinical stage and Gleason score to predict pathological stage in men with localized prostate cancer. , 1993, The Journal of urology.

[4]  Radhe Mohan,et al.  Increased risk of biochemical and local failure in patients with distended rectum on the planning CT for prostate cancer radiotherapy. , 2005, International journal of radiation oncology, biology, physics.

[5]  William M. Wells,et al.  Feature-Based Alignment of Volumetric Multi-modal Images , 2013, IPMI.

[6]  Michael Brady,et al.  MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration , 2012, Medical Image Anal..

[7]  Nicholas Ayache,et al.  Rigid registration of 3-D ultrasound with MR images: a new approach combining intensity and gradient information , 2001, IEEE Transactions on Medical Imaging.

[8]  Hervé Delingette,et al.  Extended Modality Propagation: Image Synthesis of Pathological Cases , 2016, IEEE Transactions on Medical Imaging.

[9]  Adam Johansson,et al.  CT substitute derived from MRI sequences with ultrashort echo time. , 2011, Medical physics.

[10]  Amod Jog,et al.  Improving magnetic resonance resolution with supervised learning , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[11]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[12]  Olivier Salvado,et al.  An atlas-based electron density mapping method for magnetic resonance imaging (MRI)-alone treatment planning and adaptive MRI-based prostate radiation therapy. , 2012, International journal of radiation oncology, biology, physics.

[13]  J. Ashburner,et al.  Multimodal Image Coregistration and Partitioning—A Unified Framework , 1997, NeuroImage.

[14]  G. Marchal,et al.  Multi-modal volume registration by maximization of mutual information , 1997 .

[15]  Dinggang Shen,et al.  HAMMER: hierarchical attribute matching mechanism for elastic registration , 2002, IEEE Transactions on Medical Imaging.

[16]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[17]  Yaozong Gao,et al.  LINKS: Learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images , 2015, NeuroImage.

[18]  Amod Jog,et al.  Random forest FLAIR reconstruction from T1, T2, and PD-weighted MRI , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[19]  Albert C. S. Chung,et al.  Learning-based multi-modal rigid image registration by using Bhattacharyya distances , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[20]  Dinggang Shen,et al.  Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis , 2014, MICCAI.

[21]  Snehashis Roy,et al.  MR contrast synthesis for lesion segmentation , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[22]  Arno Klein,et al.  A reproducible evaluation of ANTs similarity metric performance in brain image registration , 2011, NeuroImage.

[23]  Yaozong Gao,et al.  LINKS: Learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images , 2014, NeuroImage.

[24]  Yaozong Gao,et al.  Accurate Segmentation of CT Male Pelvic Organs via Regression-Based Deformable Models and Multi-Task Random Forests , 2016, IEEE Transactions on Medical Imaging.

[25]  Nassir Navab,et al.  Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention , 2008, Medical Image Anal..

[26]  Ben Glocker,et al.  Modality Propagation: Coherent Synthesis of Subject-Specific Scans with Data-Driven Regularization , 2013, MICCAI.

[27]  Yaozong Gao,et al.  Estimating CT Image From MRI Data Using Structured Random Forest and Auto-Context Model , 2016, IEEE Transactions on Medical Imaging.

[28]  Dinggang Shen,et al.  Optimized prostate biopsy via a statistical atlas of cancer spatial distribution , 2004, Medical Image Anal..

[29]  Arno Klein,et al.  Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration , 2009, NeuroImage.

[30]  M. Barton,et al.  The Potential for an Enhanced Role for MRI in Radiation-therapy Treatment Planning , 2013, Technology in cancer research & treatment.

[31]  G L Sannazzari,et al.  CT-MRI image fusion for delineation of volumes in three-dimensional conformal radiation therapy in the treatment of localized prostate cancer. , 2002, The British journal of radiology.

[32]  Ben Glocker,et al.  Is Synthesizing MRI Contrast Useful for Inter-modality Analysis? , 2013, MICCAI.

[33]  Snehashis Roy,et al.  Magnetic Resonance Image Example-Based Contrast Synthesis , 2013, IEEE Transactions on Medical Imaging.

[34]  Jerry L Prince,et al.  PET Attenuation Correction Using Synthetic CT from Ultrashort Echo-Time MR Imaging , 2014, The Journal of Nuclear Medicine.

[35]  Guy Marchal,et al.  Automated multi-moda lity image registration based on information theory , 1995 .

[36]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[37]  Yaozong Gao,et al.  Learning-Based Multimodal Image Registration for Prostate Cancer Radiation Therapy , 2016, MICCAI.

[38]  Brian B. Avants,et al.  Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain , 2008, Medical Image Anal..

[39]  Zhuowen Tu,et al.  Auto-Context and Its Application to High-Level Vision Tasks and 3D Brain Image Segmentation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Dinggang Shen,et al.  Non-diffeomorphic registration of brain tumor images by simulating tissue loss and tumor growth , 2009, NeuroImage.

[41]  Stefan Klein,et al.  Automatic segmentation of the prostate in 3D MR images by atlas matching using localized mutual information. , 2008, Medical physics.

[42]  Dinggang Shen,et al.  Iterative multi-atlas-based multi-image segmentation with tree-based registration , 2012, NeuroImage.

[43]  Guy Marchal,et al.  Automated multi-modality image registration based on information theory , 1995 .

[44]  Colin Studholme,et al.  An overlap invariant entropy measure of 3D medical image alignment , 1999, Pattern Recognit..

[45]  D. Dearnaley,et al.  Comparison of radiation side-effects of conformal and conventional radiotherapy in prostate cancer: a randomised trial , 1999, The Lancet.

[46]  Yaozong Gao,et al.  Detecting Anatomical Landmarks for Fast Alzheimer’s Disease Diagnosis , 2016, IEEE Transactions on Medical Imaging.

[47]  Guy Marchal,et al.  Multimodality image registration by maximization of mutual information , 1997, IEEE Transactions on Medical Imaging.

[48]  Snehashis Roy,et al.  Magnetic resonance image synthesis through patch regression , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[49]  Nikos Paragios,et al.  DRAMMS: Deformable Registration via Attribute Matching and Mutual-Saliency Weighting , 2009, IPMI.

[50]  Tom Vercauteren,et al.  Diffeomorphic demons: Efficient non-parametric image registration , 2009, NeuroImage.

[51]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[52]  W. Eric L. Grimson,et al.  Multi-modal Image Registration by Minimising Kullback-Leibler Distance , 2002, MICCAI.

[53]  Snehashis Roy,et al.  A Compressed Sensing Approach for MR Tissue Contrast Synthesis , 2011, IPMI.

[54]  Daniel Rueckert,et al.  Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II , 2017, Lecture Notes in Computer Science.

[55]  H. Hricak,et al.  Prostate volumes defined by magnetic resonance imaging and computerized tomographic scans for three-dimensional conformal radiotherapy. , 1996, International journal of radiation oncology, biology, physics.