Unsupervised Multi-modality Registration Network Based on Spatially Encoded Gradient Information

Multi-modality medical images can provide relevant or complementary information for a target (organ, tumor or tissue). Registering multi-modality images to a common space can fuse these comprehensive information, and bring convenience for clinical application. Recently, neural networks have been widely investigated to boost registration methods. However, it is still challenging to develop a multi-modality registration network due to the lack of robust criteria for network training. In this work, we propose a multi-modality registration network (MMRegNet), which can perform registration between multi-modality images. Meanwhile, we present spatially encoded gradient information to train MMRegNet in an unsupervised manner. The proposed network was evaluated on the public dataset from MM-WHS 2017. Results show that MMRegNet can achieve promising performance for left ventricle registration tasks. Meanwhile, to demonstrate the versatility of MMRegNet, we further evaluate the method using a liver dataset from CHAOS 2019. Our source code is publicly available.

[1]  Nassir Navab,et al.  Entropy and Laplacian images: Structural representations for multi-modal registration , 2012, Medical Image Anal..

[2]  Michael Brady,et al.  Towards Realtime Multimodal Fusion for Image-Guided Interventions Using Self-similarities , 2013, MICCAI.

[3]  Xiahai Zhuang,et al.  Cross-Modality Multi-Atlas Segmentation Using Deep Neural Networks , 2020, MICCAI.

[4]  Xiahai Zhuang,et al.  MvMM-RegNet: A new image registration framework based on multivariate mixture model and neural network estimation , 2020, MICCAI.

[5]  Daniel Cohen-Or,et al.  Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Jianfeng Xu,et al.  Medical Image Alignment by Normal Vector Information , 2005, CIS.

[7]  Simon R. Arridge,et al.  A Nonrigid Registration Framework Using Spatially Encoded Mutual Information and Free-Form Deformations , 2011, IEEE Transactions on Medical Imaging.

[8]  M V Knopp,et al.  Image fusion using CT, MRI and PET for treatment planning, navigation and follow up in percutaneous RFA. , 2009, Experimental oncology.

[9]  Guy Marchal,et al.  Multimodality image registration by maximization of mutual information , 1997, IEEE Transactions on Medical Imaging.

[10]  Xiahai Zhuang,et al.  Multivariate Mixture Model for Myocardial Segmentation Combining Multi-Source Images , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Andreas Nürnberger,et al.  CHAOS Challenge - Combined (CT-MR) Healthy Abdominal Organ Segmentation , 2020, Medical Image Anal..

[12]  Daniel Rueckert,et al.  Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations , 2019, IPMI.

[13]  Hans J. Johnson,et al.  Advanced Normalization Tools (ANTs) , 2020 .

[14]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[15]  Michael Brady,et al.  MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration , 2012, Medical Image Anal..

[16]  Lin Yang,et al.  Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Sehat Ullah,et al.  Medical image registration in image guided surgery: Issues, challenges and research opportunities , 2017 .

[18]  Thomas E Yankeelov,et al.  Co-registration of multi-modality imaging allows for comprehensive analysis of tumor-induced bone disease. , 2014, Bone.

[19]  Sébastien Ourselin,et al.  Weakly-supervised convolutional neural networks for multimodal image registration , 2018, Medical Image Anal..

[20]  Guang Yang,et al.  Evaluation of algorithms for Multi-Modality Whole Heart Segmentation: An open-access grand challenge , 2019, Medical Image Anal..

[21]  Colin Studholme,et al.  An overlap invariant entropy measure of 3D medical image alignment , 1999, Pattern Recognit..

[22]  Eldad Haber,et al.  Intensity Gradient Based Registration and Fusion of Multi-modal Images , 2006, MICCAI.

[23]  Yabo Fu,et al.  Deep Learning in Medical Image Registration: A Review , 2020, Physics in medicine and biology.

[24]  R. Baggott.  DISEASE , 1947, Social Policy & Administration.

[25]  Jan Kautz,et al.  Multimodal Unsupervised Image-to-Image Translation , 2018, ECCV.