Triple-Input-Unsupervised neural Networks for deformable image registration

Abstract Deep-learning frameworks have been widely used in deformable image registration(DIR), which is a fundamental step in medical image analysis. In traditional deep-learning-based DIR methods, pairwise registration is conducted for each image, cooperated with a dual-input structure. They only considered that the registered images need to be as similar as possible to the target image, while ignored that all registered images to the same target image should have similar appearance too. Therefore, in this paper we add the deformation constraints between different registered images to the registration network in order to enhance the appearance consistency of the registered images and improve the registration accuracy. Specifically, a Triple-Input-Unsupervised neural Network (TIUNet) is proposed to register two source images to a common target image at the same time. Then, a novel deformation-strengthened loss is designed to simulate the deformation constraint between the two warped source images. Finally, the deformation constraints between the source images and the target image are jointly optimized to obtain the deformation field model. The experimental results on the available tissue segmentation database demonstrate that our proposed TIUNet method shows a superior performance of deformation field model over several other advanced deformable image registration algorithms. Moreover, benefited from the triple-input structure, our proposed TIUNet method could expand the available training sets naturally and is suitable for the registration task with fewer training data.

[1]  Jean-Philippe Thirion,et al.  Image matching as a diffusion process: an analogy with Maxwell's demons , 1998, Medical Image Anal..

[2]  Maxime Sermesant,et al.  SVF-Net: Learning Deformable Image Registration Using Shape Matching , 2017, MICCAI.

[3]  Ben Glocker,et al.  On the Adaptability of Unsupervised CNN-Based Deformable Image Registration to Unseen Image Domains , 2018, MLMI@MICCAI.

[4]  P. Ellen Grant,et al.  Frequency Diffeomorphisms for Efficient Image Registration , 2017, IPMI.

[5]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[6]  Nikos Paragios,et al.  Slice-to-volume medical image registration: a survey , 2017, Medical Image Anal..

[7]  Ajmal Mian,et al.  Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey , 2018, IEEE Access.

[8]  Nassir Navab,et al.  Dense image registration through MRFs and efficient linear programming , 2008, Medical Image Anal..

[9]  Arthur W. Toga,et al.  Construction of a 3D probabilistic atlas of human cortical structures , 2008, NeuroImage.

[10]  Ji Luo,et al.  Unsupervised 3D End-to-End Medical Image Registration With Volume Tweening Network , 2019, IEEE Journal of Biomedical and Health Informatics.

[11]  Brian B. Avants,et al.  Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain , 2008, Medical Image Anal..

[12]  Dinggang Shen,et al.  Deformable Image Registration Using a Cue-Aware Deep Regression Network , 2018, IEEE Transactions on Biomedical Engineering.

[13]  Z. Jane Wang,et al.  A CNN Regression Approach for Real-Time 2D/3D Registration , 2016, IEEE Transactions on Medical Imaging.

[14]  Diego H. Milone,et al.  Learning Deformable Registration of Medical Images with Anatomical Constraints , 2020, Neural Networks.

[15]  Max A. Viergever,et al.  End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network , 2017, DLMIA/ML-CDS@MICCAI.

[16]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[18]  Pingkun Yan,et al.  Deep learning in medical image registration: a survey , 2020, Machine Vision and Applications.

[19]  Sheng Xu,et al.  Adversarial Image Registration with Application for MR and TRUS Image Fusion , 2018, MLMI@MICCAI.

[20]  Hans J. Johnson,et al.  Advanced Normalization Tools (ANTs) , 2020 .

[21]  Ruzena Bajcsy,et al.  Multiresolution elastic matching , 1989, Comput. Vis. Graph. Image Process..

[22]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[23]  Qolamreza R. Razlighi,et al.  Evaluating similarity measures for brain image registration , 2013, J. Vis. Commun. Image Represent..

[24]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.

[25]  Karl J. Friston,et al.  Voxel-Based Morphometry—The Methods , 2000, NeuroImage.

[26]  Yong Fan,et al.  Non-rigid image registration using fully convolutional networks with deep self-supervision , 2017, ArXiv.

[27]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[28]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[29]  Polina Golland,et al.  Patch-Based Discrete Registration of Clinical Brain Images , 2016, Patch-MI@MICCAI.

[30]  Jun Zhang,et al.  Inverse-Consistent Deep Networks for Unsupervised Deformable Image Registration , 2018, ArXiv.

[31]  Bruce Fischl,et al.  FreeSurfer , 2012, NeuroImage.

[32]  Arthur W. Toga,et al.  Atlas-based whole brain white matter analysis using large deformation diffeomorphic metric mapping: Application to normal elderly and Alzheimer's disease participants , 2009, NeuroImage.

[33]  Olivier D. Faugeras,et al.  Variational Methods for Multimodal Image Matching , 2002, International Journal of Computer Vision.

[34]  Z. Jane Wang,et al.  Real-time 2D/3D registration via CNN regression , 2015, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[35]  Yong Fan,et al.  Non-rigid image registration using self-supervised fully convolutional networks without training data , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[36]  Jong Chul Ye,et al.  Unsupervised Deformable Image Registration Using Cycle-Consistent CNN , 2019, MICCAI.

[37]  Yabo Fu,et al.  Deep Learning in Medical Image Registration: A Review , 2020, Physics in medicine and biology.