Unsupervised 3D End-to-End Medical Image Registration With Volume Tweening Network

3D medical image registration is of great clinical importance. However, supervised learning methods require a large amount of accurately annotated corresponding control points (or morphing), which are very difficult to obtain. Unsupervised learning methods ease the burden of manual annotation by exploiting unlabeled data without supervision. In this article, we propose a new unsupervised learning method using convolutional neural networks under an end-to-end framework, Volume Tweening Network (VTN), for 3D medical image registration. We propose three innovative technical components: (1) An end-to-end cascading scheme that resolves large displacement; (2) An efficient integration of affine registration network; and (3) An additional invertibility loss that encourages backward consistency. Experiments demonstrate that our algorithm is 880x faster (or 3.3x faster without GPU acceleration) than traditional optimization-based methods and achieves state-of-the-art performance in medical image registration.

[1]  Andrew Melbourne,et al.  Image similarity metrics in image registration , 2010, Medical Imaging.

[2]  Yue Dong,et al.  Recursive Cascaded Networks for Unsupervised Medical Image Registration , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[3]  Hervé Delingette,et al.  Unsupervised Probabilistic Deformation Modeling for Robust Diffeomorphic Registration , 2018, DLMIA/ML-CDS@MICCAI.

[4]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[5]  Bingbing Ni,et al.  Unsupervised Deep Learning for Optical Flow Estimation , 2017, AAAI.

[6]  Olivier D. Faugeras,et al.  Variational Methods for Multimodal Image Matching , 2002, International Journal of Computer Vision.

[7]  Oisin Mac Aodha,et al.  Unsupervised Monocular Depth Estimation with Left-Right Consistency , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[9]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[10]  Shu Liao,et al.  Unsupervised Deep Feature Learning for Deformable Registration of MR Brain Images , 2013, MICCAI.

[11]  Daniel S. Margulies,et al.  The Neuro Bureau ADHD-200 Preprocessed repository , 2016, NeuroImage.

[12]  Max A. Viergever,et al.  A deep learning framework for unsupervised affine and deformable image registration , 2018, Medical Image Anal..

[13]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Konstantinos G. Derpanis,et al.  Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness , 2016, ECCV Workshops.

[15]  Max A. Viergever,et al.  End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network , 2017, DLMIA/ML-CDS@MICCAI.

[16]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[17]  Wen Yan,et al.  Unsupervised End-to-end Learning for Deformable Medical Image Registration , 2017, ArXiv.

[18]  Gustavo Carneiro,et al.  Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue , 2016, ECCV.

[19]  Jan Modersitzki,et al.  FAIR: Flexible Algorithms for Image Registration , 2009 .

[20]  Thomas Brox,et al.  FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Xu Han,et al.  Networks for Joint Affine and Non-Parametric Image Registration , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Hervé Delingette,et al.  Learning a Probabilistic Model for Diffeomorphic Registration , 2018, IEEE Transactions on Medical Imaging.

[23]  Martin Styner,et al.  Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets , 2009, IEEE Transactions on Medical Imaging.

[24]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Brian C. Ross.  Mutual Information between Discrete and Continuous Data Sets , 2014, PloS one.

[26]  Arthur W. Toga,et al.  Construction of a 3D probabilistic atlas of human cortical structures , 2008, NeuroImage.

[27]  Z. Jane Wang,et al.  A CNN Regression Approach for Real-Time 2D/3D Registration , 2016, IEEE Transactions on Medical Imaging.

[28]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[29]  Gregory Valiant,et al.  Estimating the Unseen , 2017, J. ACM.

[30]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[31]  Daniel P. Kennedy,et al.  The Autism Brain Imaging Data Exchange: Towards Large-Scale Evaluation of the Intrinsic Brain Architecture in Autism , 2013, Molecular Psychiatry.

[32]  João Manuel R S Tavares,et al.  Medical image registration: a review , 2014, Computer methods in biomechanics and biomedical engineering.

[33]  Colin Studholme,et al.  An overlap invariant entropy measure of 3D medical image alignment , 1999, Pattern Recognit..

[34]  Nikos Paragios,et al.  Shape registration in implicit spaces using information theory and free form deformations , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[36]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[37]  C. Jack,et al.  Ways toward an early diagnosis in Alzheimer’s disease: The Alzheimer’s Disease Neuroimaging Initiative (ADNI) , 2005, Alzheimer's & Dementia.

[38]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[39]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[40]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[41]  Daniel Rueckert,et al.  Nonrigid registration using free-form deformations: application to breast MR images , 1999, IEEE Transactions on Medical Imaging.

[42]  Bernhard Schölkopf,et al.  Learning similarity measure for multi-modal 3D image registration , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.