Inverse-Consistent Deep Networks for Unsupervised Deformable Image Registration

Deformable image registration is a fundamental task in medical image analysis, aiming to establish a dense and non-linear correspondence between a pair of images. Previous deep-learning studies usually employ supervised neural networks to directly learn the spatial transformation from one image to another, requiring task-specific ground-truth registration for model training. Due to the difficulty in collecting precise ground-truth registration, implementation of these supervised methods is practically challenging. Although several unsupervised networks have been recently developed, these methods usually ignore the inherent inverse-consistent property (essential for diffeomorphic mapping) of transformations between a pair of images. Also, existing approaches usually encourage the to-be-estimated transformation to be locally smooth via a smoothness constraint only, which could not completely avoid folding in the resulting transformation. To this end, we propose an Inverse-Consistent deep Network (ICNet) for unsupervised deformable image registration. Specifically, we develop an inverse-consistent constraint to encourage that a pair of images are symmetrically deformed toward one another, until both warped images are matched. Besides using the conventional smoothness constraint, we also propose an anti-folding constraint to further avoid folding in the transformation. The proposed method does not require any supervision information, while encouraging the diffeomoprhic property of the transformation via the proposed inverse-consistent and anti-folding constraints. We evaluate our method on T1-weighted brain magnetic resonance imaging (MRI) scans for tissue segmentation and anatomical landmark detection, with results demonstrating the superior performance of our ICNet over several state-of-the-art approaches for deformable image registration. Our code will be made publicly available.

[1]  M. Staring,et al.  A rigidity penalty term for nonrigid registration. , 2007, Medical physics.

[2]  Wen Yan,et al.  Unsupervised End-to-end Learning for Deformable Medical Image Registration , 2017, ArXiv.

[3]  Nick C Fox,et al.  The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods , 2008, Journal of magnetic resonance imaging : JMRI.

[4]  Guy Marchal,et al.  Multimodality image registration by maximization of mutual information , 1997, IEEE Transactions on Medical Imaging.

[5]  Torsten Rohlfing,et al.  Volume-preserving nonrigid registration of MR breast images using free-form deformation with an incompressibility constraint , 2003, IEEE Transactions on Medical Imaging.

[6]  Dinggang Shen,et al.  HAMMER: hierarchical attribute matching mechanism for elastic registration , 2002, IEEE Transactions on Medical Imaging.

[7]  Dinggang Shen,et al.  Deformable Image Registration Using a Cue-Aware Deep Regression Network , 2018, IEEE Transactions on Biomedical Engineering.

[8]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[9]  Scott T. Grafton,et al.  Automated image registration: I. General methods and intrasubject, intramodality validation. , 1998, Journal of computer assisted tomography.

[10]  Yaozong Gao,et al.  Learning‐based deformable registration for infant MRI by integrating random forest with auto‐context model , 2017, Medical physics.

[11]  Simon R. Arridge,et al.  A survey of hierarchical non-linear medical image registration , 1999, Pattern Recognit..

[12]  Yaozong Gao,et al.  Region-Adaptive Deformable Registration of CT/MRI Pelvic Images via Learning-Based Image Synthesis , 2018, IEEE Transactions on Image Processing.

[13]  Gary E. Christensen,et al.  Consistent image registration , 2001, IEEE Transactions on Medical Imaging.

[14]  Zhe Chen,et al.  Constrained non-rigid registration for use in image-guided adaptive radiotherapy , 2009, Medical Image Anal..

[15]  Brian B. Avants,et al.  Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain , 2008, Medical Image Anal..

[16]  Daniel Rueckert,et al.  Diffeomorphic Registration Using B-Splines , 2006, MICCAI.

[17]  Max A. Viergever,et al.  End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network , 2017, DLMIA/ML-CDS@MICCAI.

[18]  Nicholas Ayache,et al.  LCC-Demons: A robust and accurate symmetric diffeomorphic registration algorithm , 2013, NeuroImage.

[19]  Julien Dauguet,et al.  Locally constrained cubic B-spline deformations to control volume variations , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[20]  Qian Wang,et al.  Deformable Image Registration Based on Similarity-Steered CNN Regression , 2017, MICCAI.

[21]  Dinggang Shen,et al.  Multi‐channel multi‐scale fully convolutional network for 3D perivascular spaces segmentation in 7T MR images , 2018, Medical Image Anal..

[22]  Mark Holden,et al.  A Review of Geometric Transformations for Nonrigid Body Registration , 2008, IEEE Transactions on Medical Imaging.

[23]  Hervé Delingette,et al.  International Journal of Computer Vision Manuscript No. 10.1007/s11263-010-0405-z The final publication is available at www.springerlink.com iLogDemons: A Demons-Based Registration Algorithm for Tracking Incompressible Elastic Biological Tissues , 2022 .

[24]  Jan Modersitzki,et al.  FLIRT with Rigidity—Image Registration with a Local Non-rigidity Penalty , 2008, International Journal of Computer Vision.

[25]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[26]  Michaël Sdika,et al.  A Fast Nonrigid Image Registration With Constraints on the Jacobian Using Large Scale Constrained Optimization , 2008, IEEE Transactions on Medical Imaging.

[27]  G. Marchal,et al.  Multi-modal volume registration by maximization of mutual information , 1997 .

[28]  Fabrice Heitz,et al.  Topology preserving deformable image matching using constrained hierarchical parametric models , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[29]  Daniel Rueckert,et al.  Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy , 2009, NeuroImage.

[30]  Z. Jane Wang,et al.  A CNN Regression Approach for Real-Time 2D/3D Registration , 2016, IEEE Transactions on Medical Imaging.

[31]  Xiao Yang,et al.  Fast Predictive Image Registration , 2016, LABELS/DLMIA@MICCAI.

[32]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[33]  Dong Liu,et al.  Robust visual domain adaptation with low-rank reconstruction , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Mark Jenkinson,et al.  Medical image registration , 2001 .

[35]  Daniel Rueckert,et al.  Nonrigid registration using free-form deformations: application to breast MR images , 1999, IEEE Transactions on Medical Imaging.

[36]  Ruzena Bajcsy,et al.  Multiresolution elastic matching , 1989, Comput. Vis. Graph. Image Process..

[37]  Nassir Navab,et al.  Dense image registration through MRFs and efficient linear programming , 2008, Medical Image Anal..

[38]  Paul Suetens,et al.  Nonrigid Image Registration Using Free-Form Deformations with a Local Rigidity Constraint , 2004, MICCAI.

[39]  Dinggang Shen,et al.  Predict brain MR image registration via sparse learning of appearance and transformation , 2015, Medical Image Anal..

[40]  Nikos Paragios,et al.  DRAMMS: Deformable Registration via Attribute Matching and Mutual-Saliency Weighting , 2009, IPMI.

[41]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[42]  Daniel Rueckert,et al.  Volume and Shape Preservation of Enhancing Lesions when Applying Non-rigid Registration to a Time Series of Contrast Enhancing MR Breast Images , 2000, MICCAI.

[43]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.

[44]  Babak A. Ardekani,et al.  Quantitative comparison of algorithms for inter-subject registration of 3D volumetric brain MRI scans , 2005, Journal of Neuroscience Methods.

[45]  Dinggang Shen,et al.  Simulating deformations of MR brain images for validation of atlas-based segmentation and registration algorithms , 2006, NeuroImage.

[46]  Yaozong Gao,et al.  Dual‐core steered non‐rigid registration for multi‐modal images via bi‐directional image synthesis , 2017, Medical Image Anal..

[47]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[48]  Paul M. Thompson,et al.  Inverse Consistent Mapping in 3D Deformable Image Registration: Its Construction and Statistical Properties , 2005, IPMI.

[49]  Nassir Navab,et al.  Learning Optimization Updates for Multimodal Registration , 2016, MICCAI.

[50]  Shu Liao,et al.  Unsupervised Deep Feature Learning for Deformable Registration of MR Brain Images , 2013, MICCAI.

[51]  Nikhil Ketkar,et al.  Introduction to PyTorch , 2021, Deep Learning with Python.

[52]  D. Louis Collins,et al.  Animal: Validation and Applications of Nonlinear Registration-Based Segmentation , 1997, Int. J. Pattern Recognit. Artif. Intell..

[53]  Hervé Delingette,et al.  Robust Non-rigid Registration Through Agent-Based Action Learning , 2017, MICCAI.

[54]  Isabelle Corouge,et al.  Inter Subject Registration of Functional and Anatomical Data Using SPM , 2002, MICCAI.

[55]  Dinggang Shen,et al.  A General Fast Registration Framework by Learning Deformation–Appearance Correlation , 2012, IEEE Transactions on Image Processing.

[56]  Tom Vercauteren,et al.  Diffeomorphic demons: Efficient non-parametric image registration , 2009, NeuroImage.

[57]  Alan C. Evans,et al.  Enhancement of MR Images Using Registration for Signal Averaging , 1998, Journal of Computer Assisted Tomography.

[58]  Jean-Philippe Thirion,et al.  Image matching as a diffusion process: an analogy with Maxwell's demons , 1998, Medical Image Anal..

[59]  Rama Chellappa,et al.  Unsupervised Adaptation Across Domain Shifts by Generating Intermediate Data Representations , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Dinggang Shen,et al.  Landmark‐based deep multi‐instance learning for brain disease diagnosis , 2018, Medical Image Anal..

[61]  Jun Zhang,et al.  Detecting Anatomical Landmarks From Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural Networks , 2017, IEEE Transactions on Image Processing.

[62]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[63]  Maxime Sermesant,et al.  SVF-Net: Learning Deformable Image Registration Using Shape Matching , 2017, MICCAI.

[64]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[65]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[66]  Oskar M. Skrinjar,et al.  Myocardial deformation recovery from cine MRI using a nearly incompressible biventricular model , 2008, Medical Image Anal..

[67]  Gary E. Christensen,et al.  Large Deformation Inverse Consistent Elastic Image Registration , 2003, IPMI.

[68]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.