ContraReg: Contrastive Learning of Multi-modality Unsupervised Deformable Image Registration

Establishing voxelwise semantic correspondence across distinct imaging modalities is a foundational yet formidable computer vision task. Current multi-modality registration techniques maximize hand-crafted inter-domain similarity functions, are limited in modeling nonlinear intensity-relationships and deformations, and may require significant re-engineering or underperform on new tasks, datasets, and domain pairs. This work presents ContraReg, an unsupervised contrastive representation learning approach to multi-modality deformable registration. By projecting learned multi-scale local patch features onto a jointly learned inter-domain embedding space, ContraReg obtains representations useful for non-rigid multi-modality alignment. Experimentally, ContraReg achieves accurate and robust results with smooth and invertible deformations across a series of baselines and ablations on a neonatal T1-T2 brain MRI registration task with all methods validated over a wide range of deformation regularization strengths.

[1]  M. Styner,et al.  Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis , 2022, NeurIPS.

[2]  Albert C. S. Chung,et al.  Learn2Reg: Comprehensive Multi-Task Medical Image Registration Challenge, Dataset and Evaluation in the Era of Deep Learning , 2021, IEEE Transactions on Medical Imaging.

[3]  Yann LeCun,et al.  Understanding Dimensional Collapse in Contrastive Self-supervised Learning , 2021, ICLR.

[4]  Albert C. S. Chung,et al.  Conditional Deformable Image Registration with Convolutional Neural Network , 2021, MICCAI.

[5]  Adrian V. Dalca,et al.  Generative Adversarial Registration for Improved Conditional Deformable Templates , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Steffen Czolbe,et al.  Semantic similarity metrics for learned image registration , 2021, MIDL.

[7]  M. Shoeiby,et al.  Dual Contrastive Learning for Unsupervised Image-to-Image Translation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Joakim Lindblad,et al.  Is image-to-image translation the panacea for multimodal image registration? A comparative study , 2021, PloS one.

[9]  James S. Duncan,et al.  Anatomy-guided Multimodal Registration by Learning Segmentation without Ground Truth: Application to Intraprocedural CBCT/MR Liver Segmentation and Registration , 2021, Medical Image Anal..

[10]  G. Gerig,et al.  Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization , 2021, IEEE Transactions on Medical Imaging.

[11]  Adrian V. Dalca,et al.  HyperMorph: Amortized Hyperparameter Learning for Image Registration , 2021, IPMI.

[12]  Alexei A. Efros,et al.  Contrastive Learning for Unpaired Image-to-Image Translation , 2020, ECCV.

[13]  Geoffrey E. Hinton,et al.  Big Self-Supervised Models are Strong Semi-Supervised Learners , 2020, NeurIPS.

[14]  Joakim Lindblad,et al.  CoMIR: Contrastive Multimodal Image Representation for Registration , 2020, NeurIPS.

[15]  Bruce Fischl,et al.  SynthMorph: Learning Contrast-Invariant Registration Without Acquired Images , 2020, IEEE Transactions on Medical Imaging.

[16]  Daniel Cohen-Or,et al.  Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Daniel Rueckert,et al.  Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations , 2019, IPMI.

[18]  Mert R. Sabuncu,et al.  Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces , 2019, Medical Image Anal..

[19]  Guido Gerig,et al.  Multi-modal image fusion for multispectral super-resolution in microscopy , 2019, Medical Imaging: Image Processing.

[20]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[21]  Nassir Navab,et al.  Guiding multimodal registration with learned optimization updates , 2017, Medical Image Anal..

[22]  Aaron C. Courville,et al.  FiLM: Visual Reasoning with a General Conditioning Layer , 2017, AAAI.

[23]  Andreas Schuh,et al.  Computational models of the morphology of the developing neonatal human brain , 2017 .

[24]  Daniel Rueckert,et al.  The developing human connectome project: A minimal processing pipeline for neonatal cortical surface reconstruction , 2017, NeuroImage.

[25]  Quoc V. Le,et al.  HyperNetworks , 2016, ICLR.

[26]  Nikos Komodakis,et al.  A Deep Metric for Multimodal Registration , 2016, MICCAI.

[27]  Jerry L. Prince,et al.  Multimodal Registration via Mutual Information Incorporating Geometric and Spatial Context , 2015, IEEE Transactions on Image Processing.

[28]  Daniel Rueckert,et al.  Automatic Whole Brain MRI Segmentation of the Developing Neonatal Brain , 2014, IEEE Transactions on Medical Imaging.

[29]  Michael Brady,et al.  MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration , 2012, Medical Image Anal..

[30]  William Wells,et al.  Multimodal image registration for preoperative planning and image-guided neurosurgical procedures. , 2011, Neurosurgery clinics of North America.

[31]  Arno Klein,et al.  A reproducible evaluation of ANTs similarity metric performance in brain image registration , 2011, NeuroImage.

[32]  Bernhard Schölkopf,et al.  Learning similarity measure for multi-modal 3D image registration , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Eldad Haber,et al.  Intensity Gradient Based Registration and Fusion of Multi-modal Images , 2006, MICCAI.

[34]  Christopher Nimsky,et al.  Intraoperative visualization of the pyramidal tract by diffusion-tensor-imaging-based fiber tracking , 2006, NeuroImage.

[35]  Carlo Tomasi,et al.  Image Similarity Using Mutual Information of Regions , 2004, ECCV.

[36]  Takeyoshi Dohi,et al.  Multimodality Deformable Registration of Pre- and Intraoperative Images for MRI-guided Brain Surgery , 1998, MICCAI.

[37]  Paul A. Viola,et al.  Multi-modal volume registration by maximization of mutual information , 1996, Medical Image Anal..

[38]  Daniel Rueckert,et al.  Learning Diffeomorphic and Modality-invariant Registration using B-splines , 2021, MIDL.

[39]  Nassir Navab,et al.  Entropy and Laplacian images: Structural representations for multi-modal registration , 2012, Medical Image Anal..

[40]  Paul Suetens,et al.  Nonrigid Image Registration Using Conditional Mutual Information , 2010, IEEE Transactions on Medical Imaging.

[41]  Colin Studholme,et al.  An overlap invariant entropy measure of 3D medical image alignment , 1999, Pattern Recognit..