Semantic similarity metrics for learned image registration

We propose a semantic similarity metric for image registration. Existing metrics like euclidean distance or normalized cross-correlation focus on aligning intensity values, giving difficulties with low intensity contrast or noise. Our approach learns dataset-specific features that drive the optimization of a learning-based registration model. We train both an unsupervised approach using an auto-encoder, and a semi-supervised approach using supplemental segmentation data to extract semantic features for image registration. Comparing to existing methods across multiple image modalities and applications, we achieve consistently high registration accuracy. A learned invariance to noise gives smoother transformations on low-quality images. Code and experiments are available at github.com/SteffenCzolbe/DeepSimRegistration.

[1]  Sheng Xu,et al.  Learning deep similarity metric for 3D MR–TRUS image registration , 2018, International Journal of Computer Assisted Radiology and Surgery.

[2]  Yong Fan,et al.  Non-rigid image registration using self-supervised fully convolutional networks without training data , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[3]  Daniel Rueckert,et al.  Nonrigid registration using free-form deformations: application to breast MR images , 1999, IEEE Transactions on Medical Imaging.

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  Alain Trouvé,et al.  Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms , 2005, International Journal of Computer Vision.

[6]  Mert R. Sabuncu,et al.  Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration , 2018, MICCAI.

[7]  Adam Anderson,et al.  Designing deep neural networks to automate segmentation for serial block-face electron microscopy , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[8]  C. Igel,et al.  A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model , 2020, NeurIPS.

[9]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[10]  Max A. Viergever,et al.  A deep learning framework for unsupervised affine and deformable image registration , 2018, Medical Image Anal..

[11]  Nicholas Ayache,et al.  Non-parametric Diffeomorphic Image Registration with the Demons Algorithm , 2007, MICCAI.

[12]  Brian B. Avants,et al.  Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain , 2008, Medical Image Anal..

[13]  Nikos Komodakis,et al.  A Deep Metric for Multimodal Registration , 2016, MICCAI.

[14]  Nathalie Harder,et al.  A benchmark for comparison of cell tracking algorithms , 2014, Bioinform..

[15]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[16]  Nathalie Harder,et al.  An Objective Comparison of Cell Tracking Algorithms , 2017, Nature Methods.

[17]  Pheng-Ann Heng,et al.  Probabilistic Multilayer Regularization Network for Unsupervised 3D Brain Image Registration , 2019, MICCAI.

[18]  Dean C. Barratt,et al.  Conditional Segmentation in Lieu of Image Registration , 2019, MICCAI.

[19]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[20]  Ruzena Bajcsy,et al.  The Institute For Research In Cognitive Science Elastically Deforming a Three-Dimensional Atlas to Match Anatomical Brain Images , 1993 .

[21]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[22]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[23]  Michael Schöll,et al.  A Deep Learning Approach to MR-less Spatial Normalization for Tau PET Images , 2019, MICCAI.

[24]  LinLin Shen,et al.  Deep Feature Consistent Variational Autoencoder , 2016, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[25]  Mert R. Sabuncu,et al.  Learning Conditional Deformable Templates with Convolutional Networks , 2019, NeurIPS.

[26]  Arno Klein,et al.  A reproducible evaluation of ANTs similarity metric performance in brain image registration , 2011, NeuroImage.

[27]  Hervé Delingette,et al.  Robust Non-rigid Registration Through Agent-Based Action Learning , 2017, MICCAI.

[28]  Ben Glocker,et al.  Image-and-Spatial Transformer Networks for Structure-Guided Image Registration , 2019, MICCAI.

[29]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[30]  Marc Niethammer,et al.  DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation , 2019, MICCAI.

[31]  Michael Brady,et al.  MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration , 2012, Medical Image Anal..

[32]  Miao Kang,et al.  Dual-Stream Pyramid Registration Network , 2019, MICCAI.

[33]  Daniel S. Marcus,et al.  OASIS-3: Longitudinal Neuroimaging, Clinical, and Cognitive Dataset for Normal Aging and Alzheimer Disease , 2019 .

[34]  Jean-Philippe Thirion,et al.  Image matching as a diffusion process: an analogy with Maxwell's demons , 1998, Medical Image Anal..

[35]  Alexei A. Efros,et al.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[36]  Daniel P. Kennedy,et al.  The Autism Brain Imaging Data Exchange: Towards Large-Scale Evaluation of the Intrinsic Brain Architecture in Autism , 2013, Molecular Psychiatry.

[37]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.