A Deep Metric for Multimodal Registration

Multimodal registration is a challenging problem due the high variability of tissue appearance under different imaging modalities. The crucial component here is the choice of the right similarity measure. We make a step towards a general learning-based solution that can be adapted to specific situations and present a metric based on a convolutional neural network. Our network can be trained from scratch even from a few aligned image pairs. The metric is validated on intersubject deformable registration on a dataset different from the one used for training, demonstrating good generalization. In this task, we outperform mutual information by a significant margin.

[1]  Shaohua Kevin Zhou,et al.  Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network , 2015, MICCAI.

[2]  Nikos Komodakis,et al.  Learning to compare image patches via convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Bernhard Schölkopf,et al.  Learning similarity measure for multi-modal 3D image registration , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Daniel Rueckert,et al.  Magnetic resonance imaging of the newborn brain: Manual segmentation of labelled atlases in term-born and preterm infants , 2012, NeuroImage.

[5]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[7]  Michael Brady,et al.  MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration , 2012, Medical Image Anal..

[8]  Frank Sauer,et al.  Learning Based Non-rigid Multi-modal Image Registration Using Kullback-Leibler Divergence , 2005, MICCAI.

[9]  Ben Glocker,et al.  Deformable medical image registration: setting the state of the art with discrete methods. , 2011, Annual review of biomedical engineering.

[10]  Nikos Paragios,et al.  Boosted metric learning for 3D multi-modal deformable registration , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[11]  Muhammad Usman Akram,et al.  Information Extraction From Images , 2014 .

[12]  Marc Niethammer,et al.  Robust Multimodal Dictionary Learning , 2013, MICCAI.

[13]  Li Zhang,et al.  Deep similarity learning for multimodal medical images , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[14]  David R. Haynor,et al.  Nonrigid multimodality image registration , 2001, SPIE Medical Imaging.