Learning-Based Affine Registration of Histological Images

The use of different stains for histological sample preparation reveals distinct tissue properties and may result in a more accurate diagnosis. However, as a result of the staining process, the tissue slides are being deformed and registration is required before further processing. The importance of this problem led to organizing an open challenge named Automatic Non-rigid Histological Image Registration Challenge (ANHIR), organized jointly with the IEEE ISBI 2019 conference. The challenge organizers provided several hundred image pairs and a server-side evaluation platform. One of the most difficult sub-problems for the challenge participants was to find an initial, global transform, before attempting to calculate the final, non-rigid deformation field. This article solves the problem by proposing a deep network trained in an unsupervised way with a good generalization. We propose a method that works well for images with different resolutions, aspect ratios, without the necessity to perform image padding, while maintaining a low number of network parameters and fast forward pass time. The proposed method is orders of magnitude faster than the classical approach based on the iterative similarity metric optimization or computer vision descriptors. The success rate is above 98% for both the training set and the evaluation set. We make both the training and inference code freely available.

[1]  Max A. Viergever,et al.  A deep learning framework for unsupervised affine and deformable image registration , 2018, Medical Image Anal..

[2]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[3]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[4]  Jan Kybic,et al.  Benchmarking of Image Registration Methods for Differently Stained Histological Slides , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[5]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[6]  Stefan Heldmann,et al.  Robust, fast and accurate: a 3-step method for automatic histological image registration , 2019, ArXiv.

[7]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[8]  Marc Modat,et al.  Evaluation of MRI to Ultrasound Registration Methods for Brain Shift Correction: The CuRIOUS2018 Challenge , 2019, IEEE Transactions on Medical Imaging.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Xiaohuan Cao,et al.  Adversarial learning for mono- or multi-modal registration , 2019, Medical Image Anal..

[11]  Jochen Roller The Source Code , 2017 .

[12]  Spyridon Bakas,et al.  Accurate and Robust Alignment of Variable-stained Histologic Images Using a General-purpose Greedy Diffeomorphic Registration Tool , 2019, ArXiv.

[13]  Mary Helen Barcellos-Hoff,et al.  System for combined three‐dimensional morphological and molecular analysis of thick tissue specimens , 2002, Microscopy research and technique.

[14]  Kaiming He,et al.  Group Normalization , 2018, ECCV.

[15]  Dwarikanath Mahapatra,et al.  Deformable medical image registration using generative adversarial networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[16]  Mert R. Sabuncu,et al.  Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces , 2019, Medical Image Anal..

[17]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[18]  Raquel Urtasun,et al.  Understanding the Effective Receptive Field in Deep Convolutional Neural Networks , 2016, NIPS.

[19]  Tomasz Malisiewicz,et al.  Deep Image Homography Estimation , 2016, ArXiv.

[20]  Michael Gadermayr,et al.  Stain independent segmentation of whole slide images: A case study in renal histology , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[21]  Marek Wodzinski,et al.  Automatic Nonrigid Histological Image Registration with Adaptive Multistep Algorithm , 2019, ArXiv.

[22]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[23]  Zhenzhou Wu,et al.  AIRNet: Self-Supervised Affine Registration for 3D Medical Images using Neural Networks , 2018, ArXiv.

[24]  Pingkun Yan,et al.  Deep learning in medical image registration: a survey , 2020, Machine Vision and Applications.

[25]  Spyridon Bakas,et al.  ANHIR: Automatic Non-Rigid Histological Image Registration Challenge , 2020, IEEE Transactions on Medical Imaging.