D-net: Siamese Based Network for Arbitrarily Oriented Volume Alignment

Alignment of contrast and non contrast-enhanced imaging is essential for quantification of changes in several biomedical applications. In particular, the extraction of cartilage shape from contrast-enhanced Computed Tomography (CT) of tibiae requires accurate alignment of the bone, currently performed manually. Existing deep learning-based methods for alignment require a common template or are limited in rotation range. Therefore, we present a novel network, D-net, to estimate arbitrary rotation and translation between 3D CT scans that additionally does not require a prior template. D-net is an extension to the branched Siamese encoder-decoder structure connected by new mutual, non-local links, which efficiently capture long-range connections of similar features between two branches. The 3D supervised network is trained and validated using preclinical CT scans of mouse tibiae with and without contrast enhancement in cartilage. The presented results show a significant improvement in the estimation of CT alignment, outperforming the current comparable methods.

[1]  Yi Zhou,et al.  On the Continuity of Rotation Representations in Neural Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Marc Modat,et al.  Label-driven weakly-supervised learning for multimodal deformarle image registration , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[3]  Deniz Erdogmus,et al.  Real-Time Deep Pose Estimation With Geodesic Loss for Image-to-Template Rigid Registration , 2018, IEEE Transactions on Medical Imaging.

[4]  Boudewijn P. F. Lelieveldt,et al.  Automated Registration of Whole-Body Follow-Up MicroCT Data of Mice , 2011, MICCAI.

[5]  Minho Lee,et al.  Siamese U-Net with Healthy Template for Accurate Segmentation of Intracranial Hemorrhage , 2019, MICCAI.

[6]  Sheng Xu,et al.  Learning deep similarity metric for 3D MR–TRUS image registration , 2018, International Journal of Computer Assisted Radiology and Surgery.

[7]  Sotirios A. Tsaftaris,et al.  FIRE: Unsupervised bi-directional inter- and intra-modality registration using deep networks , 2019, 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS).

[8]  Mattias P. Heinrich,et al.  Advances and challenges in deformable image registration: From image fusion to complex motion modelling , 2016, Medical Image Anal..

[9]  Nikos Paragios,et al.  DRAMMS: Deformable Registration via Attribute Matching and Mutual-Saliency Weighting , 2009, IPMI.

[10]  Gustavo Carneiro,et al.  Siam-U-Net: encoder-decoder siamese network for knee cartilage tracking in ultrasound images , 2019, Medical Image Anal..

[11]  Jiangping Wang,et al.  Multimodal Image Registration with Deep Context Reinforcement Learning , 2017, MICCAI.

[12]  Pingkun Yan,et al.  Deep learning in medical image registration: a survey , 2020, Machine Vision and Applications.

[13]  Michael Brady,et al.  Non-local Graph-Based Regularization for Deformable Image Registration , 2016, MCV/BAMBI@MICCAI.

[14]  Dorin Comaniciu,et al.  An Artificial Agent for Robust Image Registration , 2016, AAAI.

[15]  J. Paul Siebert,et al.  Learning Rigid Image Registration - Utilizing Convolutional Neural Networks for Medical Image Registration , 2018, BIOIMAGING.

[16]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Max A. Viergever,et al.  A deep learning framework for unsupervised affine and deformable image registration , 2018, Medical Image Anal..