An Optimized Registration Method Based on Distribution Similarity and DVF Smoothness for 3D PET and CT Images

A fusion image combining both anatomical and functional information obtained by registering medical images of two different modalities, Positron Emission Tomography (PET) and Computed Tomography (CT), is of great significance for medical image analysis and diagnosis. Medical image registration relies on similarity measure which is low between PET/CT image voxels and therefore PET/CT registration is a challenging task. To address this issue, this paper presents an unsupervised end-to-end method, DenseRegNet, for deformable 3D PET/CT image registration. The method consists of two stages: (1) predicting 3D displacement vector field (DVF); and (2) registering 3D image. In the 3D DVF prediction stage, a two-level similarity measure together with a deformation regularization is proposed as loss function to optimize network training.In the image registration stage, a resampler and a spatial transformer are utilized to obtain the registration results. In this paper, 663 pairs of Uptake Value (SUV) and Hounsfield Unit (Hu) patches of 106 patients, 227 pairs of SUV and Hu patches of 35 patients and 259 pairs of SUV and Hu patches of 35 patients are randomly selected as training, validation and test set, respectively. Normalized cross correlation (NCC), intersection over union (IoU) of liver bounding box and euclidean distance (ED) on landmark points are used to evaluate the registration results. Experiment results show that the proposed method, DenseRegNet, achieves the best results in terms of liver bounding box IoU and ED, and the second highest value of NCC. For a trained model, given a new pair of PET/CT images, the registration result can be obtained with only one forward calculation within 10 seconds. Through qualitative and quantitative analyses, we demonstrate that, compared with other deep learning registration models, the proposed DenseRegNet achieves improved results in the challenging deformable PET/CT registration task.

[1]  Jürgen Weese,et al.  A comparison of similarity measures for use in 2-D-3-D medical image registration , 1998, IEEE Transactions on Medical Imaging.

[2]  Michael Unser,et al.  Fast parametric elastic image registration , 2003, IEEE Trans. Image Process..

[3]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[5]  Xiangrong Zhou,et al.  Learning 3D non-rigid deformation based on an unsupervised deep learning for PET/CT image registration , 2019, Medical Imaging.

[6]  Wojciech Zaremba,et al.  B-test: A Non-parametric, Low Variance Kernel Two-sample Test , 2013, NIPS.

[7]  Habib Zaidi,et al.  B-spline based free form deformation thoracic non-rigid registration of CT and PET images , 2011, International Conference on Graphic and Image Processing.

[8]  Huiyan Jiang,et al.  A Registration Method Based on Contour Point Cloud for 3D Whole-Body PET and CT Images , 2017, BioMed research international.

[9]  A Uneri,et al.  3D–2D image registration for target localization in spine surgery: investigation of similarity metrics providing robustness to content mismatch , 2016, Physics in medicine and biology.

[10]  Dorin Comaniciu,et al.  An Artificial Agent for Robust Image Registration , 2016, AAAI.

[11]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[12]  Yong Fan,et al.  Non-rigid image registration using fully convolutional networks with deep self-supervision , 2017, ArXiv.

[13]  David W Townsend,et al.  PET/CT today and tomorrow. , 2004, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[14]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[15]  Z. Jane Wang,et al.  Real-time 2D/3D registration via CNN regression , 2015, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[16]  Mattias P. Heinrich,et al.  Advances and challenges in deformable image registration: From image fusion to complex motion modelling , 2016, Medical Image Anal..

[17]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[18]  David R. Haynor,et al.  PET-CT image registration in the chest using free-form deformations , 2003, IEEE Transactions on Medical Imaging.

[19]  Albert C. S. Chung,et al.  A Global Optimization Strategy for 3D-2D Registration of Vascular Images , 2006, BMVC.

[20]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[21]  Torsten Rohlfing,et al.  Volume-preserving nonrigid registration of MR breast images using free-form deformation with an incompressibility constraint , 2003, IEEE Transactions on Medical Imaging.

[22]  C. Hwang.  Simulated annealing: Theory and applications , 1988, Acta Applicandae Mathematicae - An International Survey Journal on Applying Mathematics and Mathematical Applications.

[23]  Max A. Viergever,et al.  End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network , 2017, DLMIA/ML-CDS@MICCAI.

[24]  Qilong Wang,et al.  Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[26]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[27]  R. Shekhar,et al.  Automated 3-dimensional elastic registration of whole-body PET and CT from separate or combined scanners. , 2005, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.