Learning 3D non-rigid deformation based on an unsupervised deep learning for PET/CT image registration

This paper proposes a novel method to learn a 3D non-rigid deformation for automatic image registration between Positron Emission Tomography (PET) and Computed Tomography (CT) scans obtained from the same patient. There are two modules in the proposed scheme including (1) low-resolution displacement vector field (LR-DVF) estimator, which uses a 3D deep convolutional network (ConvNet) to directly estimate the voxel-wise displacement (a 3D vector field) between PET/CT images, and (2) 3D spatial transformer and re-sampler, which warps the PET images to match the anatomical structures in the CT images using the estimated 3D vector field. The parameters of the ConvNet are learned from a number of PET/CT image pairs via an unsupervised learning method. The Normalized Cross Correlation (NCC) between PET/CT images is used as the similarity metric to guide an end-to-end learning process with a constraint (regular term) to preserve the smoothness of the 3D deformations. A dataset with 170 PET/CT scans is used in experiments based on 10-fold cross-validation, where a total of 22,338 3D patches are sampled from the dataset. In each fold, 3D patches from 153 patients (90%) are used for training the parameters, while the remaining whole-body voxels from 17 patients (10%) are used for testing the performance of the image registration. The experimental results demonstrate that the image registration accuracy (the mean value of NCCs) is increased from 0.402 (the initial situation) to 0.567 on PET/CT scans using the proposed scheme. We also compare the performance of our scheme with previous work (DIRNet) and the advantage of our scheme is confirmed via the promising results.