Semantic Hierarchy Guided Registration Networks for Intra-subject Pulmonary CT Image Alignment

CT scanning has been widely used for diagnosis, staging and follow-up studies of pulmonary nodules, where image registration plays an essential role in follow-up assessment of CT images. However, it is challenging to align subtle structures in the lung CTs often with large deformation. Unsupervised learning-based registration methods, optimized according to the image similarity metrics, become popular in recent years due to their efficiency and robustness. In this work, we consider segmented tissues, i.e., airways, lobules, and pulmonary vessel structures, in a hierarchical way and propose a multi-stage registration workflow to predict deformation fields. The proposed workflow consists of two registration networks. The first network is the label alignment network, used to align the given segmentations. The second network is the vessel alignment network, used to further predict deformation fields to register vessels in lungs. By combining these two networks, we can register lung CT images not only in the semantic level but also in the texture level. In experiments, we evaluated the proposed algorithm on lung CT images for clinical follow-ups. The results indicate that our method has better performance especially in aligning critical structures such as airways and vessel branches in the lung, compared to the existing methods.

[1]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[2]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[4]  Nikos Paragios,et al.  Linear and Deformable Image Registration with 3D Convolutional Neural Networks , 2018, RAMBO+BIA+TIA@MICCAI.

[5]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[6]  Josien P. W. Pluim,et al.  Pulmonary CT Registration Through Supervised Learning With Convolutional Neural Networks , 2019, IEEE Transactions on Medical Imaging.

[7]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[8]  A. Jemal,et al.  Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries , 2018, CA: a cancer journal for clinicians.

[9]  C. Gatsonis,et al.  Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening , 2012 .

[10]  Arno Klein,et al.  A reproducible evaluation of ANTs similarity metric performance in brain image registration , 2011, NeuroImage.

[11]  Mattias P. Heinrich,et al.  Learning Deformable Point Set Registration with Regularized Dynamic Graph CNNs for Large Lung Motion in COPD Patients , 2019, GLMI@MICCAI.

[12]  Guido Gerig,et al.  Three-dimensional multi-scale line filter for segmentation and visualization of curvilinear structures in medical images , 1998, Medical Image Anal..