Constraining Volume Change in Learned Image Registration for Lung CTs

Deep-learning-based registration methods emerged as a fast alternative to conventional registration methods. However, these methods often still cannot achieve the same performance as conventional registration methods, because they are either limited to small deformation or they fail to handle a superposition of large and small deformations without producing implausible deformation fields with foldings inside. In this paper, we identify important strategies of conventional registration methods for lung registration and successfully developed the deep-learning counterpart. We employ a Gaussian-pyramid-based multilevel framework that can solve the image registration optimization in a coarse-to-fine fashion. Furthermore, we prevent foldings of the deformation field and restrict the determinant of the Jacobian to physiologically meaningful values by combining a volume change penalty with a curvature regularizer in the loss function. Keypoint correspondences are integrated to focus on the alignment of smaller structures. We perform an extensive evaluation to assess the accuracy, the robustness, the plausibility of the estimated deformation fields, and the transferability of our registration approach. We show that it archives state-of-the-art results on the COPDGene dataset compared to the challenge winning conventional registration method with much shorter execution time.

[1]  Eldad Haber,et al.  Intensity Gradient Based Registration and Fusion of Multi-modal Images , 2006, MICCAI.

[2]  Yong Fan,et al.  Non-rigid image registration using self-supervised fully convolutional networks without training data , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[3]  Heinz Handels,et al.  Automatic Landmark Detection and Non-linear Landmark-and Surface-based Registration of Lung CT Images , 2010 .

[4]  Jan Modersitzki,et al.  FAIR: Flexible Algorithms for Image Registration , 2009 .

[5]  Yang Lei,et al.  LungRegNet: an unsupervised deformable image registration method for 4D-CT lung. , 2020, Medical physics.

[6]  René Werner,et al.  GDL-FIRE ^\text 4D : Deep Learning-Based Fast 4D CT Image Registration , 2018, MICCAI.

[7]  Brian B. Avants,et al.  Evaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge , 2011, IEEE Transactions on Medical Imaging.

[8]  Michael Brady,et al.  MRF-Based Deformable Registration and Ventilation Estimation of Lung CT , 2013, IEEE Transactions on Medical Imaging.

[9]  Mitko Veta,et al.  Deformable image registration using convolutional neural networks , 2018, Medical Imaging.

[10]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[12]  Ruzena Bajcsy,et al.  Multiresolution elastic matching , 1989, Comput. Vis. Graph. Image Process..

[13]  Stefan Heldmann,et al.  Memory-efficient 2.5D convolutional transformer networks for multi-modal deformable registration with weak label supervision applied to whole-heart CT and MRI scans , 2019, International Journal of Computer Assisted Radiology and Surgery.

[14]  R. Castillo,et al.  A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets , 2009, Physics in medicine and biology.

[15]  E. Regan,et al.  Genetic Epidemiology of COPD (COPDGene) Study Design , 2011, COPD.

[16]  Martin Styner,et al.  Fast predictive multimodal image registration , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[17]  Hervé Delingette,et al.  Learning a Probabilistic Model for Diffeomorphic Registration , 2018, IEEE Transactions on Medical Imaging.

[18]  Thomas Guerrero,et al.  A reference dataset for deformable image registration spatial accuracy evaluation using the COPDgene study archive , 2013, Physics in medicine and biology.

[19]  Max A. Viergever,et al.  A deep learning framework for unsupervised affine and deformable image registration , 2018, Medical Image Anal..

[20]  Max A. Viergever,et al.  Registration of organs with sliding interfaces and changing topologies , 2014, Medical Imaging.

[21]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[22]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[23]  Mattias P. Heinrich,et al.  Closing the Gap between Deep and Conventional Image Registration using Probabilistic Dense Displacement Networks , 2019, MICCAI.

[24]  Josien P. W. Pluim,et al.  Pulmonary CT Registration Through Supervised Learning With Convolutional Neural Networks , 2019, IEEE Transactions on Medical Imaging.

[25]  Stefan Heldmann,et al.  Deep-learning-based CT-CBCT image registration for adaptive radio therapy , 2020, Medical Imaging: Image Processing.

[26]  Ben Glocker,et al.  On the Adaptability of Unsupervised CNN-Based Deformable Image Registration to Unseen Image Domains , 2018, MLMI@MICCAI.

[27]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[28]  Mattias P. Heinrich,et al.  Tackling the Problem of Large Deformations in Deep Learning Based Medical Image Registration Using Displacement Embeddings , 2020, ArXiv.

[29]  Max A. Viergever,et al.  End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network , 2017, DLMIA/ML-CDS@MICCAI.

[30]  Stefan Heldmann,et al.  Unsupervised learning for large motion thoracic CT follow-up registration , 2019, Image Processing.

[31]  Josien P W Pluim,et al.  Progressively Trained Convolutional Neural Networks for Deformable Image Registration , 2019, IEEE Transactions on Medical Imaging.

[32]  Stefan Heldmann,et al.  mlVIRNET: Multilevel Variational Image Registration Network , 2019, MICCAI.

[33]  Stefan Heldmann,et al.  Enhancing Label-Driven Deep Deformable Image Registration with Local Distance Metrics for State-of-the-Art Cardiac Motion Tracking , 2018, Bildverarbeitung für die Medizin.

[34]  Haiying Liu,et al.  A Generic Framework for Non-rigid Registration Based on Non-uniform Multi-level Free-Form Deformations , 2001, MICCAI.

[35]  Jan Modersitzki,et al.  Numerical Methods for Image Registration , 2004 .

[36]  Stefan Heldmann,et al.  Estimation of Large Motion in Lung CT by Integrating Regularized Keypoint Correspondences into Dense Deformable Registration , 2017, IEEE Transactions on Medical Imaging.

[37]  Sébastien Ourselin,et al.  Weakly-supervised convolutional neural networks for multimodal image registration , 2018, Medical Image Anal..

[38]  Michael Brady,et al.  Globally Optimal Deformable Registration on a Minimum Spanning Tree Using Dense Displacement Sampling , 2012, MICCAI.

[39]  Heinz Handels,et al.  Landmark-driven parameter optimization for non-linear image registration , 2011, Medical Imaging.

[40]  Jan Rühaak,et al.  Combining Automatic Landmark Detection and Variational Methods for Lung CT Registration , 2013 .

[41]  V.R.S Mani,et al.  Survey of Medical Image Registration , 2013 .

[42]  Albert C. S. Chung,et al.  Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks , 2020, MICCAI.

[43]  Marc Modat,et al.  Label-driven weakly-supervised learning for multimodal deformarle image registration , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[44]  Thomas Lange,et al.  Landmark Constrained Non-parametric Image Registration with Isotropic Tolerances , 2009, Bildverarbeitung für die Medizin.

[45]  Maxime Sermesant,et al.  SVF-Net: Learning Deformable Image Registration Using Shape Matching , 2017, MICCAI.

[46]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[47]  Jan Rühaak,et al.  A matrix-free approach to parallel and memory-efficient deformable image registration , 2018, SIAM J. Sci. Comput..

[48]  Fang-Fang Yin,et al.  A multi-scale framework with unsupervised joint training of convolutional neural networks for pulmonary deformable image registration , 2019, Physics in medicine and biology.

[49]  Hervé Delingette,et al.  Robust Non-rigid Registration Through Agent-Based Action Learning , 2017, MICCAI.

[50]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[51]  Eldad Haber,et al.  Cofir: Coarse and Fine Image Registration , 2004 .

[52]  Jan Modersitzki,et al.  Curvature Based Image Registration , 2004, Journal of Mathematical Imaging and Vision.

[53]  Jose Dolz,et al.  Bounding boxes for weakly supervised segmentation: Global constraints get close to full supervision , 2020, MIDL.

[54]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.

[55]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[56]  Jan Modersitzki,et al.  Combination of automatic non-rigid and landmark based registration: the best of both worlds , 2003, SPIE Medical Imaging.