Error estimation of deformable image registration of pulmonary CT scans using convolutional neural networks

Abstract. Error estimation in nonlinear medical image registration is a nontrivial problem that is important for validation of registration methods. We propose a supervised method for estimation of registration errors in nonlinear registration of three-dimensional (3-D) images. The method is based on a 3-D convolutional neural network that learns to estimate registration errors from a pair of image patches. By applying the network to patches centered around every voxel, we construct registration error maps. The network is trained using a set of representative images that have been synthetically transformed to construct a set of image pairs with known deformations. The method is evaluated on deformable registrations of inhale–exhale pairs of thoracic CT scans. Using ground truth target registration errors on manually annotated landmarks, we evaluate the method’s ability to estimate local registration errors. Estimation of full domain error maps is evaluated using a gold standard approach. The two evaluation approaches show that we can train the network to robustly estimate registration errors in a predetermined range, with subvoxel accuracy. We achieved a root-mean-square deviation of 0.51 mm from gold standard registration errors and of 0.66 mm from ground truth landmark registration errors.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  Dinggang Shen,et al.  Scalable High-Performance Image Registration Framework by Unsupervised Deep Feature Representations Learning , 2016, IEEE Transactions on Biomedical Engineering.

[3]  Brian B. Avants,et al.  Evaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge , 2011, IEEE Transactions on Medical Imaging.

[4]  Michael Brady,et al.  MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration , 2012, Medical Image Anal..

[5]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[6]  Torsten Rohlfing,et al.  Image Similarity and Tissue Overlaps as Surrogates for Image Registration Accuracy: Widely Used but Unreliable , 2012, IEEE Transactions on Medical Imaging.

[7]  V. Boldea,et al.  Simulation of four-dimensional CT images from deformable registration between inhale and exhale breath-hold CT scans. , 2006, Medical physics.

[8]  R. Castillo,et al.  Four-dimensional deformable image registration using trajectory modeling , 2010, Physics in medicine and biology.

[9]  R. Castillo,et al.  A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets , 2009, Physics in medicine and biology.

[10]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[11]  Z. Jane Wang,et al.  A CNN Regression Approach for Real-Time 2D/3D Registration , 2016, IEEE Transactions on Medical Imaging.

[12]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[13]  Hervé Delingette,et al.  Robust Non-rigid Registration Through Agent-Based Action Learning , 2017, MICCAI.

[14]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[15]  Colin Raffel,et al.  Lasagne: First release. , 2015 .

[16]  William M. Wells,et al.  Bayesian characterization of uncertainty in intra-subject non-rigid registration , 2013, Medical Image Anal..

[17]  Ben Glocker,et al.  Accuracy Estimation for Medical Image Registration Using Regression Forests , 2016, MICCAI.

[18]  Josien P. W. Pluim,et al.  Supervised local error estimation for nonlinear image registration using convolutional neural networks , 2017, Medical Imaging.

[19]  Josien P. W. Pluim,et al.  Supervised quality assessment of medical image registration: Application to intra-patient CT lung registration , 2012, Medical Image Anal..

[20]  Sascha E. A. Muenzing.  Learning-Based Approaches to Deformable Image Registration , 2014 .

[21]  Nikos Komodakis,et al.  A Deep Metric for Multimodal Registration , 2016, MICCAI.

[22]  Max A. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[23]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Stefan Klein,et al.  Pulmonary Image Registration with elastix using a Standard Intensity-Based Algorithm , 2010 .

[25]  Max A. Viergever,et al.  DIRBoost-An algorithm for boosting deformable image registration: Application to lung CT intra-subject registration , 2014, Medical Image Anal..

[26]  Wei Shao,et al.  Population Shape Collapse in Large Deformation Registration of MR Brain Images , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[27]  Patrick Clarysse,et al.  Automated segmentation of a motion mask to preserve sliding motion in deformable registration of thoracic CT. , 2012, Medical physics.

[28]  Benoit M. Dawant,et al.  Validation of a Nonrigid Registration Error Detection Algorithm Using Clinical MRI Brain Data , 2015, IEEE Transactions on Medical Imaging.

[29]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[30]  Patrick Clarysse,et al.  Spatiotemporal motion estimation for respiratory-correlated imaging of the lungs. , 2010, Medical physics.

[31]  Thomas Guerrero,et al.  A reference dataset for deformable image registration spatial accuracy evaluation using the COPDgene study archive , 2013, Physics in medicine and biology.

[32]  Ipek Oguz,et al.  The Shape Collapse Problem in Image Registration , 2013 .

[33]  Nassir Navab,et al.  Learning Optimization Updates for Multimodal Registration , 2016, MICCAI.

[34]  Emile A. Hendriks,et al.  Confidence Estimation for Medical Image Registration Based On Stereo Confidences , 2016, IEEE Transactions on Medical Imaging.

[35]  Jan Kybic,et al.  Bootstrap Resampling for Image Registration Uncertainty Estimation Without Ground Truth , 2010, IEEE Transactions on Image Processing.

[36]  William M. Wells,et al.  Bayesian Characterization of Uncertainty in Multi-modal Image Registration , 2012, WBIR.

[37]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.