3D Convolutional Neural Networks Image Registration Based on Efficient Supervised Learning from Artificial Deformations

We propose a supervised nonrigid image registration method, trained using artificial displacement vector fields (DVF), for which we propose and compare three network architectures. The artificial DVFs allow training in a fully supervised and voxel-wise dense manner, but without the cost usually associated with the creation of densely labeled data. We propose a scheme to artificially generate DVFs, and for chest CT registration augment these with simulated respiratory motion. The proposed architectures are embedded in a multi-stage approach, to increase the capture range of the proposed networks in order to more accurately predict larger displacements. The proposed method, RegNet, is evaluated on multiple databases of chest CT scans and achieved a target registration error of 2.32 $\pm$ 5.33 mm and 1.86 $\pm$ 2.12 mm on SPREAD and DIR-Lab-4DCT studies, respectively. The average inference time of RegNet with two stages is about 2.2 s.

[1]  Hervé Delingette,et al.  Robust Non-rigid Registration Through Agent-Based Action Learning , 2017, MICCAI.

[2]  Heinz Handels,et al.  Training CNNs for Image Registration from Few Samples with Model-based Data Augmentation , 2017, MICCAI.

[3]  Nikos Komodakis,et al.  A Deep Metric for Multimodal Registration , 2016, MICCAI.

[4]  Dinggang Shen,et al.  Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning Based Registration , 2018, MICCAI.

[5]  Marc L. Kessler,et al.  A Stochastic Approach to Estimate the UncertaintyInvolved in B-Spline Image Registration , 2009, IEEE Transactions on Medical Imaging.

[6]  Mitko Veta,et al.  Deformable image registration using convolutional neural networks , 2018, Medical Imaging.

[7]  Ben Glocker,et al.  On the Adaptability of Unsupervised CNN-Based Deformable Image Registration to Unseen Image Domains , 2018, MLMI@MICCAI.

[8]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[9]  Ben Glocker,et al.  Accuracy Estimation for Medical Image Registration Using Regression Forests , 2016, MICCAI.

[10]  Arthur W. Toga,et al.  Learning based coarse-to-fine image registration , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Max A. Viergever,et al.  A deep learning framework for unsupervised affine and deformable image registration , 2018, Medical Image Anal..

[12]  Johan H C Reiber,et al.  Progression parameters for emphysema: a clinical investigation. , 2007, Respiratory medicine.

[13]  Josien P. W. Pluim,et al.  Supervised local error estimation for nonlinear image registration using convolutional neural networks , 2017, Medical Imaging.

[14]  Josien P. W. Pluim,et al.  Supervised quality assessment of medical image registration: Application to intra-patient CT lung registration , 2012, Medical Image Anal..

[15]  Daniel Rueckert,et al.  Nonrigid registration using free-form deformations: application to breast MR images , 1999, IEEE Transactions on Medical Imaging.

[16]  Frank Sauer,et al.  Learning Based Non-rigid Multi-modal Image Registration Using Kullback-Leibler Divergence , 2005, MICCAI.

[17]  R. Castillo,et al.  A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets , 2009, Physics in medicine and biology.

[18]  Qian Wang,et al.  Deformable Image Registration Based on Similarity-Steered CNN Regression , 2017, MICCAI.

[19]  Thomas Guerrero,et al.  A reference dataset for deformable image registration spatial accuracy evaluation using the COPDgene study archive , 2013, Physics in medicine and biology.

[20]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[21]  Yaozong Gao,et al.  Learning‐based deformable image registration for infant MR images in the first year of life , 2017, Medical physics.

[22]  Maxime Sermesant,et al.  SVF-Net: Learning Deformable Image Registration Using Shape Matching , 2017, MICCAI.

[23]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Jiangping Wang,et al.  Multimodal Image Registration with Deep Context Reinforcement Learning , 2017, MICCAI.

[25]  Dean C. Barratt,et al.  Adversarial Deformation Regularization for Training Image Registration Neural Networks , 2018, MICCAI.

[26]  Mert R. Sabuncu,et al.  Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration , 2018, MICCAI.

[27]  Max A. Viergever,et al.  Semi-automatic construction of reference standards for evaluation of image registration , 2011, Medical Image Anal..

[28]  Z. Jane Wang,et al.  A CNN Regression Approach for Real-Time 2D/3D Registration , 2016, IEEE Transactions on Medical Imaging.

[29]  B C Stoel,et al.  Towards local progression estimation of pulmonary emphysema using CT. , 2014, Medical physics.

[30]  Xiao Yang,et al.  Fast Predictive Image Registration , 2016, LABELS/DLMIA@MICCAI.

[31]  Dwarikanath Mahapatra,et al.  Joint Registration And Segmentation Of Xray Images Using Generative Adversarial Networks , 2018, MLMI@MICCAI.

[32]  Max A. Viergever,et al.  End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network , 2017, DLMIA/ML-CDS@MICCAI.

[33]  Marius Staring,et al.  Adversarial optimization for joint registration and segmentation in prostate CT radiotherapy , 2019, MICCAI.

[34]  Marc Modat,et al.  Label-driven weakly-supervised learning for multimodal deformarle image registration , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[35]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..