Target organ non-rigid registration on abdominal CT images via deep-learning based detection

Abstract Abdominal target organ registration is essentially important for medical diagnosis and clinical treatment, and the main challenge comes from the complex non-rigid deformations of organ structure and volume. In this paper, an improved deep convolutional neural network (DCNN) is first proposed to automatically detect the abdominal computed tomography (CT) target organ regions of interest (ROIs), in which a CoordConv layer is added to obtain more coordinate information for different target organs, and a transfer learning technique is used for pre-training on a non-medical database to deal with the medical training data scarce situation. Then, the pair-wise target organ ROIs are registered by an intensity-based dissimilarity measure combined with a standard Tikhonov regularization. Finally, the proposed method is evaluated on a self-created clinical database and several public databases. The experimental results demonstrate that our method achieves high accuracy on the target organ ROIs detection with the mean intersection over union (mIOU) and mean average precision (mAP) values of 0.8719 and 0.9445, respectively, and the registration performance on these ROIs is superior to that of some existing methods. Moreover, the proposed method is capable of detecting and registering abdominal target organ ROIs on a single GPU with good network convergence and less time consumption.

[1]  Sarada Musala,et al.  A novel optimized rigid image registration of brain using ACMI , 2010, 2010 IEEE International Conference on Computational Intelligence and Computing Research.

[2]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[3]  Simon K. Warfield,et al.  Abdominal Images Non-rigid Registration Using Local-Affine Diffeomorphic Demons , 2011, Abdominal Imaging.

[4]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Vipin Chaudhary,et al.  Intervertebral disc detection in X-ray images using faster R-CNN , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[6]  Leo Grady,et al.  Spectral Log-Demons: Diffeomorphic Image Registration with Very Large Deformations , 2013, International Journal of Computer Vision.

[7]  Fabrice Heitz,et al.  Hierarchical multi-scale supervoxel matching using random forests for automatic semi-dense abdominal image registration , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[8]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Hore,et al.  [IEEE 2010 20th International Conference on Pattern Recognition (ICPR) - Istanbul, Turkey (2010.08.23-2010.08.26)] 2010 20th International Conference on Pattern Recognition - Image Quality Metrics: PSNR vs. SSIM , 2010 .

[10]  Jan-Jakob Sonke,et al.  First clinical experience with a multiple region of interest registration and correction method in radiotherapy of head-and-neck cancer patients. , 2010, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[11]  Nikos Paragios,et al.  Slice-to-volume medical image registration: a survey , 2017, Medical Image Anal..

[12]  Lubomir M. Hadjiiski,et al.  Mass detection in digital breast tomosynthesis: Deep convolutional neural network with transfer learning from mammography. , 2016, Medical physics.

[13]  Edward Y. Chang,et al.  Transfer representation learning for medical image analysis , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[14]  Won-Jin Yi,et al.  ROI-based image registration for digital subtraction radiography. , 2006, Oral surgery, oral medicine, oral pathology, oral radiology, and endodontics.

[15]  Jason Yosinski,et al.  An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution , 2018, NeurIPS.

[16]  B Denis de Senneville,et al.  Accelerating multi-modal image registration using a supervoxel-based variational framework , 2018, Physics in medicine and biology.

[17]  Benjamin Berkels,et al.  Mesh-to-raster region-of-interest-based nonrigid registration of multimodal images , 2017, Journal of medical imaging.

[18]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[19]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[20]  Mikhail Karchevskiy,et al.  Automatic salt deposits segmentation: A deep learning approach , 2018, ArXiv.

[21]  Deshan Yang,et al.  An adaptive motion regularization technique to support sliding motion in deformable image registration , 2018, Medical physics.

[22]  Hui Li,et al.  A New Multiresolution Medical Image Registration Algorithm Based on Intensity and Edge Information , 2008, 2008 Fourth International Conference on Natural Computation.

[23]  Jose George,et al.  Using YOLO based deep learning network for real time detection and localization of lung nodules from low dose CT scans , 2018, Medical Imaging.

[24]  Mario Ceresa,et al.  Fully automatic detection and segmentation of abdominal aortic thrombus in post‐operative CTA images using Deep Convolutional Neural Networks , 2018, Medical Image Anal..

[25]  Alan L. Yuille,et al.  Recurrent Saliency Transformation Network: Incorporating Multi-stage Visual Cues for Small Organ Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[28]  Hongyang Chao,et al.  Face Recognition from Sequential Sparse 3D Data via Deep Registration , 2018, 2019 International Conference on Biometrics (ICB).

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Taesu Kim,et al.  Robust and Fine-grained Prosody Control of End-to-end Speech Synthesis , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  Suryadiputra Liawatimena,et al.  Lung nodule detection and classification from Thorax CT-scan using RetinaNet with transfer learning , 2020, Journal of King Saud University - Computer and Information Sciences.

[32]  Andre Esteva,et al.  A guide to deep learning in healthcare , 2019, Nature Medicine.

[33]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Hemant D. Tagare,et al.  Symmetric Non-rigid Registration: A Geometric Theory and Some Numerical Techniques , 2009, Journal of Mathematical Imaging and Vision.

[35]  K. V. Arya,et al.  An efficient low-dose CT reconstruction technique using partial derivatives based guided image filter , 2018, Multimedia Tools and Applications.

[36]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[37]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[38]  Denis Friboulet,et al.  Cycle GAN-Based Data Augmentation For Multi-Organ Detection In CT Images Via Yolo , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[39]  Bruce Fischl,et al.  Mid-space-independent deformable image registration , 2017, NeuroImage.

[40]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[41]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[42]  Isabelle Bloch,et al.  Explicit Incorporation of Prior Anatomical Information Into a Nonrigid Registration of Thoracic and Abdominal CT and 18-FDG Whole-Body Emission PET Images , 2007, IEEE Transactions on Medical Imaging.

[43]  Lothar Reichel,et al.  On the choice of subspace for large-scale Tikhonov regularization problems in general form , 2018, Numerical Algorithms.

[44]  Cristian A. Linte,et al.  A robust automated left ventricle region of interest localization technique using a cardiac cine MRI atlas , 2016, SPIE Medical Imaging.

[45]  Jie Zhao,et al.  Automated Segmentation Of Pulmonary Lobes Using Coordination-Guided Deep Neural Networks , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[46]  Yang Liu,et al.  Privacy-Preserving Object Detection for Medical Images With Faster R-CNN , 2019, IEEE Transactions on Information Forensics and Security.

[47]  Kavkirat Kaur,et al.  Computed tomography reconstruction on distributed storage using hybrid regularization approach , 2019, Modern Physics Letters B.