Multiple Organ Localization in CT Image Using Triple-Branch Fully Convolutional Networks

Organ localization is an essential prerequisite for many computed tomography (CT) image analysis tasks such as organ segmentation, lesion detection, and image registration. However, it is a challenging problem due to various causes such as the low-contrast organ boundaries, the large variations of organ appearance, the truncated organs, and the abnormalities. In this work, we propose an automatic method for multiple organ localization in CT image using a novel triple-branch fully convolutional network (FCN). First, the input CT image is extended to a three-channel image that contains both the original image and the enhanced organ structures using a density enhancement filter. Then, the extended three-channel image is fed to a 3D triple-branch FCN to parallelly predict the organs’ presence probability curves along axial, coronal and sagittal directions. Finally, the bounding boxes of the target organs are computed based on these presence probability curves. The triple-branch FCN is fully implemented in 3D manner, thus it can take full advantage of the spatial context information in CT image to perform accurate organ localization. Only several basic convolutional neural networks (ConvNet) operations (such as convolution, pooling and softmax regression) are involved in this triple-branch FCN, thus it is easy-implemented and flexible to serve as a preprocessing module in other medical image analysis tasks. By enhancing the structural information in the CT image using the density enhancement filter, the performance of the triple-branch FCN can be further improved. The proposed method is evaluated on a challenging public dataset with 201 clinical abdominal/torso CT images to locate 11 body organs. As the experimental results showed, the proposed method achieves higher localization accuracy than the current state-of-the-art methods.

[1]  Yinghuan Shi,et al.  Pelvic Organ Segmentation Using Distinctive Curve Guided Fully Convolutional Networks , 2019, IEEE Transactions on Medical Imaging.

[2]  Steve B. Jiang,et al.  Fully automated organ segmentation in male pelvic CT images , 2018, Physics in medicine and biology.

[3]  Antonio Criminisi,et al.  Decision Forests with Long-Range Spatial Context for Organ Localization in CT Volumes , 2009 .

[4]  Song Wang,et al.  Automatic localization of solid organs on 3D CT images by a collaborative majority voting decision based on ensemble learning , 2012, Comput. Medical Imaging Graph..

[5]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[7]  Céline Fouard,et al.  Light Random Regression Forests for automatic multi-organ localization in CT images , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[8]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[9]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[10]  Ghassan Hamarneh,et al.  Segmentation-Free Kidney Localization and Volume Estimation Using Aggregated Orthogonal Decision CNNs , 2017, MICCAI.

[11]  Siqi Li,et al.  Organ Location Determination and Contour Sparse Representation for Multiorgan Segmentation , 2018, IEEE Journal of Biomedical and Health Informatics.

[12]  Max A. Viergever,et al.  ConvNet-Based Localization of Anatomical Structures in 3-D Medical Images , 2017, IEEE Transactions on Medical Imaging.

[13]  Isabelle Bloch,et al.  Multi-organ localization with cascaded global-to-local regression and shape prior , 2015, Medical Image Anal..

[14]  Yaozong Gao,et al.  CT male pelvic organ segmentation using fully convolutional networks with boundary sensitive representation , 2019, Medical Image Anal..

[15]  Max A. Viergever,et al.  2D image classification for 3D anatomy localization: employing deep convolutional neural networks , 2016, SPIE Medical Imaging.

[16]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[17]  Antonio Criminisi,et al.  Regression forests for efficient anatomy detection and localization in computed tomography scans , 2013, Medical Image Anal..

[18]  Dorin Comaniciu,et al.  Four-Chamber Heart Modeling and Automatic Segmentation for 3-D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features , 2008, IEEE Transactions on Medical Imaging.

[19]  Sebastian Nowozin,et al.  On feature combination for multiclass object classification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Bram van Ginneken,et al.  Efficient organ localization using multi-label convolutional neural networks in thorax-abdomen CT scans , 2018, Physics in medicine and biology.

[22]  Yu Zhao,et al.  Knowledge-Aided Convolutional Neural Network for Small Organ Segmentation , 2019, IEEE Journal of Biomedical and Health Informatics.

[23]  Bram van Ginneken,et al.  Organ detection in thorax abdomen CT using multi-label convolutional neural networks , 2017, Medical Imaging.

[24]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  Mubarak Shah,et al.  Columbia-UCF TRECVID2010 Multimedia Event Detection: Combining Multiple Modalities, Contextual Concepts, and Temporal Matching , 2010, TRECVID.

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  Xiangzhi Bai,et al.  Efficient Multiple Organ Localization in CT Image Using 3D Region Proposal Network , 2019, IEEE Transactions on Medical Imaging.