Efficient organ localization using multi-label convolutional neural networks in thorax-abdomen CT scans

Automatic localization of organs and other structures in medical images is an important preprocessing step that can improve and speed up other algorithms such as organ segmentation, lesion detection, and registration. This work presents an efficient method for simultaneous localization of multiple structures in 3D thorax-abdomen CT scans. Our approach predicts the location of multiple structures using a single multi-label convolutional neural network for each orthogonal view. Each network takes extra slices around the current slice as input to provide extra context. A sigmoid layer is used to perform multi-label classification. The output of the three networks is subsequently combined to compute a 3D bounding box for each structure. We used our approach to locate 11 structures of interest. The neural network was trained and evaluated on a large set of 1884 thorax-abdomen CT scans from patients undergoing oncological workup. Reference bounding boxes were annotated by human observers. The performance of our method was evaluated by computing the wall distance to the reference bounding boxes. The bounding boxes annotated by the first human observer were used as the reference standard for the test set. Using the best configuration, we obtained an average wall distance of [Formula: see text] mm in the test set. The second human observer achieved [Formula: see text] mm. For all structures, the results were better than those reported in previously published studies. In conclusion, we proposed an efficient method for the accurate localization of multiple organs. Our method uses multiple slices as input to provide more context around the slice under analysis, and we have shown that this improves performance. This method can easily be adapted to handle more organs.

[1]  Antonio Criminisi,et al.  Regression forests for efficient anatomy detection and localization in computed tomography scans , 2013, Medical Image Anal..

[2]  Antonio Criminisi,et al.  Decision Forests with Long-Range Spatial Context for Organ Localization in CT Volumes , 2009 .

[3]  Horst Bischof,et al.  Global localization of 3D anatomical structures by pre-filtered Hough Forests and discrete optimization , 2013, Medical Image Anal..

[4]  Song Wang,et al.  Automatic localization of solid organs on 3D CT images by a collaborative majority voting decision based on ensemble learning , 2012, Comput. Medical Imaging Graph..

[5]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[6]  Laurent D. Cohen,et al.  Automatic Detection and Segmentation of Kidneys in 3D CT Images Using Random Forests , 2012, MICCAI.

[7]  Antonio Criminisi,et al.  Fast Multiple Organ Detection and Localization in Whole-Body MR Dixon Sequences , 2011, MICCAI.

[8]  Razvan Pascanu,et al.  Theano: new features and speed improvements , 2012, ArXiv.

[9]  Bram van Ginneken,et al.  Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images , 2016, IEEE Transactions on Medical Imaging.

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Dorin Comaniciu,et al.  Four-chamber heart modeling and automatic segmentation for 3D cardiac CT volumes , 2008, SPIE Medical Imaging.

[12]  Céline Fouard,et al.  Light Random Regression Forests for automatic multi-organ localization in CT images , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[13]  Yiqiang Zhan,et al.  Active Scheduling of Organ Detection and Segmentation in Whole-Body Medical Images , 2008, MICCAI.

[14]  Xiangrong Zhou,et al.  Automatic organ segmentation on torso CT images by using content-based image retrieval , 2012, Medical Imaging.

[15]  Shu Liao,et al.  Bodypart Recognition Using Multi-stage Deep Learning , 2015, IPMI.

[16]  Isabelle Bloch,et al.  Multi-organ localization with cascaded global-to-local regression and shape prior , 2015, Medical Image Anal..

[17]  Antonio Criminisi,et al.  Regression Forests for Efficient Anatomy Detection and Localization in CT Studies , 2010, MCV.

[18]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[19]  Xiangrong Zhou,et al.  A universal approach for automatic organ segmentations on 3D CT images based on organ localization and 3D GrabCut , 2014, Medical Imaging.

[20]  Xiangrong Zhou,et al.  Automatic anatomy partitioning of the torso region on CT images by using multiple organ localizations with a group-wise calibration technique , 2015, Medical Imaging.

[21]  B. van Ginneken,et al.  Automatic lung segmentation from thoracic computed tomography scans using a hybrid approach with error detection. , 2009, Medical physics.

[22]  Ghassan Hamarneh,et al.  Segmentation-Free Kidney Localization and Volume Estimation Using Aggregated Orthogonal Decision CNNs , 2017, MICCAI.

[23]  Domonkos Tikk,et al.  Organ detection in medical images with discriminately trained deformable part model , 2013, 2013 IEEE 9th International Conference on Computational Cybernetics (ICCC).

[24]  Amanda Clare,et al.  Knowledge Discovery in Multi-label Phenotype Data , 2001, PKDD.

[25]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Bram van Ginneken,et al.  Organ detection in thorax abdomen CT using multi-label convolutional neural networks , 2017, Medical Imaging.

[27]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Simon J. Doran,et al.  Stacked Autoencoders for Unsupervised Feature Learning and Multiple Organ Detection in a Pilot Study Using 4D Patient Data , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[30]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[31]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[32]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[33]  Max A. Viergever,et al.  ConvNet-Based Localization of Anatomical Structures in 3-D Medical Images , 2017, IEEE Transactions on Medical Imaging.

[34]  Dorin Comaniciu,et al.  Four-Chamber Heart Modeling and Automatic Segmentation for 3-D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features , 2008, IEEE Transactions on Medical Imaging.

[35]  Ronald M. Summers,et al.  Anatomy-specific classification of medical images using deep convolutional nets , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[36]  Max A. Viergever,et al.  2D image classification for 3D anatomy localization: employing deep convolutional neural networks , 2016, SPIE Medical Imaging.