ConvNet-Based Localization of Anatomical Structures in 3-D Medical Images

Localization of anatomical structures is a prerequisite for many tasks in a medical image analysis. We propose a method for automatic localization of one or more anatomical structures in 3-D medical images through detection of their presence in 2-D image slices using a convolutional neural network (ConvNet). A single ConvNet is trained to detect the presence of the anatomical structure of interest in axial, coronal, and sagittal slices extracted from a 3-D image. To allow the ConvNet to analyze slices of different sizes, spatial pyramid pooling is applied. After detection, 3-D bounding boxes are created by combining the output of the ConvNet in all slices. In the experiments, 200 chest CT, 100 cardiac CT angiography (CTA), and 100 abdomen CT scans were used. The heart, ascending aorta, aortic arch, and descending aorta were localized in chest CT scans, the left cardiac ventricle in cardiac CTA scans, and the liver in abdomen CT scans. Localization was evaluated using the distances between automatically and manually defined reference bounding box centroids and walls. The best results were achieved in the localization of structures with clearly defined boundaries (e.g., aortic arch) and the worst when the structure boundary was not clearly visible (e.g., liver). The method was more robust and accurate in localization multiple structures.

[1]  Wiro J. Niessen,et al.  Hippocampus segmentation in MR images using atlas registration, voxel classification, and graph cuts , 2008, NeuroImage.

[2]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Antonio Criminisi,et al.  Regression forests for efficient anatomy detection and localization in computed tomography scans , 2013, Medical Image Anal..

[4]  Max A. Viergever,et al.  2D image classification for 3D anatomy localization: employing deep convolutional neural networks , 2016, SPIE Medical Imaging.

[5]  Colin Raffel,et al.  Lasagne: First release. , 2015 .

[6]  Isabelle Bloch,et al.  Multi-organ localization with cascaded global-to-local regression and shape prior , 2015, Medical Image Anal..

[7]  Clément Farabet,et al.  Torch7: A Matlab-like Environment for Machine Learning , 2011, NIPS 2011.

[8]  Steven Salzberg,et al.  On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach , 1997, Data Mining and Knowledge Discovery.

[9]  Dorin Comaniciu,et al.  Marginal Space Learning for Efficient Detection of 2D/3D Anatomical Structures in Medical Images , 2009, IPMI.

[10]  D. Lynch,et al.  The National Lung Screening Trial: overview and study design. , 2011, Radiology.

[11]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[12]  John Salvatier,et al.  Theano: A Python framework for fast computation of mathematical expressions , 2016, ArXiv.

[13]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[14]  Antonio Criminisi,et al.  TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context , 2007, International Journal of Computer Vision.

[15]  John S. Bridle,et al.  Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters , 1989, NIPS.

[16]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[19]  Dorin Comaniciu,et al.  Marginal Space Deep Learning: Efficient Architecture for Volumetric Image Parsing , 2016, IEEE Transactions on Medical Imaging.

[20]  Michael Kohnen,et al.  Quality of DICOM header information for image categorization , 2002, SPIE Medical Imaging.

[21]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[22]  Max A. Viergever,et al.  Automatic segmentation of the left ventricle in cardiac CT angiography using convolutional neural networks , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[23]  Song Wang,et al.  Automatic localization of solid organs on 3D CT images by a collaborative majority voting decision based on ensemble learning , 2012, Comput. Medical Imaging Graph..

[24]  Torsten Rohlfing,et al.  Evaluation of atlas selection strategies for atlas-based image segmentation with application to confocal microscopy images of bee brains , 2004, NeuroImage.

[25]  T van Walsum,et al.  Evaluation of a multi-atlas based method for segmentation of cardiac CTA data: a large-scale, multicenter, and multivendor study. , 2010, Medical physics.

[26]  Dorin Comaniciu,et al.  Fast Automatic Heart Chamber Segmentation from 3D CT Data Using Marginal Space Learning and Steerable Features , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[27]  Max A. Viergever,et al.  Deep convolutional neural networks for automatic coronary calcium scoring in a screening study with low-dose chest CT , 2016, SPIE Medical Imaging.

[28]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[29]  Sohan R. Ranjan.  Organ localization through anatomy-aware non-rigid registration with atlas , 2011, 2011 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[30]  Marius George Linguraru,et al.  Abdominal multi-organ segmentation from CT images using conditional shape-location and unsupervised intensity priors , 2015, Medical Image Anal..

[31]  Ronald M. Summers,et al.  Anatomy-specific classification of medical images using deep convolutional nets , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[32]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[33]  Max A. Viergever,et al.  Automatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks , 2016, Medical Image Anal..

[34]  Daguang Xu,et al.  Robust 3D Organ Localization with Dual Learning Architectures and Fusion , 2016, LABELS/DLMIA@MICCAI.

[35]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Dorin Comaniciu,et al.  Hierarchical parsing and semantic navigation of full body CT data , 2009, Medical Imaging.

[37]  Max A. Viergever,et al.  Multi-Atlas-Based Segmentation With Local Decision Fusion—Application to Cardiac and Aortic Segmentation in CT Scans , 2009, IEEE Transactions on Medical Imaging.