Hybrid-feature-guided lung nodule type classification on CT images

Abstract In this paper, we propose a novel classification method for lung nodules from CT images based on hybrid features. Towards nodules of different types, including well-circumscribed, vascularized, juxta-pleural, pleural-tail, as well as ground glass optical (GGO) and non-nodule from CT scans, our method has achieved promising classification results. The proposed method utilizes hybrid descriptors consisting of statistical features from multi-view multi-scale convolutional neural networks (CNNs) and geometrical features from Fisher vector (FV) encodings based on scale-invariant feature transform (SIFT). First, we approximate the nodule radii based on icosahedron sampling and intensity analysis. Then, we apply high frequency content measure analysis to obtain sampling views with more abundant information. After that, based on re-sampled views, we train multi-view multi-scale CNNs to extract statistical features and calculate FV encodings as geometrical features. Finally, we achieve hybrid features by merging statistical and geometrical features based on multiple kernel learning (MKL) and classify nodule types through a multi-class support vector machine. The experiments on LIDC-IDRI and ELCAP have shown that our method has achieved promising results and can be of great assistance for radiologists’ diagnosis of lung cancer in clinical practice.

[1]  Adrian D. C. Chan,et al.  A Gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses , 2005, IEEE Transactions on Biomedical Engineering.

[2]  Anthony P. Reeves,et al.  Three-dimensional segmentation and growth-rate estimation of small pulmonary nodules in helical CT images , 2003, IEEE Transactions on Medical Imaging.

[3]  David Dagan Feng,et al.  Content-Based Medical Image Retrieval: A Survey of Applications to Multidimensional and Multimodality Data , 2013, Journal of Digital Imaging.

[4]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[5]  Dazhe Zhao,et al.  Accurate Lung Segmentation For X-ray CT Images , 2007, Third International Conference on Natural Computation (ICNC 2007).

[6]  Youngjin Yoo,et al.  Modeling the Variability in Brain Morphology and Lesion Distribution in Multiple Sclerosis by Deep Learning , 2014, MICCAI.

[7]  Mark D. Plumbley,et al.  Real-time temporal segmentation of note objects in music signals , 2004, ICMC.

[8]  Ulas Bagci,et al.  Near-optimal keypoint sampling for fast pathological lung segmentation , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[9]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[10]  Bai Ying Lei,et al.  Bridging Computational Features Toward Multiple Semantic Features with Multi-task Regression: A Study of CT Pulmonary Nodules , 2016, MICCAI.

[11]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[12]  David Dagan Feng,et al.  Overlapping node discovery for improving classification of lung nodules , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[13]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[14]  Aimin Hao,et al.  A CADe system for nodule detection in thoracic CT images based on artificial neural network , 2017, Science China Information Sciences.

[15]  Ahmet Ekin,et al.  Dementia diagnosis using similar and dissimilar retrieval items , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[16]  Dinggang Shen,et al.  Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis , 2014, MICCAI.

[17]  D. Shen,et al.  Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans , 2016, Scientific Reports.

[18]  Florent Perronnin,et al.  Fisher Kernels on Visual Vocabularies for Image Categorization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[20]  Jin Mo Goo,et al.  Quantitative Computed Tomography Imaging Biomarkers in the Diagnosis and Management of Lung Cancer , 2015, Investigative radiology.

[21]  A. Jemal,et al.  Cancer statistics, 2017 , 2017, CA: a cancer journal for clinicians.

[22]  Lauge Sørensen,et al.  Quantitative Analysis of Pulmonary Emphysema Using Local Binary Patterns , 2010, IEEE Transactions on Medical Imaging.

[23]  Bram van Ginneken,et al.  Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images , 2014, Medical Image Anal..

[24]  Yun Zhou,et al.  A ranking-based lung nodule image classification method using unlabeled image knowledge , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[25]  Thomas Mensink,et al.  Improving the Fisher Kernel for Large-Scale Image Classification , 2010, ECCV.

[26]  Huai Li,et al.  Artificial convolution neural network for medical image pattern recognition , 1995, Neural Networks.

[27]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[28]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[29]  Ronald M. Summers,et al.  Automatic Segmentation and Measurement of Pleural Effusions on CT , 2013, IEEE Transactions on Biomedical Engineering.

[30]  Anant Madabhushi,et al.  An integrated texton and bag of words classifier for identifying anaplastic medulloblastomas , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[31]  Aly A. Farag,et al.  Evaluation of geometric feature descriptors for detection and classification of lung nodules in low dose CT scans of the chest , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[32]  Yun Zhou,et al.  Context Curves for Classification of Lung Nodule Images , 2013, 2013 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[33]  Heng Huang,et al.  Lung Nodule Classification With Multilevel Patch-Based Context Analysis , 2014, IEEE Transactions on Biomedical Engineering.

[34]  Kim L. Boyer,et al.  Stratified learning of local anatomical context for lung nodules in CT images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Bram van Ginneken,et al.  Computer-Aided Detection of Ground Glass Nodules in Thoracic CT Images Using Shape, Intensity and Context Features , 2011, MICCAI.

[36]  Claus Bahlmann,et al.  Computer-aided gleason grading of prostate cancer histopathological images using texton forests , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[37]  David Dagan Feng,et al.  Location classification of lung nodules with optimized graph construction , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[38]  David Dagan Feng,et al.  A Multistage Discriminative Model for Tumor and Lymph Node Detection in Thoracic Images , 2012, IEEE Transactions on Medical Imaging.

[39]  B. van Ginneken,et al.  Automatic lung segmentation from thoracic computed tomography scans using a hybrid approach with error detection. , 2009, Medical physics.

[40]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[41]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[42]  Ronald M. Summers,et al.  A New 2.5D Representation for Lymph Node Detection Using Random Sets of Deep Convolutional Neural Network Observations , 2014, MICCAI.

[43]  Aly A. Farag,et al.  Toward Precise Pulmonary Nodule Descriptors for Nodule Type Classification , 2010, MICCAI.

[44]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[45]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[46]  V. Roggli,et al.  Solitary pulmonary nodules: Part I. Morphologic evaluation for differentiation of benign and malignant lesions. , 2000, Radiographics : a review publication of the Radiological Society of North America, Inc.

[47]  Bram van Ginneken,et al.  Bag-of-Frequencies: A Descriptor of Pulmonary Nodules in Computed Tomography Images , 2015, IEEE Transactions on Medical Imaging.

[48]  Hao Chen,et al.  Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[49]  Pol Cirujeda,et al.  A 3-D Riesz-Covariance Texture Model for Prediction of Nodule Recurrence in Lung CT. , 2016, IEEE transactions on medical imaging.

[50]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[51]  David Dagan Feng,et al.  Feature-Based Image Patch Approximation for Lung Tissue Classification , 2013, IEEE Transactions on Medical Imaging.