Classification of Malignant and Benign Lung Nodule and Prediction of Image Label Class using Multi-Deep Model

Lung cancer has been listed as one of the world’s leading causes of death. Early diagnosis of lung nodules has great significance for the prevention of lung cancer. Despite major improvements in modern diagnosis and treatment, the five-year survival rate is only 18%. Before diagnosis, the classification of lung nodules is one important step, in particular, because automatic classification may help doctors with a valuable opinion. Although deep learning has shown improvement in the image classifications over traditional approaches, which focus on handcraft features, due to a large number of intra-class variational images and the inter-class similar images due to various imaging modalities, it remains challenging to classify lung nodule. In this paper, a multi-deep model (MD model) is proposed for lung nodule classification as well as to predict the image label class. This model is based on three phases that include multi-scale dilated convolutional blocks (MsDc), dual deep convolutional neural networks (DCNN A/B), and multi-task learning component (MTLc). Initially, the multi-scale features are derived through the MsDc process by using different dilated rates to enlarge the respective area. This technique is applied to a pair of images. Such images are accepted by dual DCNNs, and both models can learn mutually from each other in order to enhance the model accuracy. To further improve the performance of the proposed model, the output from both DCNNs split into two portions. The multi-task learning part is used to evaluate whether the input image pair is in the same group or not and also helps to classify them between benign and malignant. Furthermore, it can provide positive guidance if there is an error. Both the intra-class and inter-class (variation and similarity) of a dataset itself increase the efficiency of single DCNN. The effectiveness of mentioned technique is tested empirically by using the popular Lung Image Consortium Database (LIDC) dataset. The results show that the strategy is highly efficient in the form of sensitivity of 90.67%, specificity 90.80%, and accuracy of 90.73%.

[1]  Zongyue Wang,et al.  Multi-scale Convolutional Neural Network Based on 3D Context Fusion for Lesion Detection , 2019, PRCV.

[2]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[3]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[4]  Alexander Wong,et al.  Lung Nodule Classification Using Deep Features in CT Images , 2015, 2015 12th Conference on Computer and Robot Vision.

[5]  Shipeng Li,et al.  A bag-of-objects retrieval model for web image search , 2012, ACM Multimedia.

[6]  Niranjan Khandelwal,et al.  A Combination of Shape and Texture Features for Classification of Pulmonary Nodules in Lung CT Images , 2016, Journal of Digital Imaging.

[7]  Vijay S. Pande,et al.  Massively Multitask Networks for Drug Discovery , 2015, ArXiv.

[8]  Wei Shen,et al.  Multi-scale Convolutional Neural Networks for Lung Nodule Classification , 2015, IPMI.

[9]  Yanjie Wang,et al.  Multi-scale dilated convolution of convolutional neural network for image denoising , 2019, Multimedia Tools and Applications.

[10]  Hong Zhao,et al.  Texture Feature Analysis for Computer-Aided Diagnosis on Pulmonary Nodules , 2015, Journal of Digital Imaging.

[11]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[12]  L. Schwartz,et al.  Automatic detection of small lung nodules on CT utilizing a local density maximum algorithm , 2003, Journal of applied clinical medical physics.

[13]  Bai Ying Lei,et al.  Automatic Scoring of Multiple Semantic Attributes With Multi-Task Feature Leverage: A Study on Pulmonary Nodules in CT Images , 2017, IEEE Transactions on Medical Imaging.

[14]  Brian Kingsbury,et al.  New types of deep neural network learning for speech recognition and related applications: an overview , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[15]  Hongfei Lin,et al.  Deep Transfer Learning for Modality Classification of Medical Images , 2017, Inf..

[16]  Hao Chen,et al.  DCAN: Deep contour‐aware networks for object instance segmentation from histology images , 2017, Medical Image Anal..

[17]  Heng Huang,et al.  Large Margin Local Estimate With Applications to Medical Image Classification , 2015, IEEE Transactions on Medical Imaging.

[18]  Qing-Yuan Meng,et al.  Improved Multi-Scale Retinex Algorithm for Medical Image Enhancement , 2012 .

[19]  Wei Shen,et al.  Lung nodule malignancy prediction using multi-task convolutional neural network , 2017, Medical Imaging.

[20]  David Dagan Feng,et al.  An Ensemble of Fine-Tuned Convolutional Neural Networks for Medical Image Classification , 2017, IEEE Journal of Biomedical and Health Informatics.

[21]  Yanfang Ye,et al.  Incomplete Label Multi-Task Deep Learning for Spatio-Temporal Event Subtype Forecasting , 2019, AAAI.

[22]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  Wei Shen,et al.  Multi-crop Convolutional Neural Networks for lung nodule malignancy suspiciousness classification , 2017, Pattern Recognit..

[24]  Nasir M. Rajpoot,et al.  Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[25]  Qi Wu,et al.  Classification of Medical Images and Illustrations in the Biomedical Literature Using Synergic Deep Learning , 2017, ArXiv.

[26]  João Manuel R. S. Tavares,et al.  Automatic 3D pulmonary nodule detection in CT images: A survey , 2016, Comput. Methods Programs Biomed..

[27]  Liang-Tien Chia,et al.  Learning image-to-class distance metric for image classification , 2013, TIST.

[28]  Yunchao Wei,et al.  Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi-Supervised Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Ashnil Kumar,et al.  Subfigure and Multi-Label Classification using a Fine-Tuned Convolutional Neural Network , 2016, CLEF.

[31]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[32]  Shuiwang Ji,et al.  Deep Learning Segmentation of Optical Microscopy Images Improves 3-D Neuron Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[33]  Dennis Koelma,et al.  The ImageNet Shuffle: Reorganized Pre-training for Video Event Detection , 2016, ICMR.

[34]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Yang Li,et al.  Melanoma Classification on Dermoscopy Images Using a Neural Network Ensemble Model , 2017, IEEE Transactions on Medical Imaging.

[36]  Jaewoo Kang,et al.  Classification of lung nodules in CT scans using three-dimensional deep convolutional neural networks with a checkpoint ensemble method , 2018, BMC Medical Imaging.

[37]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[38]  Yan Xu,et al.  Deep learning of feature representation with multiple instance learning for medical image analysis , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).