An effective approach for CT lung segmentation using mask region-based convolutional neural networks

Computer vision systems have numerous tools to assist in various medical fields, notably in image diagnosis. Computed tomography (CT) is the principal imaging method used to assist in the diagnosis of diseases such as bone fractures, lung cancer, heart disease, and emphysema, among others. Lung cancer is one of the four main causes of death in the world. The lung regions in the CT images are marked manually by a specialist as this initial step is a significant challenge for computer vision techniques. Once defined, the lung regions are segmented for clinical diagnoses. This work proposes an automatic segmentation of the lungs in CT images, using the Convolutional Neural Network (CNN) Mask R-CNN, to specialize the model for lung region mapping, combined with supervised and unsupervised machine learning methods (Bayes, Support Vectors Machine (SVM), K-means and Gaussian Mixture Models (GMMs)). Our approach using Mask R-CNN with the K-means kernel produced the best results for lung segmentation reaching an accuracy of 97.68 ± 3.42% and an average runtime of 11.2 s. We compared our results against other works for validation purposes, and our approach had the highest accuracy and was faster than some state-of-the-art methods.

[1]  David Gavaghan,et al.  Review of automatic pulmonary lobe segmentation methods from CT , 2015, Comput. Medical Imaging Graph..

[2]  Hieu Trung Huynh,et al.  A Deep Learning Method for Lung Segmentation on Large Size Chest X-Ray Image , 2019, RIVF.

[3]  Ken Goldberg,et al.  Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[4]  João Manuel R. S. Tavares,et al.  Novel and powerful 3D adaptive crisp active contour method applied in the segmentation of CT lung images , 2017, Medical Image Anal..

[5]  Hui Zhang,et al.  Mapping and candidate-gene screening of the novel Turnip mosaic virus resistance gene retr02 in Chinese cabbage (Brassica rapa L.) , 2012, Theoretical and Applied Genetics.

[6]  Zhen Xie,et al.  Instance Segmentation of Anatomical Structures in Chest Radiographs , 2019, 2019 IEEE 32nd International Symposium on Computer-Based Medical Systems (CBMS).

[7]  Parisa Rashidi,et al.  Autonomous Detection of Disruptions in the Intensive Care Unit Using Deep Mask R-CNN , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Weiguo Fan,et al.  A new image classification method using CNN transfer learning and web data augmentation , 2018, Expert Syst. Appl..

[9]  L. R. Dice Measures of the Amount of Ecologic Association Between Species , 1945 .

[10]  Lisheng Xu,et al.  Segmentation of lung parenchyma in CT images using CNN trained with the clustering algorithm generated dataset , 2019, BioMedical Engineering OnLine.

[11]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[12]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[13]  F Neukirch,et al.  An international survey of chronic obstructive pulmonary disease in young adults according to GOLD stages , 2004, Thorax.

[14]  A. Buist,et al.  Similarities and differences between asthma and chronic obstructive pulmonary disease: treatment and early outcomes , 2003, European Respiratory Journal.

[15]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[16]  Ali Hassan Sodhro,et al.  A multi-sensor data fusion enabled ensemble approach for medical data from body sensor networks , 2020, Inf. Fusion.

[17]  Wenqing Sun,et al.  Fast and fully-automated detection and segmentation of pulmonary nodules in thoracic CT scans using deep convolutional neural networks , 2019, Comput. Medical Imaging Graph..

[18]  Kenji Suzuki,et al.  Overview of deep learning in medical imaging , 2017, Radiological Physics and Technology.

[19]  Ali Hassan Sodhro,et al.  Artificial Intelligence based QoS optimization for multimedia communication in IoV systems , 2019, Future Gener. Comput. Syst..

[20]  Michaël Gharbi,et al.  Convolutional neural network for earthquake detection and location , 2017, Science Advances.

[21]  Jiang Hsieh,et al.  Computed Tomography: Principles, Design, Artifacts, and Recent Advances, Fourth Edition , 2022 .

[22]  Ali Hassan Sodhro,et al.  Medical-QoS Based Telemedicine Service Selection Using Analytic Hierarchy Process , 2017, Handbook of Large-Scale Distributed Computing in Smart Healthcare.

[23]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[24]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Ali Hassan Sodhro,et al.  Artificial Intelligence-Driven Mechanism for Edge Computing-Based Industrial Applications , 2019, IEEE Transactions on Industrial Informatics.

[26]  Oluwarotimi Williams Samuel,et al.  A joint resource-aware and medical data security framework for wearable healthcare systems , 2019, Future Gener. Comput. Syst..

[27]  Xiaoyong Wang,et al.  High Throughput Lung and Lobar Segmentation by 2D and 3D CNN on Chest CT with Diffuse Lung Disease , 2018, RAMBO+BIA+TIA@MICCAI.

[28]  Hanqing Lu,et al.  Mask Guided Knowledge Distillation for Single Shot Detector , 2019, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[29]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[30]  Zhenmin Tang,et al.  Feature mask network for person re-identification , 2020, Pattern Recognit. Lett..

[31]  Jugurta R. Montalvão Filho,et al.  Speech Recognition in Noisy Environments with Convolutional Neural Networks , 2015, 2015 Brazilian Conference on Intelligent Systems (BRACIS).

[32]  Georgy Lebedev,et al.  Application of artificial intelligence methods to recognize pathologies on medical images , 2018, KES.

[33]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[34]  Catarina Eloy,et al.  Classification of breast cancer histology images using Convolutional Neural Networks , 2017, PloS one.

[35]  Yuxing Tang,et al.  Uldor: A Universal Lesion Detector For Ct Scans With Pseudo Masks And Hard Negative Example Mining , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[36]  Jonathan G. Goldin,et al.  Prediction of progression in idiopathic pulmonary fibrosis using CT scans at baseline: A quantum particle swarm optimization - Random forest approach , 2019, Artif. Intell. Medicine.

[37]  Alexei A. Efros,et al.  Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Roland S. Zimmermann,et al.  Faster Training of Mask R-CNN by Focusing on Instance Boundaries , 2018, Comput. Vis. Image Underst..

[39]  U. Rajendra Acharya,et al.  Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals , 2017, Comput. Biol. Medicine.

[40]  J. Ancochea,et al.  Importance of early diagnosis and treatment in idiopathic pulmonary fibrosis , 2018, Expert review of respiratory medicine.

[41]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[42]  Jakub Nalepa,et al.  Towards Detecting High-Uptake Lesions from Lung CT Scans Using Deep Learning , 2017, ICIAP.

[43]  Maria Wimmer,et al.  Fully Convolutional Architectures for Multiclass Segmentation in Chest Radiographs , 2017, IEEE Transactions on Medical Imaging.

[44]  Bin Song,et al.  BoSR: A CNN-based aurora image retrieval method , 2019, Neural Networks.

[45]  I. Bloch,et al.  Automatic knee meniscus tear detection and orientation classification with Mask-RCNN. , 2019, Diagnostic and interventional imaging.

[46]  Ankita Pramanik,et al.  Deep Learning Techniques—R-CNN to Mask R-CNN: A Survey , 2019, Computational Intelligence in Pattern Recognition.

[47]  Xiang Chen,et al.  Software defect number prediction: Unsupervised vs supervised methods , 2019, Inf. Softw. Technol..

[48]  Ali Hassan Sodhro,et al.  An adaptive QoS computation for medical data processing in intelligent healthcare applications , 2019, Neural Computing and Applications.

[49]  Mingqi Lu,et al.  Dilated Light-Head R-CNN using tri-center loss for driving behavior recognition , 2019, Image Vis. Comput..

[50]  Souvik Ghosh,et al.  Using Convolutions and Image Processing Techniques to Segment Lungs from CT Data , 2019, Advances in Intelligent Systems and Computing.

[51]  Simon Fong,et al.  Automated 3-D lung tumor detection and classification by an active contour model and CNN classifier , 2019, Expert Syst. Appl..

[52]  Bernd Sturmfels,et al.  Moment Varieties of Gaussian Mixtures , 2015, 1510.04654.

[53]  Yi Liu,et al.  Just-in-time semi-supervised soft sensor for quality prediction in industrial rubber mixers , 2018, Chemometrics and Intelligent Laboratory Systems.

[54]  Paul Aljabar,et al.  Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[55]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[56]  James A. Bartholomai,et al.  Lung Cancer Survival Prediction via Machine Learning Regression, Classification, and Statistical Techniques , 2018, 2018 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[57]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[59]  Ron Kikinis,et al.  Improved watershed transform for medical image segmentation using prior information , 2004, IEEE Transactions on Medical Imaging.

[60]  Guanglin Li,et al.  Optimization of signal quality over comfortability of textile electrodes for ECG monitoring in fog computing based medical applications , 2018, Future Gener. Comput. Syst..

[61]  Juan C. Celedón,et al.  Chronic Obstructive Pulmonary Disease in Hispanics. A 9-Year Update. , 2018, American journal of respiratory and critical care medicine.

[62]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[63]  Christian Wachinger,et al.  DeepNAT: Deep convolutional neural network for segmenting neuroanatomy , 2017, NeuroImage.

[64]  Hui Yu,et al.  Segmentation of Lung Nodule in CT Images Based on Mask R-CNN , 2018, 2018 9th International Conference on Awareness Science and Technology (iCAST).

[65]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[66]  Bram van Ginneken,et al.  Computer analysis of computed tomography scans of the lung: a survey , 2006, IEEE Transactions on Medical Imaging.

[67]  Sangheum Hwang,et al.  Accurate Lung Segmentation via Network-Wise Training of Convolutional Networks , 2017, DLMIA/ML-CDS@MICCAI.

[68]  Richard Geoffrey White,et al.  Comparing the performance of SAR image segmentation algorithms , 1992 .

[69]  Jeremiah W. Johnson,et al.  Adapting Mask-RCNN for Automatic Nucleus Segmentation , 2018, ArXiv.

[70]  Ali Hassan Sodhro,et al.  Towards an optimal resource management for IoT based Green and sustainable smart cities , 2019, Journal of Cleaner Production.

[71]  W. Guan,et al.  Impact of air pollution on the burden of chronic respiratory diseases in China: time for urgent action , 2016, The Lancet.

[72]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[73]  Ali Hassan Sodhro,et al.  Energy-efficient adaptive transmission power control for wireless body area networks , 2016, IET Commun..

[74]  P. Barnes,et al.  Inflammatory mechanisms in patients with chronic obstructive pulmonary disease. , 2016, The Journal of allergy and clinical immunology.

[75]  Joel J. P. C. Rodrigues,et al.  A new effective and powerful medical image segmentation algorithm based on optimum path snakes , 2019, Appl. Soft Comput..

[76]  Pedro Pedrosa Rebouças Filho,et al.  3D Adaptive Balloon Active Contour: method of segmentation of structures in three dimensions , 2015 .

[77]  J. Lelieveld,et al.  The contribution of outdoor air pollution sources to premature mortality on a global scale , 2015, Nature.

[78]  Ashish Khare,et al.  Soft-Thresholding for Denoising of Medical Images - a Multiresolution Approach , 2005, Int. J. Wavelets Multiresolution Inf. Process..

[79]  Wei Zeng,et al.  Deep Learning with Lung Segmentation and Bone Shadow Exclusion Techniques for Chest X-Ray Analysis of Lung Cancer , 2017, ArXiv.

[80]  Marios Anthimopoulos,et al.  Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[81]  Lei Sun,et al.  Mask R-CNN With Pyramid Attention Network for Scene Text Detection , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[82]  Huimin Ma,et al.  Salient object detection via fast R-CNN and low-level cues , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[83]  Xavier Lladó,et al.  Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review , 2017, Artif. Intell. Medicine.

[84]  Xin Jiang,et al.  A semi-supervised convolutional transfer neural network for 3D pulmonary nodules detection , 2020, Neurocomputing.

[85]  Fátima N. S. de Medeiros,et al.  Rotation-invariant feature extraction using a structural co-occurrence matrix , 2016 .

[86]  Kun Zhang,et al.  Lung sounds classification using convolutional neural networks , 2018, Artif. Intell. Medicine.

[87]  Li Chen,et al.  AU R-CNN: Encoding Expert Prior Knowledge into R-CNN for Action Unit Detection , 2018, Neurocomputing.

[88]  Luo Xinggang,et al.  Medical Image Segmentation Based on Improved Ostu Algorithm and Regional Growth Algorithm , 2006 .

[89]  Victor Hugo C. de Albuquerque,et al.  Novel Adaptive Balloon Active Contour Method based on internal force for image segmentation - A systematic evaluation on synthetic and real images , 2014, Expert Syst. Appl..

[90]  Arun Ross,et al.  On automated source selection for transfer learning in convolutional neural networks , 2018, Pattern Recognit..

[91]  Mohd Javaid,et al.  Industry 4.0 applications in medical field: A brief review , 2019, Current Medicine Research and Practice.

[92]  Yang Yu,et al.  Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN , 2019, Comput. Electron. Agric..

[93]  Y. Yang,et al.  Medical image enhancement algorithm based on wavelet transform , 2010 .