Skin Lesion Segmentation in Dermoscopic Images with Combination of YOLO and GrabCut Algorithm

Skin lesion segmentation has a critical role in the early and accurate diagnosis of skin cancer by computerized systems. However, automatic segmentation of skin lesions in dermoscopic images is a challenging task owing to difficulties including artifacts (hairs, gel bubbles, ruler markers), indistinct boundaries, low contrast and varying sizes and shapes of the lesion images. This paper proposes a novel and effective pipeline for skin lesion segmentation in dermoscopic images combining a deep convolutional neural network named as You Only Look Once (YOLO) and the GrabCut algorithm. This method performs lesion segmentation using a dermoscopic image in four steps: 1. Removal of hairs on the lesion, 2. Detection of the lesion location, 3. Segmentation of the lesion area from the background, 4. Post-processing with morphological operators. The method was evaluated on two publicly well-known datasets, that is the PH2 and the ISBI 2017 (Skin Lesion Analysis Towards Melanoma Detection Challenge Dataset). The proposed pipeline model has achieved a 90% sensitivity rate on the ISBI 2017 dataset, outperforming other deep learning-based methods. The method also obtained close results according to the results obtained from other methods in the literature in terms of metrics of accuracy, specificity, Dice coefficient, and Jaccard index.

[1]  Yading Yuan,et al.  Improving Dermoscopic Image Segmentation With Enhanced Convolutional-Deconvolutional Networks , 2017, IEEE Journal of Biomedical and Health Informatics.

[2]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[3]  Gerald Schaefer,et al.  Lesion Border Detection in Dermoscopy Images Using Ensembles of Thresholding Methods , 2013, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[4]  M A Weinstock,et al.  The global burden of melanoma: results from the Global Burden of Disease Study 2015 , 2017, The British journal of dermatology.

[5]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[7]  T Lee,et al.  Dullrazor®: A software approach to hair removal from images , 1997, Comput. Biol. Medicine.

[8]  Qaisar Abbas,et al.  Skin tumor area extraction using an improved dynamic programming approach , 2012, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[9]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[10]  Junji Maeda,et al.  Comparison of Segmentation Methods for Melanoma Diagnosis in Dermoscopy Images , 2009, IEEE Journal of Selected Topics in Signal Processing.

[11]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[12]  W. Jaschke,et al.  Automated melanoma recognition , 2001, IEEE Transactions on Medical Imaging.

[13]  L. R. Dice Measures of the Amount of Ecologic Association Between Species , 1945 .

[14]  Mehmet Türkan,et al.  A survey on automated melanoma detection , 2018, Eng. Appl. Artif. Intell..

[15]  Mutlu Mete,et al.  Lesion detection in demoscopy images with novel density-based and active contour approaches , 2010, BMC Bioinformatics.

[16]  Dong Wang,et al.  The Experimental Implementation of GrabCut for Hardcode Subtitle Extraction , 2018, 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS).

[17]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Pedro M. Ferreira,et al.  PH2 - A dermoscopic image database for research and benchmarking , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[19]  Xudong Jiang,et al.  Melanoma Recognition in Dermoscopy Images via Aggregated Deep Convolutional Features , 2019, IEEE Transactions on Biomedical Engineering.

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  P. C. Siddalingaswamy,et al.  Techniques and algorithms for computer aided diagnosis of pigmented skin lesions - A review , 2018, Biomed. Signal Process. Control..

[22]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[23]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[24]  A. Jemal,et al.  Cancer statistics, 2019 , 2019, CA: a cancer journal for clinicians.

[25]  Hang Li,et al.  Dense Deconvolutional Network for Skin Lesion Segmentation , 2019, IEEE Journal of Biomedical and Health Informatics.

[26]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[27]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Noel C. F. Codella,et al.  Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC) , 2016, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[29]  Mutlu Mete,et al.  An improved border detection in dermoscopy images for density based clustering , 2011, BMC Bioinformatics.

[30]  Yading Yuan,et al.  Automatic Skin Lesion Segmentation Using Deep Fully Convolutional Networks With Jaccard Distance , 2017, IEEE Transactions on Medical Imaging.

[31]  Yanhui Guo,et al.  A novel optimized neutrosophic k-means using genetic algorithm for skin lesion detection in dermoscopy images , 2018, Signal Image Video Process..

[32]  Gerald Schaefer,et al.  Lesion border detection in dermoscopy images , 2009, Comput. Medical Imaging Graph..

[33]  Gerald Schaefer,et al.  An ensemble classification approach for melanoma diagnosis , 2014, Memetic Computing.

[34]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  David Dagan Feng,et al.  Automatic Skin Lesion Analysis using Large-scale Dermoscopy Images and Deep Residual Networks , 2017, ArXiv.

[36]  Huiyu Zhou,et al.  A State-of-the-Art Survey on Lesion Border Detection in Dermoscopy Images , 2015 .

[37]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  A. Jemal,et al.  Cancer statistics, 2018 , 2018, CA: a cancer journal for clinicians.

[39]  Jian Zhi Hu,et al.  Studies of Secondary Melanoma on C57BL/6J Mouse Liver Using 1H NMR Metabolomics , 2013, Metabolites.

[40]  Gerald Schaefer,et al.  Anisotropic Mean Shift Based Fuzzy C-Means Segmentation of Dermoscopy Images , 2009, IEEE Journal of Selected Topics in Signal Processing.

[41]  Alan C. Bovik,et al.  Automatic segmentation of dermoscopy images using self-generating neural networks seeded by genetic algorithm , 2013, Pattern Recognit..

[42]  Nico Karssemeijer,et al.  Can radiologists improve their breast cancer detection in mammography when using a deep learning based computer system as decision support? , 2018, Other Conferences.

[43]  M. Emin Yüksel,et al.  Accurate Segmentation of Dermoscopic Images by Image Thresholding Based on Type-2 Fuzzy Logic , 2009, IEEE Transactions on Fuzzy Systems.

[44]  João Paulo Papa,et al.  Computational methods for the image segmentation of pigmented skin lesions: A review , 2016, Comput. Methods Programs Biomed..

[45]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  U. G. Dailey Cancer,Facts and Figures about. , 2022, Journal of the National Medical Association.

[48]  João Manuel R. S. Tavares,et al.  A Review of the Quantification and Classification of Pigmented Skin Lesions: From Dedicated to Hand-Held Devices , 2015, Journal of Medical Systems.

[49]  Manu Goyal,et al.  Multi-class Semantic Segmentation of Skin Lesions via Fully Convolutional Networks , 2017, BIOINFORMATICS.

[50]  Mun-Taek Choi,et al.  Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks , 2018, Comput. Methods Programs Biomed..

[51]  Dagan Feng,et al.  Dermoscopic Image Segmentation via Multistage Fully Convolutional Networks , 2017, IEEE Transactions on Biomedical Engineering.

[52]  Sumul Ashok Gandhi,et al.  Skin Cancer Epidemiology, Detection, and Management. , 2015, The Medical clinics of North America.

[53]  Qaisar Abbas,et al.  Lesion border detection in dermoscopy images using dynamic programming , 2011, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[54]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[55]  John Paoli,et al.  Accuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin , 2017, Journal of the American Academy of Dermatology.

[56]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Fred Godtliebsen,et al.  Unsupervised segmentation for digital dermoscopic images , 2010, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[58]  Ning Wang,et al.  Segmentation of dermoscopy image using adversarial networks , 2018, Multimedia Tools and Applications.

[59]  Dr. Kailash Shaw,et al.  Skin Lesion Analysis towards Melanoma Detection , 2018 .

[60]  R. Joe Stanley,et al.  Modified watershed technique and post-processing for segmentation of skin lesions in dermoscopy images , 2011, Comput. Medical Imaging Graph..

[61]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[62]  Rebecca L. Siegel Mph,et al.  Cancer statistics, 2018 , 2018 .

[63]  Ming Chao,et al.  Improving Dermoscopic Image Segmentation With Enhanced Convolutional-Deconvolutional Networks , 2017, IEEE Journal of Biomedical and Health Informatics.

[64]  Abder-Rahman Ali,et al.  A systematic review of automated melanoma detection in dermatoscopic images and its ground truth data , 2012, Medical Imaging.

[65]  Z. Zivkovic Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[66]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[67]  Mutlu Mete,et al.  A soft kinetic data structure for lesion border detection , 2010, Bioinform..

[68]  Randy H. Moss,et al.  Automatic lesion boundary detection in dermoscopy images using gradient vector flow snakes , 2005, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[69]  Enoch Peserico,et al.  Simpler, Faster, More Accurate Melanocytic Lesion Segmentation Through MEDS , 2014, IEEE Transactions on Biomedical Engineering.

[70]  Rafael García,et al.  Computerized analysis of pigmented skin lesions: A review , 2012, Artif. Intell. Medicine.

[71]  Hamid R. Tizhoosh,et al.  Skin lesion segmentation: U-Nets versus clustering , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[72]  Giovanni Pellacani,et al.  Comparison between morphological parameters in pigmented skin lesion images acquired by means of epiluminescence surface microscopy and polarized-light videomicroscopy. , 2002, Clinics in dermatology.

[73]  João Paulo Papa,et al.  WITHDRAWN: Computational methods for the image segmentation of pigmented skin lesions: A Review , 2016 .

[74]  José García Rodríguez,et al.  A Review on Deep Learning Techniques Applied to Semantic Segmentation , 2017, ArXiv.

[75]  Talicia Tarver,et al.  Cancer Facts & Figures 2012. American Cancer Society (ACS) , 2012 .

[76]  LinLin Shen,et al.  Skin Lesion Analysis towards Melanoma Detection Using Deep Learning Network , 2017, Sensors.

[77]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[78]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[79]  Borko Furht,et al.  Rethinking Skin Lesion Segmentation in a Convolutional Classifier , 2018, Journal of Digital Imaging.