Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks

BACKGROUND AND OBJECTIVE Automatic segmentation of skin lesions in dermoscopy images is still a challenging task due to the large shape variations and indistinct boundaries of the lesions. Accurate segmentation of skin lesions is a key prerequisite step for any computer-aided diagnostic system to recognize skin melanoma. METHODS In this paper, we propose a novel segmentation methodology via full resolution convolutional networks (FrCN). The proposed FrCN method directly learns the full resolution features of each individual pixel of the input data without the need for pre- or post-processing operations such as artifact removal, low contrast adjustment, or further enhancement of the segmented skin lesion boundaries. We evaluated the proposed method using two publicly available databases, the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 Challenge and PH2 datasets. To evaluate the proposed method, we compared the segmentation performance with the latest deep learning segmentation approaches such as the fully convolutional network (FCN), U-Net, and SegNet. RESULTS Our results showed that the proposed FrCN method segmented the skin lesions with an average Jaccard index of 77.11% and an overall segmentation accuracy of 94.03% for the ISBI 2017 test dataset and 84.79% and 95.08%, respectively, for the PH2 dataset. In comparison to FCN, U-Net, and SegNet, the proposed FrCN outperformed them by 4.94%, 15.47%, and 7.48% for the Jaccard index and 1.31%, 3.89%, and 2.27% for the segmentation accuracy, respectively. Furthermore, the proposed FrCN achieved a segmentation accuracy of 95.62% for some representative clinical benign cases, 90.78% for the melanoma cases, and 91.29% for the seborrheic keratosis cases in the ISBI 2017 test dataset, exhibiting better performance than those of FCN, U-Net, and SegNet. CONCLUSIONS We conclude that using the full spatial resolutions of the input image could enable to learn better specific and prominent features, leading to an improvement in the segmentation performance.

[1]  Dagan Feng,et al.  Dermoscopic Image Segmentation via Multistage Fully Convolutional Networks , 2017, IEEE Transactions on Biomedical Engineering.

[2]  A. Smilde,et al.  Assessing the statistical validity of proteomics based biomarkers. , 2007, Analytica chimica acta.

[3]  Mutlu Mete,et al.  A soft kinetic data structure for lesion border detection , 2010, Bioinform..

[4]  Randy H. Moss,et al.  Automatic lesion boundary detection in dermoscopy images using gradient vector flow snakes , 2005, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[5]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Qaisar Abbas,et al.  Lesion border detection in dermoscopy images using dynamic programming , 2011, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[7]  Gerald Schaefer,et al.  Gradient vector flow with mean shift for skin lesion segmentation , 2011, Comput. Medical Imaging Graph..

[8]  Enoch Peserico,et al.  Simpler, Faster, More Accurate Melanocytic Lesion Segmentation Through MEDS , 2014, IEEE Transactions on Biomedical Engineering.

[9]  Rafael García,et al.  Computerized analysis of pigmented skin lesions: A review , 2012, Artif. Intell. Medicine.

[10]  Gerald Schaefer,et al.  Anisotropic Mean Shift Based Fuzzy C-Means Segmentation of Dermoscopy Images , 2009, IEEE Journal of Selected Topics in Signal Processing.

[11]  Hayit Greenspan,et al.  Chest pathology identification using deep feature selection with non-medical training , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[12]  M. Emin Yüksel,et al.  Accurate Segmentation of Dermoscopic Images by Image Thresholding Based on Type-2 Fuzzy Logic , 2009, IEEE Transactions on Fuzzy Systems.

[13]  Pedro M. Ferreira,et al.  PH2 - A dermoscopic image database for research and benchmarking , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[14]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[15]  Alan C. Bovik,et al.  Automatic segmentation of dermoscopy images using self-generating neural networks seeded by genetic algorithm , 2013, Pattern Recognit..

[16]  P. C. Siddalingaswamy,et al.  Techniques and algorithms for computer aided diagnosis of pigmented skin lesions - A review , 2018, Biomed. Signal Process. Control..

[17]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[18]  Andrew P. Bradley,et al.  Automated Analysis of Unregistered Multi-View Mammograms With Deep Learning , 2017, IEEE Transactions on Medical Imaging.

[19]  Md. Zia Uddin,et al.  A Single Depth Sensor Based Human Activity Recognition via Convolutional Neural Network , 2017 .

[20]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[21]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[22]  Edwin Valarezo,et al.  Simultaneous Detection and Classification of Breast Masses in Digital Mammograms via a Deep Learning YOLO-based CAD System , 2018, Comput. Methods Programs Biomed..

[23]  Qiang Guo,et al.  Convolutional feature learning and Hybrid CNN-HMM for scene number recognition , 2016, Neurocomputing.

[24]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[25]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[26]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[27]  M. A. Al-masni,et al.  Detection and classification of the breast abnormalities in digital mammograms via regional Convolutional Neural Network , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[28]  M. Binder,et al.  Epiluminescence microscopy. A useful tool for the diagnosis of pigmented skin lesions for formally trained dermatologists. , 1995, Archives of dermatology.

[29]  Lubomir M. Hadjiiski,et al.  Mass detection in digital breast tomosynthesis: Deep convolutional neural network with transfer learning from mammography. , 2016, Medical physics.

[30]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[31]  Yong Fan,et al.  A deep learning model integrating FCNNs and CRFs for brain tumor segmentation , 2017, Medical Image Anal..

[32]  Gustavo Carneiro,et al.  A deep learning approach for the analysis of masses in mammograms with minimal user intervention , 2017, Medical Image Anal..

[33]  Mutlu Mete,et al.  An improved border detection in dermoscopy images for density based clustering , 2011, BMC Bioinformatics.

[34]  Gerald Schaefer,et al.  Lesion border detection in dermoscopy images , 2009, Comput. Medical Imaging Graph..

[35]  H. HELM CLAYTON.  Eleven-year Sun-spot Weather Period and its Multiples , 1895, Nature.

[36]  Sven Loncaric,et al.  Retinal Vessel Segmentation using Deep Neural Networks , 2015, VISAPP.

[37]  O. Stegle,et al.  Deep learning for computational biology , 2016, Molecular systems biology.

[38]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Noel C. F. Codella,et al.  Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC) , 2016, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[40]  Gerald Schaefer,et al.  Lesion Border Detection in Dermoscopy Images Using Ensembles of Thresholding Methods , 2013, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[41]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[42]  Hao Chen,et al.  Automated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks , 2017, IEEE Transactions on Medical Imaging.

[43]  Gerald Schaefer,et al.  An ensemble classification approach for melanoma diagnosis , 2014, Memetic Computing.

[44]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Huiyu Zhou,et al.  A State-of-the-Art Survey on Lesion Border Detection in Dermoscopy Images , 2015 .

[46]  David Dagan Feng,et al.  Automatic Skin Lesion Analysis using Large-scale Dermoscopy Images and Deep Residual Networks , 2017, ArXiv.

[47]  Yide Ma,et al.  An Efficient Approach for Automated Mass Segmentation and Classification in Mammograms , 2015, Journal of Digital Imaging.

[48]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[49]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[50]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[51]  Yasser M. Kadah,et al.  An Automatic Computer-Aided Diagnosis System for Breast Cancer in Digital Mammograms via Deep Belief Network , 2018 .

[52]  Fred Godtliebsen,et al.  Unsupervised segmentation for digital dermoscopic images , 2010, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[53]  R. Joe Stanley,et al.  Modified watershed technique and post-processing for segmentation of skin lesions in dermoscopy images , 2011, Comput. Medical Imaging Graph..

[54]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[55]  Byunghan Lee,et al.  Deep learning in bioinformatics , 2016, Briefings Bioinform..

[56]  Stefan Holban,et al.  Segmentation of bone structure in X-ray images using convolutional neural network , 2013 .

[57]  Antonio Criminisi,et al.  Segmentation of Brain Tumor Tissues with Convolutional Neural Networks , 2014 .

[58]  W. Jaschke,et al.  Automated melanoma recognition , 2001, IEEE Transactions on Medical Imaging.

[59]  Parashkev Nachev,et al.  Computer Methods and Programs in Biomedicine NiftyNet: a deep-learning platform for medical imaging , 2022 .

[60]  Jeffrey E Gershenwald,et al.  Final version of 2009 AJCC melanoma staging and classification. , 2009, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[61]  Harald Ganster,et al.  Automated Melanoma Recognition , 2001, IEEE Trans. Medical Imaging.

[62]  Xinbo Gao,et al.  A deep feature based framework for breast masses classification , 2016, Neurocomputing.

[63]  Age K. Smilde,et al.  Double-check: validation of diagnostic statistics for PLS-DA models in metabolomics studies , 2011, Metabolomics.

[64]  Qaisar Abbas,et al.  Skin tumor area extraction using an improved dynamic programming approach , 2012, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[65]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[66]  Junji Maeda,et al.  Comparison of Segmentation Methods for Melanoma Diagnosis in Dermoscopy Images , 2009, IEEE Journal of Selected Topics in Signal Processing.

[67]  Yading Yuan,et al.  Automatic Skin Lesion Segmentation Using Deep Fully Convolutional Networks With Jaccard Distance , 2017, IEEE Transactions on Medical Imaging.

[68]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[69]  A. Jemal,et al.  Cancer statistics, 2017 , 2017, CA: a cancer journal for clinicians.

[70]  S. Menzies,et al.  Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta‐analysis of studies performed in a clinical setting , 2008, The British journal of dermatology.

[71]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[72]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[73]  Mutlu Mete,et al.  Lesion detection in demoscopy images with novel density-based and active contour approaches , 2010, BMC Bioinformatics.