A Smartphone based Application for Skin Cancer Classification Using Deep Learning with Clinical Images and Lesion Information

Over the last decades, the incidence of skin cancer, melanoma and non-melanoma, has increased at a continuous rate. In particular for melanoma, the deadliest type of skin cancer, early detection is important to increase patient prognosis. Recently, deep neural networks (DNNs) have become viable to deal with skin cancer detection. In this work, we present a smartphone-based application to assist on skin cancer detection. This application is based on a Convolutional Neural Network (CNN) trained on clinical images and patients demographics, both collected from smartphones. Also, as skin cancer datasets are imbalanced, we present an approach, based on the mutation operator of Differential Evolution (DE) algorithm, to balance data. In this sense, beyond provides a flexible tool to assist doctors on skin cancer screening phase, the method obtains promising results with a balanced accuracy of 85% and a recall of 96%.

[1]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[2]  Abhishek Verma,et al.  New Compact Deep Learning Model for Skin Cancer Recognition , 2018, 2018 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON).

[3]  Kosin Chamnongthai,et al.  Detection skin cancer using SVM and snake model , 2018, 2018 International Workshop on Advanced Image Technology (IWAIT).

[4]  Ni Zhang,et al.  Skin cancer diagnosis based on optimized convolutional neural network , 2020, Artif. Intell. Medicine.

[5]  Tim Holland-Letz,et al.  Deep neural networks are superior to dermatologists in melanoma image classification. , 2019, European journal of cancer.

[6]  Thomas P. Trappenberg,et al.  A Deep Learning Based Approach to Skin Lesion Border Extraction With a Novel Edge Detector in Dermoscopy Images , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[7]  H. Feng,et al.  Comparison of Dermatologist Density Between Urban and Rural Counties in the United States , 2018, JAMA dermatology.

[8]  Ahmet Demir,et al.  Early detection of skin cancer using deep learning architectures: resnet-101 and inception-v3 , 2019, 2019 Medical Technologies Congress (TIPTEKNO).

[9]  Danail Stoyanov,et al.  OR 2.0 Context-Aware Operating Theaters, Computer Assisted Robotic Endoscopy, Clinical Image-Based Procedures, and Skin Image Analysis , 2018, Lecture Notes in Computer Science.

[10]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[11]  Renato A. Krohling,et al.  The impact of patient clinical information on automated skin cancer detection , 2020, Comput. Biol. Medicine.

[12]  Alper Arik,et al.  Deep learning based skin cancer diagnosis , 2017, 2017 25th Signal Processing and Communications Applications Conference (SIU).

[13]  Serestina Viriri,et al.  Deep Learning-Based System for Automatic Melanoma Detection , 2020, IEEE Access.

[14]  André G. C. Pacheco,et al.  PAD-UFES-20: A skin lesion dataset composed of patient data and clinical images collected from smartphones , 2020, Data in brief.

[15]  Matt Berseth,et al.  ISIC 2017 - Skin Lesion Analysis Towards Melanoma Detection , 2017, ArXiv.

[16]  Dorin Moldovan,et al.  Transfer Learning Based Method for Two-Step Skin Cancer Images Classification , 2019, 2019 E-Health and Bioengineering Conference (EHB).

[17]  Carola Berking,et al.  Melanoma , 2018, The Lancet.

[18]  Young Im Cho,et al.  Adaptive Activation Functions for Skin Lesion Classification Using Deep Neural Networks , 2018, 2018 Joint 10th International Conference on Soft Computing and Intelligent Systems (SCIS) and 19th International Symposium on Advanced Intelligent Systems (ISIS).

[19]  André G. C. Pacheco,et al.  Skin lesion segmentation using deep learning for images acquired from smartphones , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[20]  Tokunbo Ogunfunmi,et al.  Deep Learning Based Image Classification for Remote Medical Diagnosis , 2018, 2018 IEEE Global Humanitarian Technology Conference (GHTC).

[21]  Tim Holland-Letz,et al.  Superior skin cancer classification by the combination of human and artificial intelligence. , 2019, European journal of cancer.

[22]  Frédéric Andrès,et al.  Machine Learning on Mobile: An On-device Inference App for Skin Cancer Detection , 2019, 2019 Fourth International Conference on Fog and Mobile Edge Computing (FMEC).

[23]  A. Jemal,et al.  Cancer statistics, 2019 , 2019, CA: a cancer journal for clinicians.

[24]  Achim Hekler,et al.  Skin Cancer Classification Using Convolutional Neural Networks: Systematic Review , 2018, Journal of medical Internet research.

[25]  Ismail Jouny,et al.  Mobile melanoma detection application for Android smart phones , 2015, 2015 41st Annual Northeast Biomedical Engineering Conference (NEBEC).

[26]  Amirreza Mahbod,et al.  Skin Lesion Classification Using Hybrid Deep Neural Networks , 2017, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[27]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[28]  Eduardo Valle,et al.  Data Augmentation for Skin Lesion Analysis , 2018, OR 2.0/CARE/CLIP/ISIC@MICCAI.

[29]  Jorge S. Marques,et al.  Deep Learning For Skin Cancer Diagnosis With Hierarchical Architectures , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[30]  A. Enk,et al.  Systematic outperformance of 112 dermatologists in multiclass skin cancer image classification by convolutional neural networks. , 2019, European journal of cancer.

[31]  Fernando Pereira dos Santos,et al.  Robust Feature Spaces from Pre-Trained Deep Network Layers for Skin Lesion Classification , 2018, 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI).

[32]  Anandi Giridharan,et al.  Convolutional Neural Networks for classifying skin lesions , 2019, TENCON 2019 - 2019 IEEE Region 10 Conference (TENCON).

[33]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[35]  Harald Kittler,et al.  Descriptor : The HAM 10000 dataset , a large collection of multi-source dermatoscopic images of common pigmented skin lesions , 2018 .

[36]  Ali Mohammad Alqudah,et al.  The melanoma skin cancer detection and classification using support vector machine , 2017, 2017 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT).

[37]  Ayyaz Hussain,et al.  Classification of Melanoma and Nevus in Digital Images for Diagnosis of Skin Cancer , 2019, IEEE Access.

[38]  Achim Hekler,et al.  Comparing artificial intelligence algorithms to 157 German dermatologists: the melanoma classification benchmark. , 2019, European journal of cancer.

[39]  Andre G. C. Pacheco,et al.  Learning dynamic weights for an ensemble of deep models applied to medical imaging classification , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[40]  Sharath Pankanti,et al.  Deep learning ensembles for melanoma recognition in dermoscopy images , 2016, IBM J. Res. Dev..

[41]  András Hajdu,et al.  Classification Of Skin Lesions Using An Ensemble Of Deep Neural Networks , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[42]  Andre G. C. Pacheco,et al.  An app to detect melanoma using deep learning: An approach to handle imbalanced data based on evolutionary algorithms , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[43]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[44]  Stephen Kwek,et al.  Applying Support Vector Machines to Imbalanced Datasets , 2004, ECML.

[45]  Indu Sreedevi,et al.  Attention-guided deep convolutional neural networks for skin cancer classification , 2019, 2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA).

[46]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).