Machine Learning in Dermatology: Current Applications, Opportunities, and Limitations

Machine learning (ML) has the potential to improve the dermatologist’s practice from diagnosis to personalized treatment. Recent advancements in access to large datasets (e.g., electronic medical records, image databases, omics), faster computing, and cheaper data storage have encouraged the development of ML algorithms with human-like intelligence in dermatology. This article is an overview of the basics of ML, current applications of ML, and potential limitations and considerations for further development of ML. We have identified five current areas of applications for ML in dermatology: (1) disease classification using clinical images; (2) disease classification using dermatopathology images; (3) assessment of skin diseases using mobile applications and personal monitoring devices; (4) facilitating large-scale epidemiology research; and (5) precision medicine. The purpose of this review is to provide a guide for dermatologists to help demystify the fundamentals of ML and its wide range of applications in order to better evaluate its potential opportunities and challenges.

[1]  Juan Lu,et al.  Automatic Segmentation of Scaling in 2-D Psoriasis Skin Images , 2013, IEEE Transactions on Medical Imaging.

[2]  L. Naldi,et al.  Mobile teledermatology for melanoma detection: Assessment of the validity in the framework of a population-based skin cancer awareness campaign in northern Italy. , 2019, Journal of the American Academy of Dermatology.

[3]  Cristian Navarrete-Dechent,et al.  Automated Dermatological Diagnosis: Hype or Reality? , 2018, The Journal of investigative dermatology.

[4]  Anabik Pal,et al.  Psoriasis skin biopsy image segmentation using Deep Convolutional Neural Network , 2018, Comput. Methods Programs Biomed..

[5]  Leslie J. Crofford,et al.  Rule-based and machine learning algorithms identify patients with systemic sclerosis accurately in the electronic health record , 2019, Arthritis Research & Therapy.

[6]  M. Kulldorff,et al.  Research Techniques Made Simple: An Introduction to Use and Analysis of Big Data in Dermatology. , 2017, The Journal of investigative dermatology.

[7]  Dani Ihtatho,et al.  Area assessment of psoriasis lesions for PASI scoring , 2009, Journal of medical engineering & technology.

[8]  M. Abràmoff,et al.  Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices , 2018, npj Digital Medicine.

[9]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[10]  Yi Han,et al.  Overview of Artificial Neural Networks , 2009, Artificial Neural Networks.

[11]  Achim Hekler,et al.  Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. , 2019, European journal of cancer.

[12]  Jordan Yap,et al.  Multimodal skin lesion classification using deep learning , 2018, Experimental dermatology.

[13]  James A. Nichols,et al.  Machine learning: applications of artificial intelligence to imaging and diagnosis , 2018, Biophysical Reviews.

[14]  Jasjit S. Suri,et al.  Exploring the color feature power for psoriasis risk stratification and classification: A data mining paradigm , 2015, Comput. Biol. Medicine.

[15]  Ghassan Hamarneh,et al.  Deep features to classify skin lesions , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[16]  E.O. David,et al.  Skin cancer detection by deep learning and sound analysis algorithms: A prospective clinical study of an elementary dermoscope , 2019, EBioMedicine.

[17]  I Olkin,et al.  A comparison of dermatologists' and primary care physicians' accuracy in diagnosing melanoma: a systematic review. , 2001, Archives of dermatology.

[18]  Ghassan Hamarneh,et al.  Multi-resolution-Tract CNN with Hybrid Pretrained and Skin-Lesion Trained Layers , 2016, MLMI@MICCAI.

[19]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[20]  Sebastian Thrun,et al.  Skin Cancer Detection and Tracking using Data Synthesis and Deep Learning , 2016, AAAI Workshops.

[21]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[22]  Xiaoyu Cui,et al.  Assessing the effectiveness of artificial intelligence methods for melanoma:A retrospective review. , 2019, Journal of the American Academy of Dermatology.

[23]  Giuseppe Argenziano,et al.  Assessment of Accuracy of an Artificial Intelligence Algorithm to Detect Melanoma in Images of Skin Lesions , 2019, JAMA network open.

[24]  John R. Smith,et al.  Deep Learning, Sparse Coding, and SVM for Melanoma Recognition in Dermoscopy Images , 2015, MLMI.

[25]  H. Haenssle,et al.  Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists , 2018, Annals of oncology : official journal of the European Society for Medical Oncology.

[26]  C. Johnson,et al.  Validation of claims data algorithms to identify nonmelanoma skin cancer , 2012, The Journal of investigative dermatology.

[27]  O. Winther,et al.  Systematic review of machine learning for diagnosis and prognosis in dermatology , 2019, The Journal of dermatological treatment.

[28]  Xianglin L. Du,et al.  Ethnic differences among patients with cutaneous melanoma. , 2006, Archives of internal medicine.

[29]  S. Han,et al.  Keratinocytic Skin Cancer Detection on the Face Using Region-Based Convolutional Neural Network. , 2019, JAMA dermatology.

[30]  Sanjiv J. Shah,et al.  Application of Machine Learning to Determine Top Predictors of Non-calcified Coronary Burden in Psoriasis: an Observational Cohort Study. , 2019, Journal of the American Academy of Dermatology.

[31]  M. Abràmoff,et al.  Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning. , 2016, Investigative ophthalmology & visual science.

[32]  S. Tian,et al.  Shrinking the Psoriasis Assessment Gap: Early Gene-Expression Profiling Accurately Predicts Response to Long-Term Treatment. , 2017, The Journal of investigative dermatology.

[33]  Y Q Jiang,et al.  Recognizing basal cell carcinoma on smartphone‐captured digital histopathology images with a deep neural network , 2019, The British journal of dermatology.

[34]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[35]  Kai Wang,et al.  A Benchmark for Automatic Visual Classification of Clinical Skin Disease Images , 2016, ECCV.

[36]  J. Coebergh,et al.  Trends in incidence and predictions of cutaneous melanoma across Europe up to 2015 , 2014, Journal of the European Academy of Dermatology and Venereology : JEADV.

[37]  Tim Holland-Letz,et al.  Superior skin cancer classification by the combination of human and artificial intelligence. , 2019, European journal of cancer.

[38]  Jasjit S. Suri,et al.  Computer-aided diagnosis of psoriasis skin images with HOS, texture and color features: A first comparative study of its kind , 2016, Comput. Methods Programs Biomed..

[39]  Aaron S Kesselheim,et al.  Lifecycle Regulation of Artificial Intelligence- and Machine Learning-Based Software Devices in Medicine. , 2019, JAMA.

[40]  Jean-Michel Nguyen,et al.  Use of Smartphones for Early Detection of Melanoma: Systematic Review , 2018, Journal of medical Internet research.

[41]  H. Valdez,et al.  Early quantification of systemic inflammatory-proteins predicts long-term treatment response to Tofacitinib and Etanercept: Psoriasis response predictions using blood. , 2019, The Journal of investigative dermatology.

[42]  William K. Thompson,et al.  A Machine Learning Algorithm for Identifying Atopic Dermatitis in Adults from Electronic Health Records , 2017, 2017 IEEE International Conference on Healthcare Informatics (ICHI).

[43]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[44]  A. Enk,et al.  Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images. , 2019, European journal of cancer.

[45]  D. Coomans,et al.  Alternative k-nearest neighbour rules in supervised pattern recognition : Part 1. k-Nearest neighbour classification by using alternative voting rules , 1982 .

[46]  A. Navarini,et al.  Observer‐independent assessment of psoriasis‐affected area using machine learning , 2020, Journal of the European Academy of Dermatology and Venereology : JEADV.

[47]  S. Youn,et al.  Dermatologist‐level classification of malignant lip diseases using a deep convolutional neural network , 2020, The British journal of dermatology.

[48]  P Kharazmi,et al.  A feature fusion system for basal cell carcinoma detection through data‐driven feature learning and patient profile , 2018, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[49]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[50]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[51]  Barry Peterson,et al.  Detection of Nocturnal Scratching Movements in Patients with Atopic Dermatitis Using Accelerometers and Recurrent Neural Networks , 2018, IEEE Journal of Biomedical and Health Informatics.

[52]  J. Lipoff,et al.  Natural Language Processing of Reddit Data to Evaluate Dermatology Patient Experiences and Therapeutics. , 2020, Journal of the American Academy of Dermatology.

[53]  Marti A. Hearst Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[54]  Achim Hekler,et al.  A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task. , 2019, European journal of cancer.

[55]  Hamidullah Binol,et al.  Ros‐NET: A deep convolutional neural network for automatic identification of rosacea lesions , 2019, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[56]  Ngai-Man Cheung,et al.  Deepmole: Deep neural networks for skin mole lesion classification , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[57]  T. Ruzicka,et al.  mHealth App for Risk Assessment of Pigmented and Nonpigmented Skin Lesions-A Study on Sensitivity and Specificity in Detecting Malignancy. , 2017, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[58]  Fabio A. González,et al.  An unsupervised feature learning framework for basal cell carcinoma image analysis , 2015, Artif. Intell. Medicine.

[59]  John Paoli,et al.  Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks , 2019, JAMA dermatology.

[60]  C. Vestergaard,et al.  Clinical heterogeneity and differential diagnosis of atopic dermatitis , 2014, The British journal of dermatology.

[61]  Thomas George Olsen,et al.  Diagnostic Performance of Deep Learning Algorithms Applied to Three Common Diagnoses in Dermatopathology , 2018, Journal of pathology informatics.

[62]  D. Longo,et al.  Precision medicine--personalized, problematic, and promising. , 2015, The New England journal of medicine.

[63]  T. Nijsten,et al.  Accuracy of a smartphone application for triage of skin lesions based on machine learning algorithms , 2019, Journal of the European Academy of Dermatology and Venereology : JEADV.

[64]  Shannon C. Trotter,et al.  Skin cancer screening: recommendations for data-driven screening guidelines and a review of the US Preventive Services Task Force controversy , 2017, Melanoma management.

[65]  L. Tsoi,et al.  The Molecular Revolution in Cutaneous Biology: The Era of Global Transcriptional Analysis. , 2017, The Journal of investigative dermatology.

[66]  W. Price,et al.  Potential Liability for Physicians Using Artificial Intelligence. , 2019, JAMA.

[67]  M. Abràmoff,et al.  Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices , 2018, npj Digital Medicine.

[68]  A. Gottlieb,et al.  Prevalence of psoriatic arthritis in patients with psoriasis: A systematic review and meta‐analysis of observational and clinical studies , 2019, Journal of the American Academy of Dermatology.

[69]  Macario O. Cordel,et al.  Design and Evaluation of a Multi-model, Multi-level Artificial Neural Network for Eczema Skin Lesion Detection , 2015, 2015 3rd International Conference on Artificial Intelligence, Modelling and Simulation (AIMS).

[70]  E. Szymańska,et al.  Psoriasis in special localizations , 2018, Reumatologia.

[71]  Saeed Hassanpour,et al.  Automated detection of nonmelanoma skin cancer using digital images: a systematic review , 2019, BMC Medical Imaging.

[72]  A. Fleischer,et al.  Is the concept of clinical equipoise maintained in psoriasis research? , 2020, The Journal of dermatological treatment.

[73]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[74]  R. Hofmann-Wellenhof,et al.  Association Between Surgical Skin Markings in Dermoscopic Images and Diagnostic Performance of a Deep Learning Convolutional Neural Network for Melanoma Recognition. , 2019, JAMA dermatology.

[75]  Jun Deng,et al.  Predicting non-melanoma skin cancer via a multi-parameterized artificial neural network , 2018, Scientific Reports.

[76]  Tim Holland-Letz,et al.  Deep neural networks are superior to dermatologists in melanoma image classification. , 2019, European journal of cancer.

[77]  N. Altman An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[78]  Kalpana Raja,et al.  Drug Repurposing Prediction for Immune-Mediated Cutaneous Diseases using a Word-Embedding-Based Machine Learning Approach. , 2019, The Journal of investigative dermatology.

[79]  Melissa Ward-Peterson,et al.  Association Between Race/Ethnicity and Survival of Melanoma Patients in the United States Over 3 Decades , 2016, Medicine.

[80]  Taghi M. Khoshgoftaar,et al.  Efficient learning from big data for cancer risk modeling: A case study with melanoma , 2019, Comput. Biol. Medicine.

[81]  Chia-Wei Liang,et al.  Assessment of Deep Learning Using Nonimaging Information and Sequential Medical Records to Develop a Prediction Model for Nonmelanoma Skin Cancer. , 2019, JAMA dermatology.

[82]  Russ Greiner,et al.  Predicting the Long-Term Outcomes of Biologics in Psoriasis Patients Using Machine Learning , 2019, ArXiv.

[83]  S. Han,et al.  Classification of the Clinical Images for Benign and Malignant Cutaneous Tumors Using a Deep Learning Algorithm. , 2018, The Journal of investigative dermatology.

[84]  R. Hofmann-Wellenhof,et al.  Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study. , 2019, The Lancet. Oncology.

[85]  Po-Hsuan Cameron Chen,et al.  How to Read Articles That Use Machine Learning: Users' Guides to the Medical Literature. , 2019, JAMA.

[86]  Mohammad Aldeen,et al.  Automatic Scale Severity Assessment Method in Psoriasis Skin Images Using Local Descriptors , 2020, IEEE Journal of Biomedical and Health Informatics.

[87]  Mohammad Aldeen,et al.  Psoriasis image representation using patch-based dictionary learning for erythema severity scoring , 2018, Comput. Medical Imaging Graph..

[88]  Xavier Giro-i-Nieto,et al.  Skin lesion classification from dermoscopic images using deep learning techniques , 2017, 2017 13th IASTED International Conference on Biomedical Engineering (BioMed).

[89]  Steven N. Hart,et al.  Classification of Melanocytic Lesions in Selected and Whole-Slide Images via Convolutional Neural Networks , 2019, Journal of pathology informatics.

[90]  Jasjit S. Suri,et al.  A novel and robust Bayesian approach for segmentation of psoriasis lesions and its risk stratification , 2017, Comput. Methods Programs Biomed..

[91]  Matthew T. Patrick,et al.  Genetic signature to provide robust risk assessment of psoriatic arthritis development in psoriasis patients , 2018, Nature Communications.

[92]  Joann G Elmore,et al.  Population-Based Analysis of Histologically Confirmed Melanocytic Proliferations Using Natural Language Processing , 2018, JAMA dermatology.

[93]  Robby T. Tan,et al.  Convolutional Neural Networks for Automated Fracture Detection and Localization on Wrist Radiographs. , 2019, Radiology. Artificial intelligence.

[94]  Ebata,et al.  The characteristics of nocturnal scratching in adults with atopic dermatitis , 1999, The British journal of dermatology.

[95]  S. Lodha,et al.  Discordance in the histopathologic diagnosis of difficult melanocytic neoplasms in the clinical setting , 2008, Journal of cutaneous pathology.

[96]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[97]  John G Kenny,et al.  A Framework for Multi-Omic Prediction of Treatment Response to Biologic Therapy for Psoriasis. , 2019, The Journal of investigative dermatology.

[98]  A. Kalloo,et al.  Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images , 2018, Journal of the American Academy of Dermatology.

[99]  J. Silverberg,et al.  Validation of International Classification of Disease Ninth Revision codes for atopic dermatitis , 2017, Allergy.

[100]  Grace K. Kim,et al.  Skin cancer in asians: part 1: nonmelanoma skin cancer. , 2009, The Journal of clinical and aesthetic dermatology.

[101]  C. Garbe,et al.  Epidemiology of skin cancer. , 2014, Advances in experimental medicine and biology.

[102]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[103]  Mehmet Türkan,et al.  A survey on automated melanoma detection , 2018, Eng. Appl. Artif. Intell..

[104]  Tim Holland-Letz,et al.  Pathologist-level classification of histopathological melanoma images with deep neural networks. , 2019, European journal of cancer.

[105]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[106]  J. Tilburt,et al.  Dermatopathologists' concerns and challenges with clinical information in the skin biopsy requisition form: a mixed‐methods study , 2015, Journal of cutaneous pathology.

[107]  Woohyung Lim,et al.  Deep neural networks show an equivalent and often superior performance to dermatologists in onychomycosis diagnosis: Automatic construction of onychomycosis datasets by region-based convolutional deep neural network , 2018, PloS one.

[108]  K. Griewank,et al.  Re: Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images. , 2020, European journal of cancer.

[109]  M. Weinstock,et al.  Prevalence and correlates of skin cancer screening among middle-aged and older white adults in the United States. , 2010, The American journal of medicine.