Mask2Lesion: Mask-Constrained Adversarial Skin Lesion Image Synthesis

Skin lesion segmentation is a vital task in skin cancer diagnosis and further treatment. Although deep learning based approaches have significantly improved the segmentation accuracy, these algorithms are still reliant on having a large enough dataset in order to achieve adequate results. Inspired by the immense success of generative adversarial networks (GANs), we propose a GAN-based augmentation of the original dataset in order to improve the segmentation performance. In particular, we use the segmentation masks available in the training dataset to train the Mask2Lesion model, and use the model to generate new lesion images given any arbitrary mask, which are then used to augment the original training dataset. We test Mask2Lesion augmentation on the ISBI ISIC 2017 Skin Lesion Segmentation Challenge dataset and achieve an improvement of 5.17% in the mean Dice score as compared to a model trained with only classical data augmentation techniques.

[1]  S. Feldman,et al.  Incidence Estimate of Nonmelanoma Skin Cancer (Keratinocyte Carcinomas) in the U.S. Population, 2012. , 2015, JAMA dermatology.

[2]  Hao Li,et al.  Visualizing the Loss Landscape of Neural Nets , 2017, NeurIPS.

[3]  W. Stolz,et al.  The ABCD rule of dermatoscopy. High prospective value in the diagnosis of doubtful melanocytic skin lesions. , 1994, Journal of the American Academy of Dermatology.

[4]  Ghassan Hamarneh,et al.  Melanoma Recognition via Visual Attention , 2019, IPMI.

[5]  Antonio Torralba,et al.  Generating Videos with Scene Dynamics , 2016, NIPS.

[6]  Nassir Navab,et al.  Generating Highly Realistic Images of Skin Lesions with GANs , 2018, OR 2.0/CARE/CLIP/ISIC@MICCAI.

[7]  Tanveer F. Syeda-Mahmood,et al.  Chest x-ray generation and data augmentation for cardiovascular abnormality classification , 2018, Medical Imaging.

[8]  Ming Chao,et al.  Improving Dermoscopic Image Segmentation With Enhanced Convolutional-Deconvolutional Networks , 2017, IEEE Journal of Biomedical and Health Informatics.

[9]  Hayit Greenspan,et al.  Synthetic data augmentation using GAN for improved liver lesion classification , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[10]  Matt Berseth,et al.  ISIC 2017 - Skin Lesion Analysis Towards Melanoma Detection , 2017, ArXiv.

[11]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[12]  Mohamed M. Foaud,et al.  Classification of skin lesions using transfer learning and augmentation with Alex-net , 2019, PloS one.

[13]  Hao Chen,et al.  Automated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks , 2017, IEEE Transactions on Medical Imaging.

[14]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[15]  Jorge S. Marques,et al.  A Survey of Feature Extraction in Dermoscopy Image Analysis of Skin Cancer , 2019, IEEE Journal of Biomedical and Health Informatics.

[16]  David Dagan Feng,et al.  Automatic Skin Lesion Analysis using Large-scale Dermoscopy Images and Deep Residual Networks , 2017, ArXiv.

[17]  Yading Yuan,et al.  Automatic Skin Lesion Segmentation Using Deep Fully Convolutional Networks With Jaccard Distance , 2017, IEEE Transactions on Medical Imaging.

[18]  R. MacKie,et al.  Seven‐point checklist for melanoma , 1991, Clinical and experimental dermatology.

[19]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[20]  Eduardo Valle,et al.  Skin Lesion Synthesis with Generative Adversarial Networks , 2018, OR 2.0/CARE/CLIP/ISIC@MICCAI.

[21]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[22]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[23]  Noel C. F. Codella,et al.  Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC) , 2016, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[24]  B. Thiers,et al.  The CASH (color, architecture, symmetry, and homogeneity) algorithm for dermoscopy , 2008 .

[25]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Costantino Grana,et al.  Augmenting data with GANs to segment melanoma skin lesions , 2019, Multimedia Tools and Applications.

[27]  Ghassan Hamarneh,et al.  CONVOLUTIONAL NEURAL NETWORK FOR SKIN LESION SEGMENTATION , 2018 .

[28]  Nassir Navab,et al.  MelanoGANs: High Resolution Skin Lesion Synthesis with GANs , 2018, ArXiv.

[29]  D. S. Guru,et al.  Segmentation and Classification of Skin Lesions for Disease Diagnosis , 2016, ArXiv.

[30]  Lisa Tang,et al.  Simulation of Ground-Truth Validation Data Via Physically- and Statistically-Based Warps , 2008, MICCAI.