Fusing attributes predicted via conditional GANs for improved skin lesion classification (Conference Presentation)

Skin cancer is the most commonly diagnosed cancer worldwide. It is estimated that there are over 5 million cases of skin cancer are diagnosed in the United States every year. Although less than 5% of all diagnosed skin cancers are melanoma it accounts for over 70% of skin cancer-related deaths. In the past decade, the number of melanoma cancer cases has increased by 53%. Recently, there has been significant work on segmentation and classification of skin lesions via deep learning. However, there is limited work on identifying attributes and clinically-meaningful visual skin lesion patterns from dermoscopic images. In this work, we propose to use conditional GANs for skin lesion segmentation and attribute detection and use these attributes to improve skin lesion classification. The proposed conditional GAN framework can generate segmentation and attribute masks from RGB dermoscopic images. The adversarial-image-to-image translation style architecture forces the generator to learn both local and global features. The Markovian discriminator classifies pairs of image and segmentation labels as being real or fake. Unlike previous approaches, such an architecture not only learns the mapping from dermoscopic images image to segmentation and attribute masks but also learns an optimal loss function to train such a mapping. We demonstrate that the such an approach significantly improves the Jaccard index for segmentation (with a 0.65 threshold) up to 0.893. Fusing the lesion attributes for classification of lesions yields a higher accuracy compared to those without predicted attributes.