Melanoma Recognition in Dermoscopy Images via Aggregated Deep Convolutional Features

In this paper, we present a novel framework for dermoscopy image recognition via both a deep learning method and a local descriptor encoding strategy. Specifically, deep representations of a rescaled dermoscopy image are first extracted via a very deep residual neural network pretrained on a large natural image dataset. Then these local deep descriptors are aggregated by orderless visual statistic features based on Fisher vector (FV) encoding to build a global image representation. Finally, the FV encoded representations are used to classify melanoma images using a support vector machine with a Chi-squared kernel. Our proposed method is capable of generating more discriminative features to deal with large variations within melanoma classes, as well as small variations between melanoma and nonmelanoma classes with limited training data. Extensive experiments are performed to demonstrate the effectiveness of our proposed method. Comparisons with state-of-the-art methods show the superiority of our method using the publicly available ISBI 2016 Skin lesion challenge dataset.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Franck Marzani,et al.  Automatic differentiation of melanoma from dysplastic nevi , 2015, Comput. Medical Imaging Graph..

[3]  Reda Kasmi,et al.  Classification of malignant melanoma and benign skin lesions: implementation of automatic ABCD rule , 2016, IET Image Process..

[4]  Gerald Schaefer,et al.  Colour and contrast enhancement for improved skin lesion segmentation , 2011, Comput. Medical Imaging Graph..

[5]  Gennady Samorodnitsky,et al.  Sign Cauchy Projections and Chi-Square Kernel , 2013, NIPS.

[6]  S. Menzies,et al.  Frequency and morphologic characteristics of invasive melanomas lacking specific surface microscopic features. , 1996, Archives of dermatology.

[7]  Jana Kosecka,et al.  Deep Convolutional Features for Image Based Retrieval and Scene Categorization , 2015, ArXiv.

[8]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[9]  Feng Zhou,et al.  Segmentation, Splitting, and Classification of Overlapping Bacteria in Microscope Images for Automatic Bacterial Vaginosis Diagnosis , 2017, IEEE Journal of Biomedical and Health Informatics.

[10]  Abder-Rahman Ali,et al.  A systematic review of automated melanoma detection in dermatoscopic images and its ground truth data , 2012, Medical Imaging.

[11]  Aurora Sáez,et al.  Model-Based Classification Methods of Global Patterns in Dermoscopic Images , 2014, IEEE Transactions on Medical Imaging.

[12]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[14]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[15]  Andrew Zisserman,et al.  Efficient additive kernels via explicit feature maps , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Noel C. F. Codella,et al.  Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC) , 2016, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[17]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[18]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[19]  Gerald Schaefer,et al.  Automated color calibration method for dermoscopy images , 2011, Comput. Medical Imaging Graph..

[20]  Thomas Mensink,et al.  Image Classification with the Fisher Vector: Theory and Practice , 2013, International Journal of Computer Vision.

[21]  Randy H. Moss,et al.  A methodological approach to the classification of dermoscopy images , 2007, Comput. Medical Imaging Graph..

[22]  Yuan Yao,et al.  Discriminative Learning for Automatic Staging of Placental Maturity via Multi-layer Fisher Vector , 2015, Scientific Reports.

[23]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[24]  In-So Kweon,et al.  Multi-scale pyramid pooling for deep convolutional representation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  John R. Smith,et al.  Deep Learning, Sparse Coding, and SVM for Melanoma Recognition in Dermoscopy Images , 2015, MLMI.

[26]  Kristin J. Dana,et al.  Deep TEN: Texture Encoding Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  W. Jaschke,et al.  Automated melanoma recognition , 2001, IEEE Transactions on Medical Imaging.

[28]  Rahil Garnavi,et al.  Classification of dermoscopy patterns using deep convolutional neural networks , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[29]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[30]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[31]  Thomas Mensink,et al.  Improving the Fisher Kernel for Large-Scale Image Classification , 2010, ECCV.

[32]  Yang Li,et al.  Melanoma Classification on Dermoscopy Images Using a Neural Network Ensemble Model , 2017, IEEE Transactions on Medical Imaging.

[33]  Xudong Jiang,et al.  Multi-modal and multi-layout discriminative learning for placental maturity staging , 2017, Pattern Recognit..

[34]  Bai Ying Lei,et al.  Hybrid dermoscopy image classification framework based on deep convolutional neural network and Fisher vector , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[35]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  David Dagan Feng,et al.  Automatic melanoma detection via multi-scale lesion-biased representation and joint reverse classification , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[37]  Xudong Jiang,et al.  Aggregating Deep Convolutional Features for Melanoma Recognition in Dermoscopy Images , 2017, MLMI@MICCAI.

[38]  Shehzad Khalid,et al.  Comparative study of classification techniques used in skin lesion detection systems , 2014, 17th IEEE International Multi Topic Conference 2014.

[39]  Svetlana Lazebnik,et al.  Multi-scale Orderless Pooling of Deep Convolutional Activation Features , 2014, ECCV.

[40]  T Lee,et al.  Dullrazor®: A software approach to hair removal from images , 1997, Comput. Biol. Medicine.

[41]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[42]  Lei Wang,et al.  Encoding High Dimensional Local Features by Sparse Coding Based Fisher Vectors , 2014, NIPS.

[43]  Hao Chen,et al.  Automated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks , 2017, IEEE Transactions on Medical Imaging.

[44]  Germán Capdehourat,et al.  Toward a combined tool to assist dermatologists in melanoma detection from dermoscopic images of pigmented skin lesions , 2011, Pattern Recognit. Lett..

[45]  Eduardo Valle,et al.  Knowledge transfer for melanoma screening with deep learning , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[46]  Trevor Darrell,et al.  Do Convnets Learn Correspondence? , 2014, NIPS.

[47]  Brian C. Lovell,et al.  Fisher tensors for classifying human epithelial cells , 2014, Pattern Recognit..

[48]  Rafael García,et al.  Computerized analysis of pigmented skin lesions: A review , 2012, Artif. Intell. Medicine.

[49]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[50]  Stephen W Dusza,et al.  The CASH (color, architecture, symmetry, and homogeneity) algorithm for dermoscopy. , 2007, Journal of the American Academy of Dermatology.

[51]  Alberto Del Bimbo,et al.  Fisher Encoded Convolutional Bag-of-Windows for Efficient Image Retrieval and Social Image Tagging , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[52]  Bai Ying Lei,et al.  Accurate Segmentation of Cervical Cytoplasm and Nuclei Based on Multiscale Convolutional Network and Graph Partitioning , 2015, IEEE Transactions on Biomedical Engineering.

[53]  Andrew G. Howard,et al.  Some Improvements on Deep Convolutional Neural Network Based Image Classification , 2013, ICLR.

[54]  Masaru Tanaka,et al.  Four-Class Classification of Skin Lesions With Task Decomposition Strategy , 2015, IEEE Transactions on Biomedical Engineering.

[55]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[56]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[57]  G. Zouridakis,et al.  Malignant melanoma detection by Bag-of-Features classification , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[58]  Ronald M. Summers,et al.  Multi-label Deep Regression and Unordered Pooling for Holistic Interstitial Lung Disease Pattern Detection , 2016, MLMI@MICCAI.

[59]  Sharath Pankanti,et al.  Deep learning ensembles for melanoma recognition in dermoscopy images , 2016, IBM J. Res. Dev..

[60]  Ghassan Hamarneh,et al.  Deep features to classify skin lesions , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[61]  Qi Tian,et al.  Good Practice in CNN Feature Transfer , 2016, ArXiv.

[62]  Douglas A. Reynolds,et al.  Gaussian Mixture Models , 2018, Encyclopedia of Biometrics.

[63]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[64]  Larry S. Davis,et al.  Exploiting local features from deep networks for image retrieval , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[65]  Andrew Zisserman,et al.  The devil is in the details: an evaluation of recent feature encoding methods , 2011, BMVC.

[66]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[67]  Mei Chen,et al.  Low Dimensional Representation of Fisher Vectors for Microscopy Image Classification , 2017, IEEE Transactions on Medical Imaging.

[68]  Subhransu Maji,et al.  Deep filter banks for texture recognition and segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Jorge S. Marques,et al.  Two Systems for the Detection of Melanomas in Dermoscopy Images Using Texture and Color Features , 2014, IEEE Systems Journal.