Identification of Melanoma From Hyperspectral Pathology Image Using 3D Convolutional Networks

Skin biopsy histopathological analysis is one of the primary methods used for pathologists to assess the presence and deterioration of melanoma in clinical. A comprehensive and reliable pathological analysis is the result of correctly segmented melanoma and its interaction with benign tissues, and therefore providing accurate therapy. In this study, we applied the deep convolution network on the hyperspectral pathology images to perform the segmentation of melanoma. To make the best use of spectral properties of three dimensional hyperspectral data, we proposed a 3D fully convolutional network named Hyper-net to segment melanoma from hyperspectral pathology images. In order to enhance the sensitivity of the model, we made a specific modification to the loss function with caution of false negative in diagnosis. The performance of Hyper-net surpassed the 2D model with the accuracy over 92%. The false negative rate decreased by nearly 66% using Hyper-net with the modified loss function. These findings demonstrated the ability of the Hyper-net for assisting pathologists in diagnosis of melanoma based on hyperspectral pathology images.

[1]  Zvi Malik,et al.  Chromatin Condensation in Erythropoiesis Resolved by Multipixel Spectral Imaging: Differentiation Versus Apoptosis , 1997, The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society.

[2]  R. Ornberg,et al.  Analysis of Stained Objects in Histological Sections by Spectral Imaging and Differential Absorption , 1999, The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society.

[3]  David A. Landgrebe,et al.  Hyperspectral image data analysis , 2002, IEEE Signal Process. Mag..

[4]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..

[5]  Robert I. Damper,et al.  Band Selection for Hyperspectral Image Classification Using Mutual Information , 2006, IEEE Geoscience and Remote Sensing Letters.

[6]  Riccardo Bono,et al.  Dermoscopic evaluation of amelanotic and hypomelanotic melanoma. , 2008, Archives of dermatology.

[7]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[8]  Jon Atli Benediktsson,et al.  Advances in Spectral-Spatial Classification of Hyperspectral Images , 2013, Proceedings of the IEEE.

[9]  Dan Savastru,et al.  Hyperspectral Imaging in the Medical Field: Present and Future , 2014 .

[10]  Guolan Lu,et al.  Medical hyperspectral imaging: a review , 2014, Journal of biomedical optics.

[11]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[12]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[13]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[14]  Mei Zhou,et al.  Red Blood Cell Count Automation Using Microscopic Hyperspectral Imaging Technology , 2015, Applied spectroscopy.

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Dongsheng Wang,et al.  A Minimum Spanning Forest-Based Method for Noninvasive Cancer Detection With Hyperspectral Imaging , 2016, IEEE Transactions on Biomedical Engineering.

[17]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[18]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[20]  Weisi Lin,et al.  Learning ECOC Code Matrix for Multiclass Classification with Application to Glaucoma Diagnosis , 2016, Journal of Medical Systems.

[21]  Lin Yang,et al.  An Automatic Learning-Based Framework for Robust Nucleus Segmentation , 2016, IEEE Transactions on Medical Imaging.

[22]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Guolan Lu,et al.  Deep convolutional neural networks for classifying head and neck cancer using hyperspectral imaging , 2017, Journal of biomedical optics.

[24]  D. Rigel,et al.  Analysis of Trends in US Melanoma Incidence and Mortality , 2017, JAMA dermatology.

[25]  Thomas A. Funkhouser,et al.  Dilated Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  M A Weinstock,et al.  The global burden of melanoma: results from the Global Burden of Disease Study 2015 , 2017, The British journal of dermatology.

[27]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[28]  Yading Yuan,et al.  Automatic Skin Lesion Segmentation Using Deep Fully Convolutional Networks With Jaccard Distance , 2017, IEEE Transactions on Medical Imaging.

[29]  Noel E. O'Connor,et al.  A Deep Residual Architecture for Skin Lesion Segmentation , 2018, OR 2.0/CARE/CLIP/ISIC@MICCAI.

[30]  Vijayan K. Asari,et al.  Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation , 2018, ArXiv.

[31]  Geert J. S. Litjens,et al.  Automatic segmentation of histopathological slides of renal tissue using deep learning , 2018, Medical Imaging.

[32]  Amy Y. Chen,et al.  Detection and delineation of squamous neoplasia with hyperspectral imaging in a mouse model of tongue carcinogenesis , 2018, Journal of biophotonics.

[33]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[34]  Mun-Taek Choi,et al.  Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks , 2018, Comput. Methods Programs Biomed..

[35]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[36]  Klaus H. Maier-Hein,et al.  nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation , 2018, Bildverarbeitung für die Medizin.

[37]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[38]  Andreas K. Maier,et al.  SkinNet: A Deep Learning Framework for Skin Lesion Segmentation , 2018, 2018 IEEE Nuclear Science Symposium and Medical Imaging Conference Proceedings (NSS/MIC).

[39]  Wei Li,et al.  Diverse Region-Based CNN for Hyperspectral Image Classification , 2018, IEEE Transactions on Image Processing.

[40]  Matthew B. Blaschko,et al.  The Lovasz-Softmax Loss: A Tractable Surrogate for the Optimization of the Intersection-Over-Union Measure in Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Keisuke Nemoto,et al.  Effective Use of Dilated Convolutions for Segmenting Small Object Instances in Remote Sensing Imagery , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[42]  Mei Zhou,et al.  Melanoma and Melanocyte Identification from Hyperspectral Pathology Images Using Object-Based Multiscale Analysis , 2018, Applied spectroscopy.

[43]  Jingyi Zhao,et al.  DeepLab-Based Spatial Feature Extraction for Hyperspectral Image Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[44]  Samuel Ortega,et al.  Hyperspectral imaging for head and neck cancer detection: specular glare and variance of the tumor margin in surgical specimens , 2019, Journal of medical imaging.

[45]  Ruichen Rong,et al.  Lesion Attributes Segmentation for Melanoma Detection with Multi-Task U-Net , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[46]  Haokui Zhang,et al.  Hyperspectral Image Classification Based on 3-D Separable ResNet and Transfer Learning , 2019, IEEE Geoscience and Remote Sensing Letters.

[47]  Kay R. J. Oskal,et al.  A U-net based approach to epidermal tissue segmentation in whole slide histopathological images , 2019, SN Applied Sciences.

[48]  Gustavo Marrero Callicó,et al.  Deep Learning-Based Framework for In Vivo Identification of Glioblastoma Tumor using Hyperspectral Images of Human Brain , 2019, Sensors.

[49]  Shihong Du,et al.  Multi-Scale Dense Networks for Hyperspectral Remote Sensing Image Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Ghassan Hamarneh,et al.  Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation , 2018, Comput. Medical Imaging Graph..

[51]  David Dagan Feng,et al.  Step-wise integration of deep class-specific learning for dermoscopic image segmentation , 2019, Pattern Recognit..

[52]  Xiaoqing Zhang,et al.  A hyperspectral image classification algorithm based on atrous convolution , 2019, EURASIP J. Wirel. Commun. Netw..

[53]  Yeqi Bai,et al.  Automated brain histology classification using machine learning , 2019, Journal of Clinical Neuroscience.

[54]  Jochen Lang,et al.  Segmentation of Prognostic Tissue Structures in Cutaneous Melanoma Using Whole Slide Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[55]  Md Zahangir Alom,et al.  Recurrent residual U-Net for medical image segmentation , 2019, Journal of medical imaging.

[56]  Ning Zhang,et al.  CoinNet: Copy Initialization Network for Multispectral Imagery Semantic Segmentation , 2019, IEEE Geoscience and Remote Sensing Letters.

[57]  Enrico Magli,et al.  Learning and Adapting Robust Features for Satellite Image Segmentation on Heterogeneous Data Sets , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[58]  Lipo Wang,et al.  Image Thresholding Improves 3-Dimensional Convolutional Neural Network Diagnosis of Different Acute Brain Hemorrhages on Computed Tomography Scans , 2019, Sensors.

[59]  Dinggang Shen,et al.  High-Resolution Encoder–Decoder Networks for Low-Contrast Medical Image Segmentation , 2020, IEEE Transactions on Image Processing.

[60]  Mohammad Sohel Rahman,et al.  MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation , 2019, Neural Networks.

[61]  Feilong Cao,et al.  Deep hybrid dilated residual networks for hyperspectral image classification , 2020, Neurocomputing.

[62]  Xiangtao Zheng,et al.  Spectral–Spatial Attention Network for Hyperspectral Image Classification , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[63]  Hongmin Gao,et al.  Multiscale 3-D-CNN based on spatial–spectral joint feature extraction for hyperspectral remote sensing images classification , 2020, J. Electronic Imaging.