Delineation of Skin Strata in Reflectance Confocal Microscopy Images with Recurrent Convolutional Networks

Reflectance confocal microscopy (RCM) is an effective, non-invasive pre-screening tool for cancer diagnosis. However, acquiring and reading RCM images requires extensive training and experience, and novice clinicians exhibit high variance in diagnostic accuracy. Consequently, there is a compelling need for quantitative tools to standardize image acquisition and analysis. In this study, we use deep recurrent convolutional neural networks to delineate skin strata in stacks of RCM images collected at consecutive depths. To perform diagnostic analysis, clinicians collect RCM images at 4-5 specific layers in the tissue. Our model automates this process by discriminating between RCM images of different layers. Testing our model on an expert labeled dataset of 504 RCM stacks, we achieve 87.97% classification accuracy, and a 9-fold reduction in the number of anatomically impossible errors compared to the previous state-of-the-art.

[1]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[4]  Trygve Randen,et al.  Filtering for Texture Classification: A Comparative Study , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[6]  Jennifer G. Dy,et al.  Pilot study of semiautomated localization of the dermal/epidermal junction in reflectance confocal microscopy images of skin. , 2011, Journal of biomedical optics.

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[9]  Erich Elsen,et al.  Deep Speech: Scaling up end-to-end speech recognition , 2014, ArXiv.

[10]  Gabriela Oana Cula,et al.  Automatic Localization of Skin Layers in Reflectance Confocal Microscopy , 2014, ICIAR.

[11]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[12]  Andrew P. Bradley,et al.  Automated segmentation of skin strata in reflectance confocal microscopy depth stacks , 2015, bioRxiv.

[13]  Jennifer G. Dy,et al.  Automated delineation of dermal-epidermal junction in reflectance confocal microscopy image stacks of human skin. , 2015, The Journal of investigative dermatology.

[14]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Kristin J. Dana,et al.  Hybrid deep learning for Reflectance Confocal Microscopy skin images , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[17]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  C. Longo,et al.  Cost–benefit of reflectance confocal microscopy in the diagnostic performance of melanoma , 2016, Journal of the European Academy of Dermatology and Venereology : JEADV.

[20]  Chong Wang,et al.  Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin , 2015, ICML.

[21]  A. Lallas,et al.  Clinical Indications for Use of Reflectance Confocal Microscopy for Skin Cancer Diagnosis. , 2016, JAMA dermatology.

[22]  A. Marghoob,et al.  Reflectance confocal microscopy of skin in vivo: From bench to bedside , 2017, Lasers in surgery and medicine.

[23]  Michael I. Jordan,et al.  A Marked Poisson Process Driven Latent Shape Model for 3D Segmentation of Reflectance Confocal Microscopy Image Stacks of Human Skin , 2017, IEEE Transactions on Image Processing.