Multimodal image analysis in tissue diagnostics for skin melanoma

Early diagnosis is a corner stone for a successful treatment of most diseases including melanoma, which cannot be achieved by traditional histopathological inspection. In this respect, multimodal imaging, the combination of TPEF and SHG, features a high diagnostic potential as an alternative approach. Multimodal imaging generates molecular contrast, but to use this technique in clinical practice, the optical signals must be translated into diagnostic relevant information. This translation requires automatic image analysis techniques. Within this contribution, we established an analysis pipeline for multimodal images to achieve melanoma diagnostics of skin tissue. The first step of the image analysis was the pre‐treatment, where the mosaicking artifacts were corrected and a standardization was performed. Afterwards, the local histogram‐based first‐order texture features and the local gray‐level co‐occurrence matrix (GLCM) texture features were extracted in multiple scales. Thereafter, we constructed a local hierarchical statistical model to distinguish melanoma, normal epithelium, and other tissue types. The results demonstrated the capability of multimodal imaging combined with image analysis to differentiate different tissue types. Furthermore, we compared the histogram and the GLCM‐based texture feature sets according to the Fisher's discriminant ratio (FDR) and the prediction of the classification, which demonstrated that the histogram‐based texture features are superior to the GLCM features for the given task. Finally, we performed a global classification to achieve a patient diagnostics with the clinical diagnosis as ground truth. The agreement of the prediction and the clinical results demonstrated the great potential of multimodal imaging for melanoma diagnostics.

[1]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[2]  Watt W Webb,et al.  Two-photon fluorescence spectroscopy and microscopy of NAD(P)H and flavoprotein. , 2002, Biophysical journal.

[3]  P. Lambin,et al.  Robust Radiomics Feature Quantification Using Semiautomatic Volumetric Segmentation , 2014, PloS one.

[4]  V. de Giorgi,et al.  Combined non‐linear laser imaging (two‐photon excitation fluorescence microscopy, fluorescence lifetime imaging microscopy, multispectral multiphoton microscopy) in cutaneous tumours: first experiences , 2009, Journal of the European Academy of Dermatology and Venereology : JEADV.

[5]  Patrick Haffner,et al.  Support vector machines for histogram-based image classification , 1999, IEEE Trans. Neural Networks.

[6]  Jürgen Popp,et al.  Common mistakes in cross-validating classification models , 2017 .

[7]  Francesco Bianconi,et al.  Multi-class texture analysis in colorectal cancer histology , 2016, Scientific Reports.

[8]  Xianghua Xie,et al.  Handbook of Texture Analysis , 2008 .

[9]  Nadine Vogler,et al.  Correction of mosaicking artefacts in multimodal images caused by uneven illumination , 2017 .

[10]  Matti Pietikäinen,et al.  Rotation Invariant Image Description with Local Binary Pattern Histogram Fourier Features , 2009, SCIA.

[11]  D. R. Cousens,et al.  SNIP, A STATISTICS-SENSITIVE BACKGROUND TREATMENT FOR THE QUANTITATIVE-ANALYSIS OF PIXE SPECTRA IN GEOSCIENCE APPLICATIONS , 1988 .

[12]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[13]  Andreas Volkmer,et al.  Femtosecond Two‐photon Excited Fluorescence of Melanin* , 1999, Photochemistry and photobiology.

[14]  M. Kashani-Sabet,et al.  Discordance in the histopathologic diagnosis of melanoma at a melanoma referral center. , 2010, Journal of the American Academy of Dermatology.

[15]  Jürgen Popp,et al.  Ex-vivo and In-vivo Optical Molecular Pathology: Popp/Ex-vivo and In-vivo Optical Molecular Pathology , 2014 .

[16]  J. Popp,et al.  Raman Based Molecular Imaging and Analytics: A Magic Bullet for Biomedical Applications!? , 2016, Analytical chemistry.

[17]  Ines Latka,et al.  Towards multimodal nonlinear optical tomography – experimental methodology , 2011 .

[18]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[19]  X. Xie,et al.  Rapid, Label-Free Detection of Brain Tumors with Stimulated Raman Scattering Microscopy , 2013, Science Translational Medicine.

[20]  Mortazavi,et al.  Supporting Online Material Materials and Methods Figs. S1 to S13 Tables S1 to S3 References Label-free Biomedical Imaging with High Sensitivity by Stimulated Raman Scattering Microscopy , 2022 .

[21]  B. Dietzek,et al.  Detection and Discrimination of Non-Melanoma Skin Cancer by Multimodal Imaging , 2013, Healthcare.

[22]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[23]  T Meyer,et al.  Seamless stitching of tile scan microscope images , 2015, Journal of microscopy.

[24]  Jürgen Popp,et al.  Multimodal nonlinear microscopy of head and neck carcinoma — toward surgery assisting frozen section analysis , 2016, Head & neck.

[25]  R. Peterson,et al.  Interjudge Agreement and the Maximum Value of Kappa , 1989 .

[26]  Stephen T. C. Wong,et al.  Multimodal non-linear optical imaging for label-free differentiation of lung cancerous lesions from normal and desmoplastic tissues. , 2013, Biomedical optics express.

[27]  Andrew A. Renshaw,et al.  Rubin??s Pathology. Clinicopathologic Foundations of Medicine , 2008 .

[28]  Jürgen Popp,et al.  Pseudo-HE images derived from CARS/TPEF/SHG multimodal imaging in combination with Raman-spectroscopy as a pathological screening tool , 2016, BMC Cancer.

[29]  W. Webb,et al.  Live tissue intrinsic emission microscopy using multiphoton-excited native fluorescence and second harmonic generation , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Jürgen Popp,et al.  Nonlinear microscopy, infrared, and Raman microspectroscopy for brain tumor analysis. , 2011, Journal of biomedical optics.

[31]  David J. Hawkes,et al.  Automated Classification of Breast Cancer Stroma Maturity From Histological Images , 2017, IEEE Transactions on Biomedical Engineering.

[32]  A. Pena,et al.  Spectroscopic analysis of keratin endogenous signal for skin multiphoton microscopy. , 2005, Optics express.

[33]  Jürgen Popp,et al.  Perspectives, potentials and trends of ex vivo and in vivo optical molecular pathology , 2018, Journal of biophotonics.

[34]  Jürgen Popp,et al.  Multimodal nonlinear microscopic investigations on head and neck squamous cell carcinoma: Toward intraoperative imaging , 2013, Head & neck.

[35]  I. Riemann,et al.  In vivo Drug Screening in Human Skin Using Femtosecond Laser Multiphoton Tomography , 2006, Skin Pharmacology and Physiology.

[36]  Jürgen Popp,et al.  Beyond endoscopic assessment in inflammatory bowel disease: real-time histology of disease activity by non-linear multimodal imaging , 2016, Scientific Reports.

[37]  E. Raso,et al.  Genetic progression of malignant melanoma , 2016, Cancer and Metastasis Reviews.

[38]  Christopher Thrasivoulou,et al.  Second harmonic generation confocal microscopy of collagen type I from rat tendon cryosections. , 2006, Biophysical journal.

[39]  N. Bendsøe,et al.  Multimodal imaging to study the morphochemistry of basal cell carcinoma , 2010, Journal of biophotonics.

[40]  Ji-Xin Cheng,et al.  Label-free molecular imaging of atherosclerotic lesions using multimodal nonlinear optical microscopy. , 2007, Journal of biomedical optics.

[41]  Irene Georgakoudi,et al.  Optical imaging using endogenous contrast to assess metabolic state. , 2012, Annual review of biomedical engineering.

[42]  Hans C Gerritsen,et al.  Advances and challenges in label-free nonlinear optical imaging using two-photon excitation fluorescence and second harmonic generation for cancer research. , 2014, Journal of photochemistry and photobiology. B, Biology.

[43]  Riccardo Cicchi,et al.  Non-linear fluorescence lifetime imaging of biological tissues , 2011, Analytical and bioanalytical chemistry.

[44]  Jürgen Popp,et al.  Multimodal Imaging Spectroscopy of Tissue. , 2015, Annual review of analytical chemistry.

[45]  C H A N G M I N G S U N,et al.  Mosaicing of microscope images with global geometric and radiometric corrections , 2005 .