Dual Structured Convolutional Neural Network with Feature Augmentation for Quantitative Characterization of Tissue Histology

We present a dual convolutional neural network (dCNN) architecture for extracting multi-scale features from histological tissue images for the purpose of automated characterization of tissue in digital pathology. The dual structure consists of two identical convolutional neural networks applied to input images with different scales, that are merged together and stacked with two fully connected layers. It has been acknowledged that deep networks can be used to extract higher-order features, and therefore, the network output at final fully connected layer was used as a deep dCNN feature vector. Further, engineered features, shown in previous studies to capture important characteristics of tissue structure and morphology, were integrated to the feature extractor module. The acquired quantitative feature representation can be further utilized to train a discriminative model for classifying tissue types. Machine learning based methods for detection of regions of interest, or tissue type classification will advance the transition to decision support systems and computer aided diagnosis in digital pathology. Here we apply the proposed feature-augmented dCNN method with supervised learning in detecting cancerous tissue from whole slide images. The extracted quantitative representation of tissue histology was used to train a logistic regression model with elastic net regularization. The model was able to accurately discriminate cancerous tissue from normal tissue, resulting in blockwise AUC=0.97, where the total number of analyzed tissue blocks was approximately 8.3 million that constitute the test set of 75 whole slide images.

[1]  George Lee,et al.  Feature Importance in Nonlinear Embeddings (FINE): Applications in Digital Pathology , 2016, IEEE Transactions on Medical Imaging.

[2]  Max A. Viergever,et al.  Breast Cancer Histopathology Image Analysis: A Review , 2014, IEEE Transactions on Biomedical Engineering.

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  Hao Chen,et al.  Gland segmentation in colon histology images: The glas challenge contest , 2016, Medical Image Anal..

[5]  George Lee,et al.  Image analysis and machine learning in digital pathology: Challenges and opportunities , 2016, Medical Image Anal..

[6]  A. Ruifrok,et al.  Quantification of histochemical staining by color deconvolution. , 2001, Analytical and quantitative cytology and histology.

[7]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[8]  Metin Nafi Gürcan,et al.  Intraoperative neuropathology of glioma recurrence: cell detection and classification , 2016, SPIE Medical Imaging.

[9]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[10]  Angel Cruz-Roa,et al.  Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features , 2014, Journal of medical imaging.

[11]  Matti Pietikäinen,et al.  Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2000, ECCV.

[12]  Vilppu J Tuominen,et al.  ImmunoRatio: a publicly available web application for quantitative image analysis of estrogen receptor (ER), progesterone receptor (PR), and Ki-67 , 2010, Breast Cancer Research.

[13]  Dayong Wang,et al.  Deep Learning for Identifying Metastatic Breast Cancer , 2016, ArXiv.

[14]  Kaisa Liimatainen,et al.  Metastasis detection from whole slide images using local features and random forests , 2017, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[15]  M. Nykter,et al.  Feature-based analysis of mouse prostatic intraepithelial neoplasia in histological tissue sections , 2016, Journal of pathology informatics.

[16]  Yizhou Yu,et al.  Visual saliency based on multiscale deep features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Emmanuelle Gouillart,et al.  scikit-image: image processing in Python , 2014, PeerJ.

[19]  Hao Chen,et al.  Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: The LUNA16 challenge , 2016, Medical Image Anal..

[20]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  Todd H. Stokes,et al.  Pathology imaging informatics for quantitative analysis of whole-slide images , 2013, Journal of the American Medical Informatics Association : JAMIA.

[22]  Anant Madabhushi,et al.  AUTOMATED GRADING OF PROSTATE CANCER USING ARCHITECTURAL AND TEXTURAL IMAGE FEATURES , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[23]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[25]  Gengsheng Qin,et al.  continuous-scale diagnostic test Comparison of non-parametric confidence intervals for the area under the ROC curve of a , 2010 .

[26]  Gaudenz Danuser,et al.  Computer Vision in Cell Biology , 2011, Cell.

[27]  Leena Latonen,et al.  Analysis of spatial heterogeneity in normal epithelium and preneoplastic alterations in mouse prostate tumor models , 2017, Scientific Reports.

[28]  James Diamond,et al.  The use of morphological characteristics and texture analysis in the identification of tissue composition in prostatic neoplasia. , 2004, Human pathology.

[29]  Richard Chen,et al.  Identifying Metastases in Sentinel Lymph Nodes with Deep Convolutional Neural Networks , 2016, ArXiv.

[30]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[31]  Kevin W Eliceiri,et al.  NIH Image to ImageJ: 25 years of image analysis , 2012, Nature Methods.