Integrating segmentation with deep learning for enhanced classification of epithelial and stromal tissues in H&E images

Abstract Initiation, progression, and therapeutic response in cancer are largely influenced by tumor microenvironment. Segmentation of tumor into epithelial vs. stromal regions constitutes the first step for the study of tumor microenvironment and its effects on disease progression. This paper proposes a new method for stromal vs. epithelial tissue identification from images of H&E stained specimens. The proposed method integrates convolutional neural networks (CNN) based supervised classification with unsupervised image segmentation. The scheme combines the strengths of deep learning (feature learning and classification) with the boundary localization accuracy of image segmentation for enhanced performance. Our experimental results on Stanford Tissue Microarray Database show that integration of CNN classification with explicit image segmentation leads to better adherence of identified class boundaries to actual tissue boundaries and improves the classification accuracy.