Spatial Statistics for Segmenting Histological Structures in H&E Stained Tissue Images

Segmenting a broad class of histological structures in transmitted light and/or fluorescence-based images is a prerequisite for determining the pathological basis of cancer, elucidating spatial interactions between histological structures in tumor microenvironments (e.g., tumor infiltrating lymphocytes), facilitating precision medicine studies with deep molecular profiling, and providing an exploratory tool for pathologists. This paper focuses on segmenting histological structures in hematoxylin- and eosin-stained images of breast tissues, e.g., invasive carcinoma, carcinoma in situ, atypical and normal ducts, adipose tissue, and lymphocytes. We propose two graph-theoretic segmentation methods based on local spatial color and nuclei neighborhood statistics. For benchmarking, we curated a data set of 232 high-power field breast tissue images together with expertly annotated ground truth. To accurately model the preference for histological structures (ducts, vessels, tumor nets, adipose, etc.) over the remaining connective tissue and non-tissue areas in ground truth annotations, we propose a new region-based score for evaluating segmentation algorithms. We demonstrate the improvement of our proposed methods over the state-of-the-art algorithms in both region- and boundary-based performance measures.

[1]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  B. S. Manjunath,et al.  Unsupervised Segmentation of Color-Texture Regions in Images and Video , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[4]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[6]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[7]  Cenk Sokmensuer,et al.  Object-oriented texture analysis for the unsupervised segmentation of biopsy images for cancer detection , 2009, Pattern Recognit..

[8]  Anant Madabhushi,et al.  Markov Random Field driven Region-Based Active Contour Model (MaRACel): Application to Medical Image Segmentation , 2010, MICCAI.

[9]  Yousef Al-Kofahi,et al.  Improved Automatic Detection and Segmentation of Cell Nuclei in Histopathology Images , 2010, IEEE Transactions on Biomedical Engineering.

[10]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Yulei Jiang,et al.  Segmentation of prostatic glands in histology images , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[12]  Cigdem Demir,et al.  Graph Run-Length Matrices for Histopathological Image Segmentation , 2011, IEEE Transactions on Medical Imaging.

[13]  Anant Madabhushi,et al.  Incorporating domain knowledge for tubule detection in breast histopathology using O'Callaghan neighborhoods , 2011, Medical Imaging.

[14]  Anant Madabhushi,et al.  Image Segmentation with Implicit Color Standardization Using Spatially Constrained Expectation Maximization: Detection of Nuclei , 2012, MICCAI.

[15]  Kanti V. Mardia,et al.  Statistics of Bivariate von Mises Distributions , 2012 .

[16]  Anindya Sarkar,et al.  Structure and Context in Prostatic Gland Segmentation and Classification , 2012, MICCAI.

[17]  Wei Wang,et al.  A flexible and robust approach for segmenting cell nuclei from 2D microscopy images using supervised learning and template matching , 2013, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[18]  Jordi Pont-Tuset,et al.  Measures and Meta-Measures for the Supervised Evaluation of Image Segmentation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Cigdem Demir,et al.  Attributed Relational Graphs for Cell Nucleus Segmentation in Fluorescence Microscopy Images , 2013, IEEE Transactions on Medical Imaging.

[20]  N. Rajpoot,et al.  HyMaP: A hybrid magnitude-phase approach to unsupervised segmentation of tumor areas in breast cancer histology images , 2013, Journal of pathology informatics.

[21]  Bih-Rong Wei,et al.  Digital pathology and image analysis augment biospecimen annotation and biobank quality assurance harmonization. , 2014, Clinical biochemistry.

[22]  Dustin G. Mixon,et al.  Images as Occlusions of Textures: A Framework for Segmentation , 2014, IEEE Transactions on Image Processing.

[23]  Jeffrey L. Fine.  21st century workflow: A proposal , 2014, Journal of pathology informatics.

[24]  Edward H. Adelson,et al.  Crisp Boundary Detection Using Pointwise Mutual Information , 2014, ECCV.

[25]  Konstantinos N. Plataniotis,et al.  A Complete Color Normalization Approach to Histopathology Images Using Color Cues Computed From Saturation-Weighted Statistics , 2015, IEEE Transactions on Biomedical Engineering.

[26]  Saining Xie,et al.  Holistically-Nested Edge Detection , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[27]  A. B. Tosun,et al.  Detection of malignant mesothelioma using nuclear structure of mesothelial cells in effusion cytology specimens , 2015, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[28]  Danny Ziyi Chen,et al.  Detection of Glands and Villi by Collaboration of Domain Knowledge and Deep Learning , 2015, MICCAI.

[29]  J. S. Marron,et al.  Appearance Normalization of Histology Slides , 2010, MLMI.

[30]  Lin Yang,et al.  A Novel Cell Detection Method Using Deep Convolutional Neural Network and Maximum-Weight Independent Set , 2015, MICCAI.

[31]  L. Wood,et al.  A robust non-linear tissue-component discrimination method for computational pathology , 2015, Laboratory Investigation.

[32]  Nico Karssemeijer,et al.  Automated Detection of DCIS in Whole-Slide H&E Stained Breast Histopathology Images , 2016, IEEE Transactions on Medical Imaging.

[33]  Nassir Navab,et al.  Structure-Preserving Color Normalization and Sparse Stain Separation for Histological Images , 2016, IEEE Transactions on Medical Imaging.