Methods for Nuclei Detection, Segmentation, and Classification in Digital Histopathology: A Review—Current Status and Future Potential

Digital pathology represents one of the major evolutions in modern medicine. Pathological examinations constitute the gold standard in many medical protocols, and also play a critical and legal role in the diagnosis process. In the conventional cancer diagnosis, pathologists analyze biopsies to make diagnostic and prognostic assessments, mainly based on the cell morphology and architecture distribution. Recently, computerized methods have been rapidly evolving in the area of digital pathology, with growing applications related to nuclei detection, segmentation, and classification. In cancer research, these approaches have played, and will continue to play a key (often bottleneck) role in minimizing human intervention, consolidating pertinent second opinions, and providing traceable clinical information. Pathological studies have been conducted for numerous cancer detection and grading applications, including brain, breast, cervix, lung, and prostate cancer grading. Our study presents, discusses, and extracts the major trends from an exhaustive overview of various nuclei detection, segmentation, feature computation, and classification techniques used in histopathology imagery, specifically in hematoxylin-eosin and immunohistochemical staining protocols. This study also enables us to measure the challenges that remain, in order to reach robust analysis of whole slide images, essential high content imaging with diagnostic biomarkers and prognosis support in digital pathology.

[1]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[2]  Steven W. Zucker,et al.  Region growing: Childhood and adolescence* , 1976 .

[3]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[4]  N Otsu,et al.  An automatic threshold selection method based on discriminate and least squares criteria , 1979 .

[5]  Jean Serra,et al.  Image Analysis and Mathematical Morphology , 1983 .

[6]  I. Ellis,et al.  Pathological prognostic factors in breast cancer. I. The value of histological grade in breast cancer: experience from a large study with long-term follow-up. , 2002, Histopathology.

[7]  P H Bartels,et al.  Bayesian belief networks in quantitative histopathology. , 1992, Analytical and quantitative cytology and histology.

[8]  Olvi L. Mangasarian,et al.  Nuclear feature extraction for breast tumor diagnosis , 1993, Electronic Imaging.

[9]  Henk J. A. M. Heijmans,et al.  Mathematical Morphology: Basic Principles , 1995 .

[10]  B. Macq,et al.  Morphological feature extraction for the classification of digital images of cancerous tissues , 1996, IEEE Transactions on Biomedical Engineering.

[11]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Deborah B. Thompson,et al.  AUTOMATED LOCATION OF DYSPLASTIC FIELDS IN COLORECTAL HISTOLOGY USING IMAGE TEXTURE ANALYSIS , 1997, The Journal of pathology.

[13]  Stephen J. Roberts,et al.  Robust cell nuclei segmentation using statistical modelling , 1998 .

[14]  Fritz Albregtsen,et al.  New texture features based on the complexity curve , 1999, Pattern Recognit..

[15]  Deborah B. Thompson,et al.  An automated machine vision system for the histological grading of cervical intraepithelial neoplasia (CIN) , 2000, The Journal of pathology.

[16]  H. Fox.  Is H&E morphology coming to an end? , 2000, Journal of clinical pathology.

[17]  Jos B. T. M. Roerdink,et al.  The Watershed Transform: Definitions, Algorithms and Parallelization Strategies , 2000, Fundam. Informaticae.

[18]  A. Ruifrok,et al.  Quantification of histochemical staining by color deconvolution. , 2001, Analytical and quantitative cytology and histology.

[19]  J. Going,et al.  Scoring nuclear pleomorphism in breast cancer , 2001, Histopathology.

[20]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[21]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[22]  Haishan Wu,et al.  Image analysis and morphometry in the diagnosis of breast cancer , 2002, Microscopy research and technique.

[23]  Hubert Cardot,et al.  Cooperation of color pixel classification schemes and color watershed: a study for microscopic images , 2002, IEEE Trans. Image Process..

[24]  Kecheng Liu,et al.  Shape recovery algorithms using level sets in 2-D/3-D medical imagery: a state-of-the-art review , 2002, IEEE Transactions on Information Technology in Biomedicine.

[25]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  F J W-M Leong,et al.  Correction of uneven illumination (vignetting) in digital microscopy images , 2003, Journal of clinical pathology.

[27]  Alexander Zelinsky,et al.  Fast Radial Symmetry for Detecting Points of Interest , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Ewert Bengtsson,et al.  A Feature Set for Cytometry on Digitized Microscopic Images , 2003, Analytical cellular pathology : the journal of the European Society for Analytical Cellular Pathology.

[29]  Cigdem Demir,et al.  The cell graphs of cancer , 2004, ISMB/ECCB.

[30]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[31]  C Wählby,et al.  Combining intensity, edge and shape information for 2D and 3D segmentation of cell nuclei in tissue sections , 2004, Journal of microscopy.

[32]  Michael Stierer,et al.  Nuclear pleomorphism, a strong prognostic factor in axillary node-negative small invasive breast cancer , 1991, Breast Cancer Research and Treatment.

[33]  Peter Meer,et al.  Unsupervised segmentation based on robust estimation and color active contour models , 2005, IEEE Transactions on Information Technology in Biomedicine.

[34]  B. Yener,et al.  Automated cancer diagnosis based on histopathological images : a systematic survey , 2005 .

[35]  J. Saltz,et al.  Image Analysis for Neuroblastoma Classification: Segmentation of Cell Nuclei , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[36]  Constantine Katsinis,et al.  Large-scale computations on histology images reveal grade-differentiating parameters for breast cancer , 2006, BMC Medical Imaging.

[37]  Xiaobo Zhou,et al.  Informatics challenges of high-throughput microscopy , 2006, IEEE Signal Processing Magazine.

[38]  Qing Yang,et al.  Iterative Voting for Inference of Structural Saliency and Characterization of Subcellular Events , 2007, IEEE Transactions on Image Processing.

[39]  Gary D Marty,et al.  Blank-field correction for achieving a uniform white background in brightfield digital photomicrographs. , 2007, BioTechniques.

[40]  Gabriela Alexe,et al.  High expression of lymphocyte-associated genes in node-negative HER2+ breast cancers correlates with lower recurrence rates. , 2007, Cancer research.

[41]  Oliver Schmitt,et al.  Radial symmetries based decomposition of cell clusters in binary and gray level images , 2008, Pattern Recognit..

[42]  John Meyer,et al.  Grading nuclear pleomorphism on histological micrographs , 2008, 2008 19th International Conference on Pattern Recognition.

[43]  Adel Hafiane,et al.  Clustering initiated multiphase active contours and robust separation of nuclei groups for tissue segmentation , 2008, 2008 19th International Conference on Pattern Recognition.

[44]  Xianghua Xie,et al.  Correction to "MAC: Magnetostatic Active Contour Model" , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Qingmin Liao,et al.  A novel shape prior based segmentation of touching or overlapping ellipse-like nuclei , 2008, SPIE Medical Imaging.

[46]  B. S. Manjunath,et al.  A biosegmentation benchmark for evaluation of bioimage analysis methods , 2009, BMC Bioinformatics.

[47]  F. Cloppet,et al.  Segmentation of overlapping/aggregating nuclei cells in biological images , 2008, 2008 19th International Conference on Pattern Recognition.

[48]  B. S. Manjunath,et al.  Object- and spatial-level quantitative analysis of multispectral histopathology images for detection and characterization of cancer , 2008 .

[49]  Fiona Ginty,et al.  Multi-modal imaging of histological tissue sections , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[50]  Vladimir Cretu,et al.  Knowledge-Guided Semantic Indexing of Breast Cancer Histopathology Images , 2008, 2008 International Conference on BioMedical Engineering and Informatics.

[51]  Joel H. Saltz,et al.  Ieee Transactions on Visualization and Computer Graphics Reconstruction of Cellular Biological Structures from Optical Microscopy Data , 2022 .

[52]  Lin Yang,et al.  Automatic Image Analysis of Histopathology Specimens Using Concave Vertex Graph , 2008, MICCAI.

[53]  Andrew A. Renshaw,et al.  Rubin??s Pathology. Clinicopathologic Foundations of Medicine , 2008 .

[54]  Abderrahim Elmoataz,et al.  Graph-based tools for microscopic cellular image segmentation , 2009, Pattern Recognit..

[55]  David Manset,et al.  XML-based approaches for the integration of heterogeneous bio-molecular data , 2009, BMC Bioinformatics.

[56]  Gyan Bhanot,et al.  Expectation–Maximization-Driven Geodesic Active Contour With Overlap Resolution (EMaGACOR): Application to Lymphocyte Segmentation on Breast Cancer Histopathology , 2010, IEEE Transactions on Biomedical Engineering.

[57]  W. Ma,et al.  Novel Agents on the Horizon for Cancer Therapy , 2009, CA: a cancer journal for clinicians.

[58]  Derek R. Magee,et al.  Colour Normalisation in Digital Histopathology Images , 2009 .

[59]  Chao-Hui Huang,et al.  Nuclear pleomorphism scoring by selective cell nuclei detection , 2009, WACV.

[60]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[61]  O. Sertel,et al.  Computer-aided Prognosis of Neuroblastoma: Detection of mitosis and karyorrhexis cells in digitized histological images , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[62]  J. S. Marron,et al.  A method for normalizing histology slides for quantitative analysis , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[63]  Po-Whei Huang,et al.  Effective segmentation and classification for HCC biopsy images , 2010, Pattern Recognit..

[64]  Yousef Al-Kofahi,et al.  Improved Automatic Detection and Segmentation of Cell Nuclei in Histopathology Images , 2010, IEEE Transactions on Biomedical Engineering.

[65]  Chanho Jung,et al.  Segmenting Clustered Nuclei Using H-minima Transform-Based Marker Extraction and Contour Parameterization , 2010, IEEE Transactions on Biomedical Engineering.

[66]  Chanho Jung,et al.  Unsupervised Segmentation of Overlapped Nuclei Using Bayesian Classification , 2010, IEEE Transactions on Biomedical Engineering.

[67]  Gyan Bhanot,et al.  Computerized Image-Based Detection and Grading of Lymphocytic Infiltration in HER2+ Breast Cancer Histopathology , 2010, IEEE Transactions on Biomedical Engineering.

[68]  Metin Nafi Gürcan,et al.  Computer-Aided Detection of Centroblasts for Follicular Lymphoma Grading Using Adaptive Likelihood-Based Cell Segmentation , 2010, IEEE Transactions on Biomedical Engineering.

[69]  Omar S. Al-Kadi,et al.  Texture measures combination for improved meningioma classification of histopathological images , 2010, Pattern Recognit..

[70]  V. Anari,et al.  Computer-aided detection of proliferative cells and mitosis index in immunohistichemically images of meningioma , 2010, 2010 6th Iranian Conference on Machine Vision and Image Processing.

[71]  Mike May,et al.  A better lens on disease. , 2010, Scientific American.

[72]  Enrico Macii,et al.  Automated segmentation of tissue images for computerized IHC analysis , 2022 .

[73]  Hui Kong,et al.  Partitioning Histopathological Images: An Integrated Framework for Supervised Color-Texture Segmentation and Cell Splitting , 2011, IEEE Transactions on Medical Imaging.

[74]  A. Mouelhi,et al.  Automatic segmentation of clustered breast cancer cells using watershed and concave vertex graph , 2011, 2011 International Conference on Communications, Computing and Control Applications (CCCA).

[75]  Abderrahim Elmoataz,et al.  Multi-resolution graph-based analysis of histopathological whole slide images: Application to mitotic cell extraction and visualization , 2011, Comput. Medical Imaging Graph..

[76]  Song Liu,et al.  Features for cells and nuclei classification , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[77]  Derek R. Magee,et al.  Context Aware Colour Classification in Digital Microscopy , 2011, MIUA.

[78]  Max A. Viergever,et al.  Marker-controlled watershed segmentation of nuclei in H&E stained breast cancer biopsy images , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[79]  Anant Madabhushi,et al.  Adaptive Energy Selective Active Contour with Shape Priors for Nuclear Segmentation and Gleason Grading of Prostate Cancer , 2011, MICCAI.

[80]  Alessandro Bevilacqua,et al.  Illumination field estimation through background detection in optical microscopy , 2011, 2011 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB).

[81]  Todd H. Stokes,et al.  Automatic batch-invariant color segmentation of histological cancer images , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[82]  Bahram Parvin,et al.  NUCLEAR SEGMENTATION IN H & E SECTIONS VIA MULTI-REFERENCE GRAPH CUT ( MRGC ) , 2011 .

[83]  Tahsin Kurc,et al.  Towards building computerized image analysis framework for nucleus discrimination in microscopy images of diffuse glioma , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[84]  Metin Nafi Gürcan,et al.  Computerized classification of intraductal breast lesions using histopathological images , 2011, IEEE Transactions on Biomedical Engineering.

[85]  Anil K. Jain,et al.  Prostate cancer detection: Fusion of cytological and textural features , 2011, Journal of pathology informatics.

[86]  Nicolas Loménie,et al.  Time-efficient sparse analysis of histopathological whole slide images , 2011, Comput. Medical Imaging Graph..

[87]  Joachim M. Buhmann,et al.  Computational Pathology: Challenges and Promises for Tissue Analysis , 2015, Comput. Medical Imaging Graph..

[88]  F Piccinini,et al.  Multi‐image based method to correct vignetting effect in light microscopy images , 2012, Journal of microscopy.

[89]  Lin Yang,et al.  Robust Segmentation of Overlapping Cells in Histopathology Specimens Using Parallel Seed Detection and Repulsive Level Set , 2012, IEEE Transactions on Biomedical Engineering.

[90]  Nicolas Loménie,et al.  Point set morphological filtering and semantic spatial configuration modeling: Application to microscopic image and bio-structure analysis , 2012, Pattern Recognit..

[91]  Daniel Heim,et al.  Detection and Segmentation of Cell Nuclei in Virtual Microscopy Images: A Minimum-Model Approach , 2012, Scientific Reports.

[92]  Anant Madabhushi,et al.  Image Segmentation with Implicit Color Standardization Using Spatially Constrained Expectation Maximization: Detection of Nuclei , 2012, MICCAI.

[93]  Akira Saito,et al.  Mitotic Figure Recognition: Agreement among Pathologists and Computerized Detector , 2011, Analytical cellular pathology.

[94]  Anant Madabhushi,et al.  An Integrated Region-, Boundary-, Shape-Based Active Contour for Multiple Object Overlap Resolution in Histological Imagery , 2012, IEEE Transactions on Medical Imaging.

[95]  Nasir M. Rajpoot,et al.  A gamma-gaussian mixture model for detection of mitotic cells in breast cancer histopathology images , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[96]  Daniel Racoceanu,et al.  Nuclei extraction from histopathological images using a marked point process approach , 2012, Medical Imaging.

[97]  J P Vink,et al.  Efficient nucleus detector in histopathology images , 2013, Journal of microscopy.

[98]  Maria S. Kulikova,et al.  Mitosis detection in breast cancer histological images An ICPR 2012 contest , 2013, Journal of pathology informatics.

[99]  Daniel Racoceanu,et al.  Cell nuclei extraction from breast cancer histopathologyimages using colour, texture, scale and shape information , 2013, Diagnostic Pathology.

[100]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[101]  Humayun Irshad,et al.  Automated mitosis detection using texture, SIFT features and HMAX biologically inspired approach , 2013, Journal of pathology informatics.

[102]  H. Irshad.  Automated mitosis detection in histopathology using morphological and multi-channel statistics features , 2013, Journal of pathology informatics.

[103]  Daniel Racoceanu,et al.  Multi-channels statistical and morphological features based mitosis detection in breast cancer histopathology , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[104]  Eric Cosatto,et al.  Classification of mitotic figures with convolutional neural networks and seeded blob features , 2013, Journal of pathology informatics.

[105]  Mohammad Ilyas,et al.  Segmenting overlapping cell nuclei in digital histopathology images , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[106]  N. Rajpoot,et al.  HyMaP: A hybrid magnitude-phase approach to unsupervised segmentation of tumor areas in breast cancer histology images , 2013, Journal of pathology informatics.

[107]  J. S. Marron,et al.  Appearance Normalization of Histology Slides , 2010, MLMI.

[108]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.