Automated mitosis detection in histopathology based on non-gaussian modeling of complex wavelet coefficients

To diagnose breast cancer, the number of mitotic cells present in histology sections is an important indicator for examining and grading biopsy specimen. This study aims at improving the accuracy of automated mitosis detection by characterizing mitotic cells in wavelet based multi-resolution representations via a non-Gaussian modeling method. The potential mitosis candidates were decomposed into multi-scale forms by an undecimated dual-tree complex wavelet transform. Two non-Gaussian models (the generalized Gaussian distribution (GGD) and the symmetric alpha-stable (SS) distributions) were used to accurately model the heavy-tailed behavior of wavelet marginal distributions. The method was evaluated on two independent data cohorts, including the benchmark dataset (MITOS), via a support vector machine classifier. The quantitative results shows that the bivariate SS model achieved superior classification performance with the area under the curve value of 0.82 in comparison with 0.79 for bivariate GGD, 0.77 for univariate SS, 0.72 for univariate GGD, and 0.59 for Gaussian model. Since both mitotic and non-mitotic cells appear as small objects with a large variety of shapes, characterization of mitosis is a hard problem. The inter-scale dependencies of wavelet coefficients allowing extraction of salient features within the cells that are more likely to appear at all different scales were captured by the bivariate non-Gaussian models, leading to more accurate detection results. The presented automated mitosis detection method might assist pathologists in enhancing the operational efficiency and productivity as well as improving diagnostic confidence. HighlightsAn automated mitosis detection method in histopathology is presented.Non-Gaussian models are used to capture the salient features within mitotic cells.The bivariate models improve the mitosis detection performance.