Heterogeneity-Aware Local Binary Patterns for Retrieval of Histopathology Images

Histopathology images exhibit considerable variability, which can make diagnosis prone to uncertainty and errors. Using retrieval systems to locate similar images when a query image is given can assist pathologists in making more reliable decisions when diagnosing, based on accurately diagnosed past cases. Local binary patterns (LBP) have been successfully used as image descriptors for different applications. However, using LBP on histopathology images is still under investigation from different perspectives. The immense texture variability of these images and the lack of labeled training data are among the challenges that must be addressed. In this paper, we propose a new extension of LBP that is explicitly aware of the heterogeneity of local texture patterns through heterogeneity-based weighting. We use both homogeneity and the second moment (variance) of local neighborhoods to extend LBP histograms with heterogeneity information so that they better capture the polymorphism in histopathology images. We applied all concepts at multiple scales and investigated the extensions through both separated and concatenated extended histograms. We applied the proposed method on three publicly available datasets, namely, KimiaPath24, invasive ductal carcinoma (IDC), and BreakHis. The experimental results revealed that our features could retrieve and classify images with the highest accuracy. For KimiaPath24, we achieved 96.4% surpassing both LBP (91.0%) and deep features from VGG network (79.5%). For IDC, we reached the highest $F1$ measure (0.7665). Only for BreakHis, the proposed method was slightly less accurate than a customized convolutional neural network with rule fusion.