A deep learning-based model of normal histology

Deep learning models have been applied on various tissues in order to recognize malignancies. However, these models focus on relatively narrow tissue context or well-defined pathologies. Here, instead of focusing on pathologies, we introduce models characterizing the diversity of normal tissues. We obtained 1,690 slides with rat tissue samples from the control groups of six preclinical toxicology studies, on which tissue regions were outlined and annotated by pathologists into 46 different tissue classes. From these annotated regions, we sampled small patches of 224 × 224 pixels at six different levels of magnification. Using four studies as training set and two studies as test set, we trained VGG-16, ResNet-50, and Inception-v3 networks separately at each of these magnification levels. Among these models, Inception-v3 consistently outperformed the other networks and attained accuracies up to 83.4% (top-3 accuracy: 96.3%). Further analysis showed that most tissue confusions occurred within clusters of histologically similar tissues. Investigation of the embedding layer using the UMAP method revealed not only pronounced clusters corresponding to the individual tissues, but also subclusters corresponding to histologically meaningful structures that had neither been annotated nor trained for. This suggests that the histological representation learned by the normal histology network could also be used to flag abnormal tissue as outliers in the embedding space without a need to explicitly train for specific types of abnormalities. Finally, we found that models trained on rat tissues can be used on non-human primate and minipig tissues with minimal retraining. Significance statement Like many other scientific disciplines, histopathology has been profoundly impacted by recent advances in machine learning with deep neural networks. In this field, most deep learning models reported in the literature are trained on pathologies in specific tissues/contexts. Here, we aim to establish a model of normal tissues as a foundation for future models of histopathology. We build models that are specific to histopathology images and we show that their embeddings are better feature vectors for describing the underlying images than those of off-the shelf CNN models. Therefore, our models could be useful for transfer learning to improve the accuracy of other histopathology models.