Metric Learning in Histopathological Image Classification: Opening the Black Box

The application of machine learning techniques to histopathology images enables advances in the field, providing valuable tools that can speed up and facilitate the diagnosis process. The classification of these images is a relevant aid for physicians who have to process a large number of images in long and repetitive tasks. This work proposes the adoption of metric learning that, beyond the task of classifying images, can provide additional information able to support the decision of the classification system. In particular, triplet networks have been employed to create a representation in the embedding space that gathers together images of the same class while tending to separate images with different labels. The obtained representation shows an evident separation of the classes with the possibility of evaluating the similarity and the dissimilarity among input images according to distance criteria. The model has been tested on the BreakHis dataset, a reference and largely used dataset that collects breast cancer images with eight pathology labels and four magnification levels. Our proposed classification model achieves relevant performance on the patient level, with the advantage of providing interpretable information for the obtained results, which represent a specific feature missed by the all the recent methodologies proposed for the same purpose.

[1]  A. Topeli,et al.  Chronic critical illness in critically ill COVID-19 patients , 2023, Chronic illness.

[2]  Giosuè Lo Bosco,et al.  Deep Metric Learning for Histopathological Image Classification , 2022, 2022 IEEE Eighth International Conference on Multimedia Big Data (BigMM).

[3]  Giosuè Lo Bosco,et al.  Deep Metric Learning for Transparent Classification of Covid-19 X-Ray Images , 2022, 2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[4]  M. Ridley Explainable Artificial Intelligence (XAI) , 2022, Information Technology and Libraries.

[5]  M. Ghassemi,et al.  The false hope of current approaches to explainable artificial intelligence in health care. , 2021, The Lancet. Digital health.

[6]  Gülsüm Alicioğlu,et al.  A survey of visual analytics for Explainable Artificial Intelligence methods , 2021, Comput. Graph..

[7]  Huiyu Zhou,et al.  Magnification-independent Histopathological Image Classification with Similarity-based Multi-scale Embeddings , 2021, ArXiv.

[8]  G. Litjens,et al.  Deep learning in histopathology: the path to the clinic , 2021, Nature Medicine.

[9]  L. Torresani,et al.  A Petri Dish for Histopathology Image Analysis , 2021, AIME.

[10]  Ganapathy Krishnamurthi,et al.  A generalized deep learning framework for whole-slide image segmentation and analysis , 2020, Scientific Reports.

[11]  Brandon M. Greenwell,et al.  Interpretable Machine Learning , 2019, Hands-On Machine Learning with R.

[12]  Neofytos Dimitriou,et al.  Deep Learning for Whole Slide Image Analysis: An Overview , 2019, Front. Med..

[13]  Hasan Şakir Bilge,et al.  Deep Metric Learning: A Survey , 2019, Symmetry.

[14]  Thomas J. Fuchs,et al.  Clinical-grade computational pathology using weakly supervised deep learning on whole slide images , 2019, Nature Medicine.

[15]  Luiz Eduardo Soares de Oliveira,et al.  Multiple instance learning for histopathological breast cancer image classification , 2019, Expert Syst. Appl..

[16]  Cynthia Rudin,et al.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead , 2018, Nature Machine Intelligence.

[17]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection , 2018, J. Open Source Softw..

[18]  M. L. Fravolini,et al.  Dimensionality Reduction Strategies for CNN-Based Classification of Histopathological Images , 2018, IIMSS.

[19]  Francisco Herrera,et al.  A first study exploring the performance of the state-of-the art CNN model in the problem of breast cancer , 2018, LOPAL '18.

[20]  Kun Zhang,et al.  Classification of Breast Cancer Based on Histology Images Using Convolutional Neural Networks , 2018, IEEE Access.

[21]  Ziba Gandomkar,et al.  MuDeRN: Multi-category classification of breast histopathological image using deep residual networks , 2018, Artif. Intell. Medicine.

[22]  Kundan Kumar,et al.  Breast cancer classification of image using convolutional neural network , 2018, 2018 4th International Conference on Recent Advances in Information Technology (RAIT).

[23]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction , 2018, ArXiv.

[24]  Luiz Eduardo Soares de Oliveira,et al.  Deep features for breast cancer histopathological image classification , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[25]  Heng Huang,et al.  Supervised Intra-embedding of Fisher Vectors for Histopathology Image Classification , 2017, MICCAI.

[26]  Lilly Suriani Affendey,et al.  Classification of Histopathology Images of Breast into Benign and Malignant using a Single-layer Convolutional Neural Network , 2017, ICISPC 2017.

[27]  Jaime S. Cardoso,et al.  Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support , 2017, Lecture Notes in Computer Science.

[28]  Anne L. Martel,et al.  Transitioning Between Convolutional and Fully Connected Layers in Neural Networks , 2017, DLMIA/ML-CDS@MICCAI.

[29]  Arnav Bhavsar,et al.  Breast Cancer Histopathological Image Classification: Is Magnification Important? , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Abhijit Guha Roy,et al.  Classifying histopathology whole-slides using fusion of decisions from deep convolutional network on a collection of random multi-views at multi-magnification , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[31]  Ju Jia Zou,et al.  Adapting fisher vectors for histopathology image classification , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[32]  Yilong Yin,et al.  Deep learning model based breast cancer histopathological image classification , 2017, 2017 IEEE 2nd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA).

[33]  Juho Kannala,et al.  Deep learning for magnification independent breast cancer histopathology image classification , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[34]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[35]  Luiz Eduardo Soares de Oliveira,et al.  Breast cancer histopathological image classification using Convolutional Neural Networks , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[36]  Luiz Eduardo Soares de Oliveira,et al.  A Dataset for Breast Cancer Histopathological Image Classification , 2016, IEEE Transactions on Biomedical Engineering.

[37]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[40]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[41]  Rich Caruana,et al.  Do Deep Nets Really Need to be Deep? , 2013, NIPS.

[42]  F. Perronnin,et al.  Image Classification with the Fisher Vector: Theory and Practice , 2013, International Journal of Computer Vision.

[43]  Matti Pietikäinen,et al.  Identification of tumor epithelium and stroma in tissue microarrays using texture analysis , 2012, Diagnostic Pathology.

[44]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Nina Linder,et al.  Xanthine oxidoreductase - clinical significance in colorectal cancer and in vitro expression of the protein in human colon cancer cells. , 2009, European journal of cancer.

[46]  Andrea Torsello,et al.  Similarity-Based Pattern Recognition , 2006, Lecture Notes in Computer Science.

[47]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[48]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[49]  Giosuè Lo Bosco,et al.  Breast Cancer Histologic Grade Identification by Graph Neural Network Embeddings , 2023, IWBBIO.

[50]  Giosuè Lo Bosco,et al.  Fuzzy clustering of histopathological images using deep learning embeddings , 2021, WILF.

[51]  Artificial Intelligence in Medicine: 19th International Conference on Artificial Intelligence in Medicine, AIME 2021, Virtual Event, June 15–18, 2021, Proceedings , 2021, AIME.

[52]  Debanjana Ghosh,et al.  Breast Cancer Histopathological Image Classification Using Convolutional Neural Networks , 2021, Proceedings of International Conference on Innovations in Software Architecture and Computational Systems.

[53]  Intelligent Interactive Multimedia Systems and Services, Proceedings of 2018 Conference, KES IIMSS 2018, Gold Cost, Australia, 20-22 June 2018, Proceedings , 2019, IIMSS.

[54]  Yinan Kong,et al.  Histopathological breast-image classification with restricted Boltzmann machine along with backpropagation , 2018 .

[55]  Maxime Descoteaux,et al.  Medical Image Computing and Computer Assisted Intervention − MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part III , 2017, Lecture Notes in Computer Science.

[56]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.