CrossPath: Top-down, Cross Data Type, Multi-Criterion Histological Analysis by Shepherding Mixed AI Models

Data-driven AI promises support for pathologists to discover sparse tumor patterns in high-resolution histological images. However, three limitations prevent AI from being adopted into clinical practice: (i) a lack of comprehensiveness where most AI algorithms only rely on single criteria/examination; (ii) a lack of explainability where AI models work as 'black-boxes' with little transparency; (iii) a lack of integrability where it is unclear how AI can become part of pathologists' existing workflow. To address these limitations, we propose CrossPath: a brain tumor grading tool that supports top-down, cross data type, multi-criterion histological analysis, where pathologists can shepherd mixed AI models. CrossPath first uses AI to discover multiple histological criteria with H and E and Ki-67 slides based on WHO guidelines. Second, CrossPath demonstrates AI findings with multi-level explainable supportive evidence. Finally, CrossPath provides a top-down shepherding workflow to help pathologists derive an evidence-based, precise grading result. To validate CrossPath, we conducted a user study with pathologists in a local medical center. The result shows that CrossPath achieves a high level of comprehensiveness, explainability, and integrability while reducing about one-third time consumption compared to using a traditional optical microscope.

[1]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[2]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[3]  J. Tait,et al.  Challenges and opportunities. , 1996, Journal of psychiatric and mental health nursing.

[4]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[5]  L. Palma,et al.  Long-term prognosis for atypical and malignant meningiomas: a study of 71 surgical cases. , 1997, Journal of neurosurgery.

[6]  I. Shih,et al.  Ki-67 labeling index in the differential diagnosis of exaggerated placental site, placental site trophoblastic tumor, and choriocarcinoma: a double immunohistochemical staining technique using Ki-67 and Mel-CAM antibodies. , 1998, Human pathology.

[7]  Ben Shneiderman,et al.  Readings in information visualization - using vision to think , 1999 .

[8]  Anne E Carpenter,et al.  CellProfiler: image analysis software for identifying and quantifying cell phenotypes , 2006, Genome Biology.

[9]  B. Scheithauer,et al.  The 2007 WHO classification of tumours of the central nervous system , 2007, Acta Neuropathologica.

[10]  Tony J Collins,et al.  ImageJ for microscopy. , 2007, BioTechniques.

[11]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[12]  S. Torp,et al.  The significance of Ki-67/MIB-1 labeling index in human meningiomas: a literature study. , 2010, Pathology, research and practice.

[13]  V. Anari,et al.  Computer-aided detection of proliferative cells and mitosis index in immunohistichemically images of meningioma , 2010, 2010 6th Iranian Conference on Machine Vision and Image Processing.

[14]  Shu-Ching Chen,et al.  Histology Image Classification Using Supervised Classification and Multimodal Fusion , 2010, 2010 IEEE International Symposium on Multimedia.

[15]  Ullrich Köthe,et al.  Ilastik: Interactive learning and segmentation toolkit , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[16]  Kevin W Eliceiri,et al.  NIH Image to ImageJ: 25 years of image analysis , 2012, Nature Methods.

[17]  Johannes E. Schindelin,et al.  Fiji: an open-source platform for biological-image analysis , 2012, Nature Methods.

[18]  Thomas Backer-Grøndahl,et al.  The histopathological spectrum of human meningiomas. , 2012, International journal of clinical and experimental pathology.

[19]  Ellen C. Jensen* Quantitative Analysis of Histological Staining and Fluorescence Using ImageJ , 2013, Anatomical record.

[20]  B. Nahed,et al.  Radiation Treatment for WHO Grade II and III Meningiomas , 2013, Front. Oncol..

[21]  R. Joe Stanley,et al.  A fusion-based approach for uterine cervical cancer histology image classification , 2013, Comput. Medical Imaging Graph..

[22]  H. Irshad Automated mitosis detection in histopathology using morphological and multi-channel statistics features , 2013, Journal of pathology informatics.

[23]  A. Huisman,et al.  Automatic Nuclei Segmentation in H&E Stained Breast Cancer Histopathology Images , 2013, PloS one.

[24]  Mrinal K. Mandal,et al.  Toward Automatic Mitotic Cell Detection and Segmentation in Multispectral Histopathological Images , 2014, IEEE Journal of Biomedical and Health Informatics.

[25]  G. Gurda,et al.  Ki-67 Index as an Ancillary Tool in the Differential Diagnosis of Proliferative Endometrial Lesions With Secretory Change , 2014, International journal of gynecological pathology : official journal of the International Society of Gynecological Pathologists.

[26]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[27]  H. Lee,et al.  Automated image based prominent nucleoli detection , 2015, Journal of pathology informatics.

[28]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Adrien Depeursinge,et al.  Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles , 2016, Medical Image Anal..

[30]  Yi Gao,et al.  A Containerized Software System for Generation, Management, and Exploration of Features from Whole Slide Tissue Images. , 2017, Cancer research.

[31]  Olaf Hellwich,et al.  Deep convolutional neural networks for automatic classification of gastric carcinoma using whole slide images in digital histopathology , 2017, Comput. Medical Imaging Graph..

[32]  C. Chakraborty,et al.  An Advanced Deep Learning Approach for Ki-67 Stained Hotspot Detection and Proliferation Rate Scoring for Prognostic Evaluation of Breast Cancer , 2017, Scientific Reports.

[33]  Yu Zhou,et al.  An Image Analysis Resource for Cancer Research: PIIP-Pathology Image Informatics Platform for Visualization, Analysis, and Management. , 2017, Cancer research.

[34]  Andreas Holzinger,et al.  Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology , 2017, ArXiv.

[35]  Peter Bankhead,et al.  QuPath: Open source software for digital pathology image analysis , 2017, Scientific Reports.

[36]  Kevin W. Eliceiri,et al.  ImageJ2: ImageJ for the next generation of scientific image data , 2017, BMC Bioinformatics.

[37]  Aleksey Boyko,et al.  Detecting Cancer Metastases on Gigapixel Pathology Images , 2017, ArXiv.

[38]  Sanghoon Lee,et al.  The Digital Slide Archive: A Software Platform for Management, Integration, and Analysis of Histology for Cancer Research. , 2017, Cancer research.

[39]  M. Graber,et al.  Diagnosis: Interpreting the Shadows , 2017 .

[40]  Adam Finkelstein,et al.  A Mixed-Initiative Interface for Animating Static Pictures , 2018, UIST.

[41]  Alexander Rakhlin,et al.  Deep Convolutional Neural Networks for Breast Cancer Histology Image Analysis , 2018, bioRxiv.

[42]  Xiang 'Anthony' Chen,et al.  Forte: User-Driven Generative Design , 2018, CHI.

[43]  Ovidiu Daescu,et al.  Convolutional Neural Network for Histopathological Analysis of Osteosarcoma , 2017, J. Comput. Biol..

[44]  Daisuke Komura,et al.  Machine Learning Methods for Histopathological Image Analysis , 2017, Computational and structural biotechnology journal.

[45]  R. Butler Challenges and opportunities , 2018, Worldwide Hospitality and Tourism Themes.

[46]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[47]  Yongxiang Huang,et al.  Improving High Resolution Histology Image Classification with Deep Spatial Fusion Network , 2018, COMPAY/OMIA@MICCAI.

[48]  Liron Pantanowitz,et al.  Artificial Intelligence and Digital Pathology: Challenges and Opportunities , 2018, Journal of pathology informatics.

[49]  Taeil Jin,et al.  SmartManikin: Virtual Humans with Agency for Design Tools , 2019, CHI.

[50]  Martin Wattenberg,et al.  Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making , 2019, CHI.

[51]  Lin Yang,et al.  Pixel-to-Pixel Learning With Weak Supervision for Single-Stage Nucleus Recognition in Ki67 Images , 2019, IEEE Transactions on Biomedical Engineering.