SlideGraph+: Whole Slide Image Level Graphs to Predict HER2Status in Breast Cancer

Human epidermal growth factor receptor 2 (HER2) is an important prognostic and predictive factor which is overexpressed in 15-20% of breast cancer (BCa). The determination of its status is a key clinical decision making step for selection of treatment regimen and prognostication. HER2 status is evaluated using transcroptomics or immunohistochemistry (IHC) through situ hybridisation (ISH) which require additional costs and tissue burden in addition to analytical variabilities in terms of manual observational biases in scoring. In this study, we propose a novel graph neural network (GNN) based model (termed SlideGraph) to predict HER2 status directly from whole-slide images of routine Haematoxylin and Eosin (H&E) slides. The network was trained and tested on slides from The Cancer Genome Atlas (TCGA) in addition to two independent test datasets. We demonstrate that the proposed model outperforms the state-of-the-art methods with area under the ROC curve (AUC) values > 0.75 on TCGA and 0.8 on independent test sets. Our experiments show that the proposed approach can be utilised for case triaging as well as preordering diagnostic tests in a diagnostic setting. It can also be used for other weakly supervised prediction problems in computational pathology. The SlideGraph code is available at https://github.com/wenqi006/SlideGraph.

[1]  G. Hortobagyi,et al.  Mechanisms of Disease: understanding resistance to HER2-targeted therapy in human breast cancer , 2006, Nature Clinical Practice Oncology.

[2]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[3]  Joel H. Saltz,et al.  Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Fabio A. González,et al.  Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks , 2014, Medical Imaging.

[5]  J. Ross,et al.  The HER-2 receptor and breast cancer: ten years of targeted anti-HER-2 therapy and personalized medicine. , 2009, The oncologist.

[6]  Liron Pantanowitz,et al.  Artificial Intelligence and Digital Pathology: Challenges and Opportunities , 2018, Journal of pathology informatics.

[7]  Olaf Hellwich,et al.  A review of graph-based methods for image analysis in digital histopathology , 2015 .

[8]  B Weyn,et al.  Computer-assisted differential diagnosis of malignant mesothelioma based on syntactic structure analysis. , 1999, Cytometry.

[9]  Nasir M. Rajpoot,et al.  PanNuke: An Open Pan-Cancer Histology Dataset for Nuclei Instance Segmentation and Classification , 2019, ECDP.

[10]  Cigdem Demir,et al.  The cell graphs of cancer , 2004, ISMB/ECCB.

[11]  Wenqi Lu,et al.  A new nonlocal forward model for diffuse optical tomography , 2019, ArXiv.

[12]  Jakob Nikolas Kather,et al.  Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer , 2019, Nature Medicine.

[13]  Maria Gabrani,et al.  HistoCartography: A Toolkit for Graph Analytics in Digital Pathology , 2021, COMPAY@MICCAI.

[14]  Jakob Nikolas Kather,et al.  Pan-cancer image-based detection of clinically actionable genetic alterations , 2019, Nature Cancer.

[15]  Lionel Hervé,et al.  Graph- and finite element-based total variation models for the inverse problem in diffuse optical tomography , 2019, Biomedical optics express.

[16]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[17]  Paul A. Viola,et al.  Multiple Instance Boosting for Object Detection , 2005, NIPS.

[18]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[19]  Nasir Rajpoot,et al.  Capturing Cellular Topology in Multi-Gigapixel Pathology Images , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[20]  Prateek Prasanna,et al.  Feature Driven Local Cell Graph (FeDeG): Predicting Overall Survival in Early Stage Lung Cancer , 2018, MICCAI.

[21]  Thomas J. Fuchs,et al.  Clinical-grade computational pathology using weakly supervised deep learning on whole slide images , 2019, Nature Medicine.

[22]  Qi Zhang,et al.  EM-DD: An Improved Multiple-Instance Learning Technique , 2001, NIPS.

[23]  Yosef Yarden,et al.  Biology of HER2 and Its Importance in Breast Cancer , 2001, Oncology.

[24]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[25]  Christopher R'e,et al.  Machine Learning on Graphs: A Model and Comprehensive Taxonomy , 2020, ArXiv.

[26]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[27]  W. McGuire,et al.  Human breast cancer: correlation of relapse and survival with amplification of the HER-2/neu oncogene. , 1987, Science.

[28]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.

[29]  Judith M. S. Prewitt,et al.  Graphs and Grammars for Histology: An Introduction. , 1979 .

[30]  Thomas Hofmann,et al.  Multiple instance learning with generalized support vector machines , 2002, AAAI/IAAI.

[31]  Steven J. M. Jones,et al.  Comprehensive molecular portraits of human breast tumours , 2013 .

[32]  David B. A. Epstein,et al.  Novel digital signatures of tissue phenotypes for predicting distant metastasis in colorectal cancer , 2018, Scientific Reports.

[33]  Shaoqun Zeng,et al.  From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge , 2019, IEEE Transactions on Medical Imaging.

[34]  D. Ruderman,et al.  Deep learned tissue “fingerprints” classify breast cancers by ER/PR/Her2 status from H&E images , 2020, Scientific Reports.

[35]  Anant Madabhushi,et al.  Cell cluster graph for prediction of biochemical recurrence in prostate cancer patients from tissue microarrays , 2013, Medical Imaging.

[36]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[37]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Daniel Müllner,et al.  Modern hierarchical, agglomerative clustering algorithms , 2011, ArXiv.

[39]  Jin Tae Kwak,et al.  Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images , 2018, Medical Image Anal..

[40]  John M S Bartlett,et al.  Human Epidermal Growth Factor Receptor 2 Testing in Breast Cancer: American Society of Clinical Oncology/College of American Pathologists Clinical Practice Guideline Focused Update. , 2018, Archives of pathology & laboratory medicine.

[41]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[42]  Nassir Navab,et al.  Structure-Preserving Color Normalization and Sparse Stain Separation for Histological Images , 2016, IEEE Transactions on Medical Imaging.

[43]  Jeffrey H. Chuang,et al.  Deep learning trained on hematoxylin and eosin tumor region of Interest predicts HER2 status and trastuzumab treatment response in HER2+ breast cancer , 2021, Modern Pathology.

[44]  Carsten Rother,et al.  Weakly supervised discriminative localization and classification: a joint learning process , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[45]  Cigdem Demir,et al.  Augmented cell-graphs for automated cancer diagnosis , 2005, ECCB/JBI.