Deep learning to detect lymphocytes with high phenotypic resolution in highly multiplexed fluorescence microscopy images of triple-negative breast cancer biopsies

Triple-negative breast cancer (TNBC) is an aggressive subtype of breast cancer defined by the lack of hormone receptor overexpression. TNBC patients are at a higher risk of recurrence than patients with other breast cancers. As this disease disproportionally affects young women of color, there is an urgent need to address this health inequity by improving disease detection, prognostication, and therapy guidance. Currently, TNBC patients are expected to have better prognosis if a biopsy analysis shows more tumor-infiltrating immune cells. However, immunotherapies such as PD-L1 checkpoint blockade have shown variable efficacy. Studies on the immune constituents of a tumor with high phenotypic resolution have largely been reliant on tissue-destructive methods. Immunofluorescence microscopy, which conserves the spatial distribution of cells, is traditionally limited in collecting high numbers of colocalized antibody markers, which limits the phenotypic specificity of a spatial analysis of tumor immunity. To better understand the immune landscape of TNBC, we have collected highly multiplexed immunofluorescence microscopy images of 19 TNBC samples with 20 cellular markers using an iterative staining and imaging method. Here, we show the generalizability of pre-trained convolutional neural networks to T cell segmentation in TNBC, and further improve these algorithms through fine-tuning. We demonstrate a significant improvement in sensitivity after fine-tuning with domain-specific data (p < 0.05, Mann-Whitney U-test with Bonferroni correction). Additionally, we demonstrate that augmenting the fine-tuning dataset with training images from a different pathology can significantly improves cell detection performance.