CellSeg: a robust, pre-trained nucleus segmentation and pixel quantification software for highly multiplexed fluorescence images

Background Algorithmic cellular segmentation is an essential step for the quantitative analysis of highly multiplexed tissue images. Current segmentation pipelines often require manual dataset annotation and additional training, significant parameter tuning, or a sophisticated understanding of programming to adapt the software to the researcher’s need. Here, we present CellSeg, an open-source, pre-trained nucleus segmentation and signal quantification software based on the Mask region-convolutional neural network (R-CNN) architecture. CellSeg is accessible to users with a wide range of programming skills. Results CellSeg performs at the level of top segmentation algorithms in the 2018 Kaggle Data Challenge both qualitatively and quantitatively and generalizes well to a diverse set of multiplexed imaged cancer tissues compared to established state-of-the-art segmentation algorithms. Automated segmentation post-processing steps in the CellSeg pipeline improve the resolution of immune cell populations for downstream single-cell analysis. Finally, an application of CellSeg to a highly multiplexed colorectal cancer dataset acquired on the CO-Detection by indEXing (CODEX) platform demonstrates that CellSeg can be integrated into a multiplexed tissue imaging pipeline and lead to accurate identification of validated cell populations. Conclusion CellSeg is a robust cell segmentation software for analyzing highly multiplexed tissue images, accessible to biology researchers of any programming skill level.

[1]  Salil S. Bhate,et al.  Deep Profiling of Mouse Splenic Architecture with CODEX Multiplexed Imaging , 2017, Cell.

[2]  Bernd Bodenmiller,et al.  Automatic single cell segmentation on highly multiplexed tissue images , 2015, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[3]  N. Greenwald,et al.  DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes , 2021, Nature methods.

[4]  R. Irizarry.  ggplot2 , 2019, Introduction to Data Science.

[5]  Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. , 2021, Nature biotechnology.

[6]  W. Schubert,et al.  Analyzing proteome topology and function by automated multidimensional fluorescence microscopy , 2006, Nature Biotechnology.

[7]  G. Nolan,et al.  Highly Multiplexed Phenotyping of Immunoregulatory Proteins in the Tumor Microenvironment by CODEX Tissue Imaging , 2021, Frontiers in Immunology.

[8]  Mirabela Rusu,et al.  A deep learning-based algorithm for 2-D cell segmentation in microscopy images , 2018, BMC Bioinformatics.

[9]  Sean C. Bendall,et al.  Multiplexed ion beam imaging of human breast tumors , 2014, Nature Medicine.

[10]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[11]  Jens Rittscher,et al.  Coupled minimum-cost flow cell tracking for high-throughput quantitative analysis , 2011, Medical Image Anal..

[12]  Carlos Caldas,et al.  Imaging mass cytometry and multiplatform genomics define the phenogenomic landscape of breast cancer , 2020, Nature Cancer.

[13]  Qing Li,et al.  Highly multiplexed single-cell analysis of formalin-fixed, paraffin-embedded cancer tissue , 2013, Proceedings of the National Academy of Sciences.

[14]  Xian Zhang,et al.  A multi‐scale convolutional neural network for phenotyping high‐content cellular images , 2017, Bioinform..

[15]  Anne E Carpenter,et al.  CellProfiler 3.0: Next-generation image processing for biology , 2018, PLoS biology.

[16]  Fred A. Hamprecht,et al.  ilastik: interactive machine learning for (bio)image analysis , 2019, Nature Methods.

[17]  Lin Yang,et al.  Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review , 2016, IEEE Reviews in Biomedical Engineering.

[18]  Marius Pachitariu,et al.  Cellpose: a generalist algorithm for cellular segmentation , 2020, Nature methods.

[19]  Edward S Boyden,et al.  Rapid Sequential in Situ Multiplexing with DNA Exchange Imaging in Neuronal Cells and Tissues. , 2017, Nano letters.

[20]  Johannes E. Schindelin,et al.  Fiji: an open-source platform for biological-image analysis , 2012, Nature Methods.

[21]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[22]  Lassi Paavolainen,et al.  nucleAIzer: A Parameter-free Deep Learning Framework for Nucleus Segmentation Using Image Style Transfer , 2020, Cell systems.

[23]  Shuchun,et al.  Multiplexed ion beam imaging ( MIBI ) of human breast tumors , 2014 .

[24]  Euan A. Ashley,et al.  Deep Learning Automates the Quantitative Analysis of Individual Cells in Live-Cell Imaging Experiments , 2016, PLoS Comput. Biol..

[25]  Wei Huang,et al.  A Colorful Future for Quantitative Pathology: Validation of Vectra Technology Using Chromogenic Multiplexed Immunohistochemistry and Prostate Tissue Microarrays , 2012 .

[26]  Virgilio Gómez-Rubio,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[27]  G. Nolan,et al.  Automated Mapping of Phenotype Space with Single-Cell Data , 2016, Nature Methods.

[28]  Lucas Pelkmans,et al.  Multiplexed protein maps link subcellular organization to cellular states , 2018, Science.

[29]  Anne E Carpenter,et al.  Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl , 2019, Nature Methods.

[30]  et al.,et al.  Jupyter Notebooks - a publishing format for reproducible computational workflows , 2016, ELPUB.

[31]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[32]  Jens Petersen,et al.  nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation , 2020, Nature Methods.

[33]  Xiaobo Zhou,et al.  A Novel Cell Segmentation Method and Cell Phase Identification Using Markov Model , 2009, IEEE Transactions on Information Technology in Biomedicine.

[34]  Immune cell topography predicts response to PD-1 blockade in cutaneous T cell lymphoma , 2021, Nature communications.

[35]  Sandro Santagata,et al.  Highly multiplexed immunofluorescence imaging of human tissues and tumors using t-CyCIF and conventional optical microscopes , 2018 .

[36]  Holger Moch,et al.  The single-cell pathology landscape of breast cancer , 2020, Nature.

[37]  Sandro Santagata,et al.  Highly multiplexed immunofluorescence imaging of human tissues and tumors using t-CyCIF and conventional optical microscopes , 2018, eLife.

[38]  Peng Yin,et al.  DNA-barcoded labeling probes for highly multiplexed Exchange-PAINT imaging† †Electronic supplementary information (ESI) available. See DOI: 10.1039/c6sc05420j Click here for additional data file. , 2017, Chemical science.

[40]  Carlos Ortiz-de-Solorzano,et al.  Segmentation and Shape Tracking of Whole Fluorescent Cells Based on the Chan–Vese Model , 2013, IEEE Transactions on Medical Imaging.

[41]  Yu Wang,et al.  Immuno-SABER enables highly multiplexed and amplified protein imaging in tissues , 2019, Nature Biotechnology.

[42]  Sean C. Bendall,et al.  A Structured Tumor-Immune Microenvironment in Triple Negative Breast Cancer Revealed by Multiplexed Ion Beam Imaging , 2018, Cell.

[43]  J. Buhmann,et al.  Highly multiplexed imaging of tumor tissues with subcellular resolution by mass cytometry , 2014, Nature Methods.

[44]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[45]  Eugene W. Myers,et al.  Cell Detection with Star-convex Polygons , 2018, MICCAI.