A benchmark for evaluation of algorithms for identification of cellular correlates of clinical outcomes

The Flow Cytometry: Critical Assessment of Population Identification Methods (FlowCAP) challenges were established to compare the performance of computational methods for identifying cell populations in multidimensional flow cytometry data. Here we report the results of FlowCAP‐IV where algorithms from seven different research groups predicted the time to progression to AIDS among a cohort of 384 HIV+ subjects, using antigen‐stimulated peripheral blood mononuclear cell (PBMC) samples analyzed with a 14‐color staining panel. Two approaches (FlowReMi.1 and flowDensity‐flowType‐RchyOptimyx) provided statistically significant predictive value in the blinded test set. Manual validation of submitted results indicated that unbiased analysis of single cell phenotypes could reveal unexpected cell types that correlated with outcomes of interest in high dimensional flow cytometry datasets. © 2015 International Society for Advancement of Cytometry

[1]  Yvan Saeys,et al.  FloReMi: Flow density survival regression using minimal feature redundancy , 2016, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[2]  Dong Ling Tong,et al.  gEM/GANN: A multivariate computational strategy for auto‐characterizing relationships between cellular and clinical phenotypes and predicting disease progression time using high‐dimensional flow cytometry data , 2015, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[3]  Greg Finak,et al.  flowDensity: reproducing manual gating of flow cytometry data by automated density-based cell population identification , 2015, Bioinform..

[4]  Stuart B. Goodman,et al.  Clinical recovery from surgery correlates with single-cell immune signatures , 2014, Science Translational Medicine.

[5]  Murat Dundar,et al.  A non-parametric Bayesian model for joint cell clustering and cluster matching: identification of anomalous sample phenotypes with random effects , 2014, BMC Bioinformatics.

[6]  Greg Finak,et al.  OpenCyto: An Open Source Infrastructure for Scalable, Robust, Reproducible, and Automated, End-to-End Flow Cytometry Data Analysis , 2014, PLoS Comput. Biol..

[7]  Xi Zhao,et al.  CCAST: A Model-Based Gating Strategy to Isolate Homogeneous Subpopulations in a Heterogeneous Population of Single Cells , 2014, PLoS Comput. Biol..

[8]  R. Tibshirani,et al.  Automated identification of stratifying signatures in cellular subpopulations , 2014, Proceedings of the National Academy of Sciences.

[9]  Holger H. Hoos,et al.  Enhanced flowType/RchyOptimyx: a Bioconductor pipeline for discovery in high-dimensional cytometry data , 2014, Bioinform..

[10]  Mark M. Davis,et al.  Automatic Classification of Cellular Expression by Nonlinear Stochastic Embedding (ACCENSE) , 2013, Proceedings of the National Academy of Sciences.

[11]  Nima Aghaeepour,et al.  Flow Cytometry Bioinformatics , 2013, PLoS Comput. Biol..

[12]  Ariful Azad,et al.  Title of the Report: Classifying Immunophenotypes with Templates from Flow Cytometry , 2022 .

[13]  Sean C. Bendall,et al.  viSNE enables visualization of high dimensional single-cell data and reveals phenotypic heterogeneity of leukemia , 2013, Nature Biotechnology.

[14]  Greg Finak,et al.  Critical assessment of automated flow cytometry data analysis techniques , 2013, Nature Methods.

[15]  Gholamreza Haffari,et al.  Scoring relevancy of features based on combinatorial analysis of Lasso with application to lymphoma diagnosis , 2013, BMC Genomics.

[16]  H. Hoos,et al.  RchyOptimyx: Cellular hierarchy optimization for flow cytometry , 2012, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[17]  Josef Spidlen,et al.  FlowRepository: A resource of annotated flow cytometry datasets associated with peer‐reviewed publications , 2012, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[18]  S. Sealfon,et al.  flowPeaks: a fast unsupervised clustering for flow cytometry data via K-means and density peak finding , 2012, Bioinform..

[19]  P. Bruhns.  Properties of mouse and human IgG receptors and their contribution to disease models. , 2012, Blood.

[20]  Yu Qian,et al.  FCSTrans: An open source software system for FCS file conversion and data transformation , 2012, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[21]  Wayne A Moore,et al.  Update for the logicle data scale including operational code implementations , 2012, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[22]  Pratip K. Chattopadhyay,et al.  Early immunologic correlates of HIV protection can be identified from computational analysis of complex multivariate T-cell flow cytometry assays , 2012, Bioinform..

[23]  Sean C. Bendall,et al.  Extracting a Cellular Hierarchy from High-dimensional Cytometry Data with SPADE , 2011, Nature Biotechnology.

[24]  Dong-Ling Tong,et al.  Hybrid genetic algorithm-neural network: Feature extraction for unpreprocessed microarray data , 2011, Artif. Intell. Medicine.

[25]  Handan Wand,et al.  Plasma levels of soluble CD14 independently predict mortality in HIV infection. , 2011, The Journal of infectious diseases.

[26]  K. Shianna,et al.  Evidence of dysregulation of dendritic cells in primary HIV infection , 2010, Blood.

[27]  Dong Ling Tong,et al.  Genetic Algorithm-Neural Network (GANN): a study of neural network activation functions and depth of genetic algorithm search applied to feature selection , 2010, Int. J. Mach. Learn. Cybern..

[28]  P. Chattopadhyay,et al.  Good cell, bad cell: Flow cytometry reveals T‐cell subsets important in HIV disease , 2010, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[29]  Mario Roederer,et al.  Immunologic and virologic events in early HIV infection predict subsequent rate of progression. , 2010, The Journal of infectious diseases.

[30]  Ryan R Brinkman,et al.  Per‐channel basis normalization methods for flow cytometry data , 2009, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[31]  Chie Sugimoto,et al.  The level of monocyte turnover predicts disease progression in the macaque model of AIDS. , 2009, Blood.

[32]  Wade T. Rogers,et al.  FlowFP: A Bioconductor Package for Fingerprinting Flow Cytometric Data , 2009, Adv. Bioinformatics.

[33]  M. Roediger,et al.  Increasing Age at HIV Seroconversion From 18 to 40 Years Is Associated With Favorable Virologic and Immunologic Responses to HAART , 2008, Journal of acquired immune deficiency syndromes.

[34]  Laurie Lamoreaux,et al.  Amine reactive dyes: an effective tool to discriminate live and dead cells in polychromatic flow cytometry. , 2006, Journal of immunological methods.

[35]  M. Müller-Trutwin,et al.  Role for plasmacytoid dendritic cells in anti‐HIV innate immunity , 2005, Immunology and cell biology.

[36]  Tao Dong,et al.  Immune Activation and CD8+ T-Cell Differentiation towards Senescence in HIV-1 Infection , 2004, PLoS biology.

[37]  M. Roederer,et al.  Probability binning comparison: a metric for quantitating univariate distribution differences. , 2001, Cytometry.

[38]  R. Tibshirani,et al.  Significance analysis of microarrays applied to the ionizing radiation response , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[39]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .