Benchmarking Single-Cell RNA Sequencing Protocols for Cell Atlas Projects

Single-cell RNA sequencing (scRNA-seq) is the leading technique for charting the molecular properties of individual cells. The latest methods are scalable to thousands of cells, enabling in-depth characterization of sample composition without prior knowledge. However, there are important differences between scRNA-seq techniques, and it remains unclear which are the most suitable protocols for drawing cell atlases of tissues, organs and organisms. We have generated benchmark datasets to systematically evaluate techniques in terms of their power to comprehensively describe cell types and states. We performed a multi-center study comparing 13 commonly used single-cell and single-nucleus RNA-seq protocols using a highly heterogeneous reference sample resource. Comparative and integrative analysis at cell type and state level revealed marked differences in protocol performance, highlighting a series of key features for cell atlas projects. These should be considered when defining guidelines and standards for international consortia, such as the Human Cell Atlas project.

Oliver Stegle | Christian Conrad | Aviv Regev | Itoshi Nikaido | Rickard Sandberg | Holger Heyn | Dominic Grün | Kaori Tanaka | Adrian Alvarez | Christoph Ziegenhain | Elisabetta Mereu | Ivo Gut | Joshua Z. Levin | Swati Parekh | Wolfgang Enard | Xian Adiconis | Lan T. Nguyen | Sagar | Davis J. McCarthy | Sascha Sauer | Stéphane C. Boutet | Catia Moutinho | Robert C. Jones | Eduard Batlle | Atefeh Lafzi | Davis J. MacCarthy | Julia K. Lau | Chad Sanada | Aik Ooi | Robert C. Jones | Kelly Kaihara | Chris Brampton | Yasha Talaga | Yohei Sasagawa | Tetsutaro Hayashi | Cornelius Fischer | Timo Trefzer | Aleksandar Janjic | Lucas E. Wange | Johannes W. Bagnoli | Marta Gut | J. Levin | X. Adiconis | A. Regev | C. Conrad | R. Sandberg | M. Gut | I. Gut | O. Stegle | W. Enard | Christoph Ziegenhain | Swati Parekh | H. Heyn | S. Sauer | I. Nikaido | S. Boutet | Tetsutaro Hayashi | E. Batlle | Lan Nguyen | C. Fischer | Kelly Kaihara | Timo Trefzer | Y. Sasagawa | A. Ooi | C. Brampton | C. Moutinho | A. Janjić | L. Wange | E. Mereu | Kaori Tanaka | Atefeh Lafzi | C. Braeuning | A. Álvarez-Varela | I. Gut | Dominic Grün | Kaori Tanaka | Chad D. Sanada | A. Janjic | Yasha Talaga | C. Ziegenhain | A. Janjic | Cornelius Fischer | Yohei Sasagawa | Holger Heyn | J. Lau | Sascha Sauer | Christian Conrad | Catia Moutinho | Davis J. McCarthy | Eduard Batlle | Dominic Grün | Stéphane C. Boutet | Chad Sanada | Aik Ooi | Robert C. Jones | Kelly Kaihara | Chris Brampton | Tetsutaro Hayashi | Christian Conrad | Oliver Stegle | Xian Adiconis | Cátia Moutinho

[1]  Wei Shi,et al.  featureCounts: an efficient general purpose program for assigning sequence reads to genomic features , 2013, Bioinform..

[2]  Thomas R. Gingeras,et al.  STAR: ultrafast universal RNA-seq aligner , 2013, Bioinform..

[3]  Camille Stephan-Otto Attolini,et al.  Mex3a Marks a Slowly Dividing Subpopulation of Lgr5+ Intestinal Stem Cells. , 2017, Cell stem cell.

[4]  G. Leparc,et al.  DMSO cryopreservation is the method of choice to preserve cells for droplet-based single-cell RNA sequencing , 2019, Scientific Reports.

[5]  Chun Jimmie Ye,et al.  Multiplexed droplet single-cell RNA-sequencing using natural genetic variation , 2017, Nature Biotechnology.

[6]  Christoph Ziegenhain,et al.  zUMIs - A fast and flexible pipeline to process RNA sequencing data with UMIs , 2017, bioRxiv.

[7]  Allon M. Klein,et al.  Single-cell mapping of gene expression landscapes and lineage in the zebrafish embryo , 2018, Science.

[8]  Lucas E. Wange,et al.  Sensitive and powerful single-cell RNA sequencing using mcSCRB-seq , 2018, Nature Communications.

[9]  Andrew C. Adey,et al.  Single-Cell Transcriptional Profiling of a Multicellular Organism , 2017 .

[10]  Sven Rahmann,et al.  Snakemake--a scalable bioinformatics workflow engine. , 2012, Bioinformatics.

[11]  Cuong To,et al.  Miniaturization Technologies for Efficient Single-Cell Library Preparation for Next-Generation Sequencing , 2016, Journal of laboratory automation.

[12]  Holger Heyn,et al.  Tutorial: guidelines for the experimental design of single-cell RNA sequencing studies , 2018, Nature Protocols.

[13]  Timur Zhiyentayev,et al.  Single-cell in situ RNA profiling by sequential hybridization , 2014, Nature Methods.

[14]  Sagar,et al.  FateID infers cell fate bias in multipotent progenitors from single-cell RNA-seq data , 2017, Nature Methods.

[15]  Sven Rahmann,et al.  Genome analysis , 2022 .

[16]  Aviv Regev,et al.  Nuclei multiplexing with barcoded antibodies for single-nucleus genomics , 2018 .

[17]  Yvan Saeys,et al.  A comparison of single-cell trajectory inference methods: towards more accurate and robust tools , 2018, bioRxiv.

[18]  Andrew J. Hill,et al.  Single-cell mRNA quantification and differential analysis with Census , 2017, Nature Methods.

[19]  Christoph Ziegenhain,et al.  The impact of amplification on differential expression analyses by RNA-seq , 2016, Scientific Reports.

[20]  I. Amit,et al.  Single-cell transcriptome conservation in cryopreserved cells and tissues , 2016, Genome Biology.

[21]  David A. Weitz,et al.  Scaling by shrinking: empowering single-cell 'omics' with microfluidic devices , 2017, Nature Reviews Genetics.

[22]  James T. Webber,et al.  Single-cell transcriptomics of 20 mouse organs creates a Tabula Muris , 2018, Nature.

[23]  Dominic Grün,et al.  High-Throughput Single-Cell RNA Sequencing and Data Analysis. , 2018, Methods in molecular biology.

[24]  Bertrand Z. Yeung,et al.  Cell Hashing with barcoded antibodies enables multiplexing and doublet detection for single cell genomics , 2018, Genome Biology.

[25]  Hans Clevers,et al.  Single-cell messenger RNA sequencing reveals rare intestinal cell types , 2015, Nature.

[26]  I. Amit,et al.  Massively Parallel Single-Cell RNA-Seq for Marker-Free Decomposition of Tissues into Cell Types , 2014, Science.

[27]  P. Kharchenko,et al.  Bayesian approach to single-cell differential expression analysis , 2014, Nature Methods.

[28]  Christian H. Holland,et al.  Robustness and applicability of transcription factor and pathway analysis tools on single-cell RNA-seq data , 2020, Genome Biology.

[29]  Francisco Azuaje,et al.  A cluster validity framework for genome expression data , 2002, Bioinform..

[30]  H. Ueda,et al.  Erratum to: Quartz-Seq: a highly reproducible and sensitive single-cell RNA sequencing method, reveals non-genetic gene-expression heterogeneity , 2017, Genome Biology.

[31]  Åsa K. Björklund,et al.  Smart-seq2 for sensitive full-length transcriptome profiling in single cells , 2013, Nature Methods.

[32]  Shuqiang Li,et al.  CEL-Seq2: sensitive highly-multiplexed single-cell RNA-Seq , 2016, Genome Biology.

[33]  Charlotte Soneson,et al.  Bias, robustness and scalability in single-cell differential expression analysis , 2018, Nature Methods.

[34]  Shila Ghazanfar,et al.  scMerge leverages factor analysis, stable expression, and pseudoreplication to merge multiple single-cell RNA-seq datasets , 2019, Proceedings of the National Academy of Sciences.

[35]  Valentine Svensson,et al.  Power Analysis of Single Cell RNA-Sequencing Experiments , 2016, Nature Methods.

[36]  David A. Knowles,et al.  Batch effects and the effective design of single-cell gene expression studies , 2016, Scientific Reports.

[37]  Tetsutaro Hayashi,et al.  Quartz-Seq2: a high-throughput single-cell RNA-sequencing method that effectively uses limited sequence reads , 2017, Genome Biology.

[38]  Fabian J Theis,et al.  The Human Cell Atlas , 2017, bioRxiv.

[39]  Hugo de Jonge,et al.  Faculty of 1000 evaluation for A single-cell atlas of the airway epithelium reveals the CFTR-rich pulmonary ionocyte. , 2018 .

[40]  Yarden Katz,et al.  A single-cell survey of the small intestinal epithelium , 2017, Nature.

[41]  Jennifer L Hu,et al.  MULTI-seq: sample multiplexing for single-cell RNA sequencing using lipid-tagged indices , 2019, Nature Methods.

[42]  I. Hellmann,et al.  Comparative Analysis of Single-Cell RNA Sequencing Methods , 2016, bioRxiv.

[43]  Somasekar Seshagiri,et al.  Massively parallel nanowell-based single-cell gene expression profiling , 2017, BMC Genomics.

[44]  Foo Wei Ten,et al.  Single nucleus RNA sequencing maps acinar cell states in a human pancreas cell atlas , 2019, bioRxiv.

[45]  Åsa K. Björklund,et al.  Tn5 transposase and tagmentation procedures for massively scaled sequencing projects , 2014, Genome research.

[46]  Yvan Saeys,et al.  A comparison of single-cell trajectory inference methods , 2019, Nature Biotechnology.

[47]  Michael C. Heinold,et al.  A comprehensive assessment of somatic mutation detection in cancer using whole-genome sequencing , 2015, Nature Communications.

[48]  A. Regev,et al.  Spatial reconstruction of single-cell gene expression data , 2015 .

[49]  Sarah A Teichmann,et al.  A test metric for assessing single-cell RNA-seq batch correction , 2018, Nature Methods.

[50]  Jonathan S. Weissman,et al.  MULTI-seq: Scalable sample multiplexing for single-cell RNA sequencing using lipid-tagged indices , 2018, bioRxiv.

[51]  S. Teichmann,et al.  Exponential scaling of single-cell RNA-seq in the past decade , 2017, Nature Protocols.

[52]  Hazen P Babcock,et al.  High-throughput single-cell gene-expression profiling with multiplexed error-robust fluorescence in situ hybridization , 2016, Proceedings of the National Academy of Sciences.

[53]  Fan Zhang,et al.  Fast, sensitive, and accurate integration of single cell data with Harmony , 2018, bioRxiv.

[54]  Fabian J Theis,et al.  Cell type atlas and lineage tree of a whole complex animal by single-cell transcriptomics , 2018, Science.

[55]  Dominic Grün,et al.  A Human Liver Cell Atlas reveals Heterogeneity and Epithelial Progenitors , 2019, Nature.

[56]  Salah Ayoub,et al.  The Drosophila Embryo at Single Cell Transcriptome Resolution , 2017, bioRxiv.

[57]  Grace X. Y. Zheng,et al.  Massively parallel digital transcriptional profiling of single cells , 2016, Nature Communications.

[58]  A. van Oudenaarden,et al.  Single-cell sequencing reveals dissociation-induced gene expression in tissue subpopulations , 2017, Nature Methods.

[59]  Giovanni Iacono,et al.  Sampling artifacts in single-cell genomics cohort studies , 2020, bioRxiv.

[60]  Aviv Regev,et al.  A revised airway epithelial hierarchy includes CFTR-expressing ionocytes , 2018, Nature.

[61]  Evan Z. Macosko,et al.  Highly Parallel Genome-wide Expression Profiling of Individual Cells Using Nanoliter Droplets , 2015, Cell.

[62]  Allon M. Klein,et al.  Droplet Barcoding for Single-Cell Transcriptomics Applied to Embryonic Stem Cells , 2015, Cell.