Parallelization of the Trinity Pipeline for De Novo Transcriptome Assembly

This paper details a distributed-memory implementation of Chrysalis, part of the popular Trinity workflow used for de novo transcripto me assembly. We have implemented changes to Chrysalis, which was previously multi-threaded for shared-memory architectures, to change it to a hybrid implementation which uses both MPI and OpenMP. With the new hybrid implementation, we report speedups of about a factor of twenty for both Graph From Fasta and Reads To Transcripts on an iDataPlex cluster for a sugar beet dataset containing around 130 million reads. Along with the hybrid implementation, we also use PyFasta to speed up Bowtie execution by a factor of three which is also part of the Trinity workflow. Overall, we reduce the runtime of the Chrysalis step of the Trinity workflow from over 50 hours to less than 5 hours for the sugar beet dataset. By enabling the use of multi-node clusters, this implementation is a significant step towards making de novo transcripto me assembly feasible for ever bigger transcripto me datasets.

[1]  François Laviolette,et al.  Ray: Simultaneous Assembly of Reads from a Mix of High-Throughput Sequencing Technologies , 2010, J. Comput. Biol..

[2]  Cole Trapnell,et al.  Ultrafast and memory-efficient alignment of short DNA sequences to the human genome , 2009, Genome Biology.

[3]  Philip L. F. Johnson,et al.  The complete genome sequence of a Neanderthal from the Altai Mountains , 2013 .

[4]  Ümit V. Çatalyürek,et al.  Exploring parallelism in short sequence mapping using Burrows-Wheeler Transform , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW).

[5]  Srinivas Aluru,et al.  Parallel short sequence assembly of transcriptomes , 2009, BMC Bioinformatics.

[6]  J. Harrow,et al.  Assessment of transcript reconstruction methods for RNA-seq , 2013, Nature Methods.

[7]  R. Wooster.  The cancer genome project , 2002 .

[8]  Le-Shin Wu,et al.  Trinity RNA-Seq assembler performance optimization , 2012, XSEDE '12.

[9]  John Shalf,et al.  Power efficiency in high performance computing , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[10]  Xuan Li,et al.  Optimizing de novo transcriptome assembly from short-read RNA-Seq data: a comparative study , 2011, BMC Bioinformatics.

[11]  Colin N. Dewey,et al.  De novo transcript sequence reconstruction from RNA-seq using the Trinity platform for reference generation and analysis , 2013, Nature Protocols.

[12]  W. Pearson.  Searching protein sequence libraries: comparison of the sensitivity and selectivity of the Smith-Waterman and FASTA algorithms. , 1991, Genomics.

[13]  Carl Kingsford,et al.  A fast, lock-free approach for efficient parallel counting of occurrences of k-mers , 2011, Bioinform..

[14]  Colin N. Dewey,et al.  RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome , 2011, BMC Bioinformatics.

[15]  N. Friedman,et al.  Trinity: reconstructing a full-length transcriptome without a genome from RNA-Seq data , 2011, Nature Biotechnology.

[16]  Steven J. M. Jones,et al.  Abyss: a Parallel Assembler for Short Read Sequence Data Material Supplemental Open Access , 2022 .

[17]  Mark D. Robinson,et al.  edgeR: a Bioconductor package for differential expression analysis of digital gene expression data , 2009, Bioinform..

[18]  Dominique Lavenier,et al.  DSK: k-mer counting with very low memory usage , 2013, Bioinform..

[19]  Zhong Wang,et al.  Next-generation transcriptome assembly , 2011, Nature Reviews Genetics.

[20]  Stephen A. Smith,et al.  Optimizing de novo assembly of short-read RNA-seq data for phylogenomics , 2013, BMC Genomics.