A Hadoop-Galaxy adapter for user-friendly and scalable data-intensive bioinformatics in Galaxy

In this work we present a strategy to integrate Hadoop-based applications into the Galaxy platform along with an extensible implementation of this adapter and related utilities. The strategy is based on the idea of introducing a new Galaxy datatype that provides a layer of indirection, thus relaxing the requirement to place data on a Galaxy-accessible file system and instead allowing the referenced data to be placed on any addressable space, including the Hadoop Distributed File System or Amazon S3. The adapter supports using Hadoop-based applications as part of Galaxy workflows. We demonstrate a practical application where this Hadoop-Galaxy adapter was used at CRS4 to accelerate the bioinformatics analysis of viral vector integration sites through the introduction of Hadoop-based computation components, while keeping the workflow under control of biologists with little specific technical training.