ACCELERATING COMPARATIVE GENOMICS WORKFLOWS IN A DISTRIBUTED ENVIRONMENT WITH OPTIMIZED DATA PARTITIONING AND WORKFLOW FUSION.
Gespeichert in:
| Titel: | ACCELERATING COMPARATIVE GENOMICS WORKFLOWS IN A DISTRIBUTED ENVIRONMENT WITH OPTIMIZED DATA PARTITIONING AND WORKFLOW FUSION. |
|---|---|
| Autoren: | CHOUDHURY, OLIVIA, HAZEKAMP, NICHOLAS L., THAIN, DOUGLAS, EMRICH, SCOTT J. |
| Quelle: | Scalable Computing: Practice & Experience; Mar2015, Vol. 16 Issue 1, p53-69, 17p |
| Schlagwörter: | COMPARATIVE genomics, WORKFLOW, CLOUD computing software, DATA analysis, COST |
| Abstract: | The advent of next generation sequencing technology has generated massive amounts of biological data at unprecendented rates. Comparative genomics applications often require compute-intensive tools for subsequent analysis of high throughput data. Although cloud computing infrastructure plays an important role in this respect, the pressure from such computationally expensive tasks can be further alleviated using efficient data partitioning and workflow fusion. Here, we implement a workflow-based model for parallelizing the data-intensive tasks of genome alignment and variant calling with BWA and GATK's HaplotypeCaller. We explore three different approaches of partitioning data, granularity-based, individual-based, and alignment-based , and how each affect the run time. We observe granularity-based partitioning for BWA and alignment-based partitioning for HaplotypeCaller to be the optimal choices for the pipeline. We further discuss the methods and impact of workflow fusion on performance by considering different levels of fusion and how it affects our results. We identify the various open problems encountered, such as understanding the extent of parallelism, using heterogenous environments without a shared file system, and determining the granularity of inputs, and provide insights into addressing them. Finally, we report significant performance improvements, from 12 days to under 2 hours while running the BWA-GATK pipeline using partitioning and fusion. [ABSTRACT FROM AUTHOR] |
| Copyright of Scalable Computing: Practice & Experience is the property of Scalable Computing: Practice & Experience and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Datenbank: | Complementary Index |
Full Text Finder 
Nájsť tento článok vo Web of Science Schreiben Sie den ersten Kommentar!