From Scripted HPC-Based NGS Pipelines to Workflows on the Cloud

In this paper we describe our initial experiences in the Cloud-e-Genome project with moving the whole exome sequencing pipeline from the scripted HPC-based solution to a workflow enactment system running in the cloud. We discuss shortcomings of the existing approach based on scripts and list benefit...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing S. 694 - 700
Hauptverfasser: Cala, Jacek, Yaobo Xu, Wijaya, Eldarina Azfar, Missier, Paolo
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.05.2014
Schlagworte:
Bioinformatics
Cloud computing
Computational modeling
Genomics
Libraries
Pipelines
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this paper we describe our initial experiences in the Cloud-e-Genome project with moving the whole exome sequencing pipeline from the scripted HPC-based solution to a workflow enactment system running in the cloud. We discuss shortcomings of the existing approach based on scripts and list benefits that a workflow-based solution can provide. Despite the effort it involved to wrap all required tools in the form of workflow blocks and the restrictions of the dataflow model used to represent workflows we expect the migration to significantly improve the current status of the pipeline. Our target is to enable flexibility, traceability and reproducibility of the solution, so that it can better fit the evolution of tools, data and pipeline itself and allow us to run it at national scale. This work will become foundation for the more complete system that includes variant filtering and interpretation for the diagnostic purposes.
DOI:10.1109/CCGrid.2014.128