Sustainable data analysis with Snakemake [version 2; peer review: 2 approved]

Data analysis often entails a multitude of heterogeneous steps, from the application of various command line tools to the usage of scripting languages like R or Python for the generation of plots and tables. It is widely recognized that data analyses should ideally be conducted in a reproducible way...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:F1000 research Jg. 10; S. 33
Hauptverfasser: Mölder, Felix, Jablonski, Kim Philipp, Letcher, Brice, Hall, Michael B, Tomkins-Tinch, Christopher H, Sochat, Vanessa, Forster, Jan, Lee, Soohyun, Twardziok, Sven O, Kanitz, Alexander, Wilm, Andreas, Holtgrewe, Manuel, Rahmann, Sven, Nahnsen, Sven, Köster, Johannes
Format: Journal Article
Sprache:Englisch
Veröffentlicht: England Faculty of 1000 Ltd 2021
F1000 Research Limited
F1000 Research Ltd
Schlagworte:
Adaptability
Automation
Data Analysis
Data processing
Method
Programming languages
Quality control
Reproducibility
Reproducibility of Results
Software
Sustainability
Workflow
Workflow software
data analysis
reproducibility
transparency
workflow management
adaptability
sustainability
scalability
ISSN:2046-1402, 2046-1402
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Data analysis often entails a multitude of heterogeneous steps, from the application of various command line tools to the usage of scripting languages like R or Python for the generation of plots and tables. It is widely recognized that data analyses should ideally be conducted in a reproducible way. Reproducibility enables technical validation and regeneration of results on the original or even new data. However, reproducibility alone is by no means sufficient to deliver an analysis that is of lasting impact (i.e., sustainable) for the field, or even just one research group. We postulate that it is equally important to ensure adaptability and transparency. The former describes the ability to modify the analysis to answer extended or slightly different research questions. The latter describes the ability to understand the analysis in order to judge whether it is not only technically, but methodologically valid. Here, we analyze the properties needed for a data analysis to become reproducible, adaptable, and transparent. We show how the popular workflow management system Snakemake can be used to guarantee this, and how it enables an ergonomic, combined, unified representation of all steps involved in data analysis, ranging from raw data processing, to quality control and fine-grained, interactive exploration and plotting of final results.
Bibliographie:new_version
ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
No competing interests were disclosed.
ISSN:2046-1402
2046-1402
DOI:10.12688/f1000research.29032.2