The five pillars of computational reproducibility: bioinformatics and beyond

Abstract Computational reproducibility is a simple premise in theory, but is difficult to achieve in practice. Building upon past efforts and proposals to maximize reproducibility and rigor in bioinformatics, we present a framework called the five pillars of reproducible computational research. Thes...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Briefings in bioinformatics Ročník 24; číslo 6; s. 1 - 13
Hlavní autoři: Ziemann, Mark, Poulain, Pierre, Bora, Anusuiya
Médium: Journal Article
Jazyk:angličtina
Vydáno: England Oxford University Press 22.09.2023
Oxford Publishing Limited (England)
Oxford University Press (OUP)
Témata:
Bioinformatics
Computational Biology
Computer applications
Computer Science
Information Dissemination
Reproducibility
Reproducibility of Results
Review
Software
Version control
research best practices
reproducibility
bioinformatics
computational research
ISSN:1467-5463, 1477-4054, 1477-4054
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract Abstract Computational reproducibility is a simple premise in theory, but is difficult to achieve in practice. Building upon past efforts and proposals to maximize reproducibility and rigor in bioinformatics, we present a framework called the five pillars of reproducible computational research. These include (1) literate programming, (2) code version control and sharing, (3) compute environment control, (4) persistent data sharing and (5) documentation. These practices will ensure that computational research work can be reproduced quickly and easily, long into the future. This guide is designed for bioinformatics data analysts and bioinformaticians in training, but should be relevant to other domains of study.
AbstractList Abstract Computational reproducibility is a simple premise in theory, but is difficult to achieve in practice. Building upon past efforts and proposals to maximize reproducibility and rigor in bioinformatics, we present a framework called the five pillars of reproducible computational research. These include (1) literate programming, (2) code version control and sharing, (3) compute environment control, (4) persistent data sharing and (5) documentation. These practices will ensure that computational research work can be reproduced quickly and easily, long into the future. This guide is designed for bioinformatics data analysts and bioinformaticians in training, but should be relevant to other domains of study.
Abstract Computational reproducibility is a simple premise in theory, but is difficult to achieve in practice. Building upon past efforts and proposals to maximize reproducibility and rigor in bioinformatics, we present a framework called the five pillars of reproducible computational research. These include (1) literate programming, (2) code version control and sharing, (3) compute environment control, (4) persistent data sharing and (5) documentation. These practices will ensure that computational research work can be reproduced quickly and easily, long into the future. This guide is designed for bioinformatics data analysts and bioinformaticians in training, but should be relevant to other domains of study.
Computational reproducibility is a simple premise in theory, but is difficult to achieve in practice. Building upon past efforts and proposals to maximize reproducibility and rigor in bioinformatics, we present a framework called the five pillars of reproducible computational research. These include (1) literate programming, (2) code version control and sharing, (3) compute environment control, (4) persistent data sharing and (5) documentation. These practices will ensure that computational research work can be reproduced quickly and easily, long into the future. This guide is designed for bioinformatics data analysts and bioinformaticians in training, but should be relevant to other domains of study.
Computational reproducibility is a simple premise in theory, but is difficult to achieve in practice. Building upon past efforts and proposals to maximize reproducibility and rigor in bioinformatics, we present a framework called the five pillars of reproducible computational research. These include (1) literate programming, (2) code version control and sharing, (3) compute environment control, (4) persistent data sharing and (5) documentation. These practices will ensure that computational research work can be reproduced quickly and easily, long into the future. This guide is designed for bioinformatics data analysts and bioinformaticians in training, but should be relevant to other domains of study.Computational reproducibility is a simple premise in theory, but is difficult to achieve in practice. Building upon past efforts and proposals to maximize reproducibility and rigor in bioinformatics, we present a framework called the five pillars of reproducible computational research. These include (1) literate programming, (2) code version control and sharing, (3) compute environment control, (4) persistent data sharing and (5) documentation. These practices will ensure that computational research work can be reproduced quickly and easily, long into the future. This guide is designed for bioinformatics data analysts and bioinformaticians in training, but should be relevant to other domains of study.
Author Poulain, Pierre
Ziemann, Mark
Bora, Anusuiya
Author_xml – sequence: 1
  givenname: Mark
  surname: Ziemann
  fullname: Ziemann, Mark
  email: m.ziemann@deakin.edu.au
– sequence: 2
  givenname: Pierre
  surname: Poulain
  fullname: Poulain, Pierre
– sequence: 3
  givenname: Anusuiya
  surname: Bora
  fullname: Bora, Anusuiya
BackLink https://www.ncbi.nlm.nih.gov/pubmed/37870287$$D View this record in MEDLINE/PubMed
https://hal.science/hal-04255776$$DView record in HAL
BookMark eNp9kdtrFDEYxYNU7EWffJcBQSxlbK6TTF-klNoWFvpSn0OSSdyUmWRMZhb2vzfLbEUX8Skh-Z3zXc4pOAoxWADeI_gFwZZcaq8vtVYd4ewVOEGU85pCRo9294bXjDbkGJzm_AwhhlygN-CYcMEhFvwErJ7WtnJ-Y6vR971KuYquMnEY50lNPgbVV8mOKXaz8dr3ftpeVdpHH1xMQyFMrlToKm23MXRvwWun-mzf7c8z8P3b7dPNfb16vHu4uV7VhnI21Y0ThhJkMGGIacG5MhpD1LaQd8Qg3RpjLSWGdJRjbqyzDWSQtYJr6rSz5Ax8XXzHWQ-2MzZMSfVyTH5QaSuj8vLvn-DX8kfcSFRcEIG8OJwvDusD3f31Su7eIMWMcd5sUGE_76ul-HO2eZKDz8aWbQUb5yyxEFDgguOCfjxAn-OcyhIL1TJGYCMIKdSHP9v_Xf8llQKgBTAp5pysk8YvcZRpfF_GkLvkZUle7pMvmosDzYvtv-lPCx3n8b_gL6AKva4
CitedBy_id crossref_primary_10_1111_jvs_13224
crossref_primary_10_1016_j_patter_2025_101172
crossref_primary_10_1002_mdr2_70028
crossref_primary_10_1016_j_jss_2024_112327
crossref_primary_10_1016_j_compbiolchem_2025_108535
crossref_primary_10_3758_s13428_024_02501_5
crossref_primary_10_17645_mac_7789
crossref_primary_10_3390_genes16020167
crossref_primary_10_1016_j_jmb_2024_168522
crossref_primary_10_1016_j_phyplu_2025_100820
crossref_primary_10_1038_s41576_024_00767_1
crossref_primary_10_1038_s41467_024_52446_8
Cites_doi 10.1371/journal.pcbi.1004947
10.1186/s13041-020-0552-2
10.1126/science.1203354
10.1038/483531a
10.1136/bmj.h2559
10.1007/978-1-4842-5313-7_15
10.1093/bib/bbs078
10.1371/journal.pone.0078080
10.7554/eLife.16800
10.1038/nm1491
10.1016/j.cels.2018.03.014
10.12688/f1000research.29032.1
10.1016/j.procs.2011.04.074
10.1038/nbt.3820
10.1093/bib/bbz124
10.1126/science.1213847
10.1038/s41597-022-01143-6
10.1093/comjnl/27.2.97
10.1371/journal.pcbi.1009935
10.1016/j.xgen.2021.100029
10.1371/journal.pone.0177459
10.1136/bmj.k510
10.1186/s13104-021-05875-3
10.21105/joss.02959
10.1371/journal.pcbi.1004668
10.1002/pmic.200401303
10.1038/s42005-020-00403-4
10.1038/ng0506-500
10.1093/bioinformatics/bts480
10.32614/RJ-2011-002
10.1186/s41073-021-00113-7
10.1371/journal.pone.0080278
10.1371/journal.pcbi.1008984
10.7717/peerj.13933
10.7717/peerj-cs.839
10.1186/s13059-016-1044-7
10.12688/f1000research.3-62.v1
10.1186/s12918-016-0288-x
10.3389/fninf.2017.00069
10.1371/journal.pcbi.1005510
10.1186/s13059-019-1649-8
10.1016/j.anbehav.2014.09.025
10.1371/journal.pbio.1002506
10.1371/journal.pcbi.1010193
10.7717/peerj.1273
10.1038/ng.295
10.1093/bioinformatics/btx192
10.1186/1471-2105-5-80
10.1093/nar/30.1.207
10.1093/gigascience/giad031
10.1109/MCSE.2022.3165626
10.1038/d41586-020-02462-7
10.1093/biostatistics/kxq028
10.1126/science.354.6308.142
10.1093/bib/bbw020
10.12688/f1000research.9315.1
10.12688/f1000research.20843.1
10.1038/nbt.4314
10.1093/bioinformatics/btz931
10.1038/s41597-022-01720-9
10.1371/journal.pcbi.1003285
10.1109/MCSE.2021.3059263
10.1038/d41586-018-07196-1
10.1186/s13059-020-02010-6
10.1126/scitranslmed.aaf5003
10.1038/ng1201-365
10.1126/scitranslmed.aaf5027
10.12688/f1000research.8414.1
10.1371/journal.pbio.1002295
10.1016/j.xgen.2021.100032
10.1186/s13742-016-0135-4
10.1038/s41597-023-02463-x
10.1038/d41586-022-00563-z
10.1371/journal.pbio.3000333
10.1145/3183558
10.1371/journal.pcbi.1008316
10.1126/science.1229566
10.1093/gigascience/giy123
10.1371/journal.pcbi.1002375
10.1038/nbt.3780
10.1186/1751-0473-8-7
10.12688/f1000research.9110.1
10.5964/qcmb.3763
10.1093/bib/bbt086
10.1159/000294150
10.1038/545387a
10.1126/science.aah6168
10.1371/journal.pbio.1002456
10.1038/s41562-016-0021
10.1038/nm0111-135
10.1186/1471-2105-14-S14-S5
10.1371/journal.pone.0183591
10.1002/jwmg.21855
10.1093/bib/bbu043
10.1109/HICSS.1996.495416
10.1145/3486897
10.1109/MCSE.2018.042781334
10.1038/s41576-020-0257-5
10.1093/gigascience/giaa140
10.1038/nrg3305
10.17226/25104
10.1146/annurev-psych-020821-114157
10.1016/j.jclinepi.2022.05.019
10.1186/1471-2164-11-S4-S27
10.1038/sdata.2016.18
10.1371/journal.pcbi.1009651
10.1093/nar/gkr854
10.1093/gigascience/giy077
10.1038/d41586-023-01469-0
10.1371/journal.pcbi.1007007
10.1201/9781138359444
ContentType Journal Article
Copyright The Author(s) 2023. Published by Oxford University Press. 2023
The Author(s) 2023. Published by Oxford University Press.
licence_http://creativecommons.org/publicdomain/zero
Copyright_xml – notice: The Author(s) 2023. Published by Oxford University Press. 2023
– notice: The Author(s) 2023. Published by Oxford University Press.
– notice: licence_http://creativecommons.org/publicdomain/zero
DBID TOX
AAYXX
CITATION
CGR
CUY
CVF
ECM
EIF
NPM
7QO
7SC
8FD
FR3
JQ2
K9.
L7M
L~C
L~D
P64
RC3
7X8
1XC
VOOES
5PM
DOI 10.1093/bib/bbad375
DatabaseName Oxford Journals Open Access Collection
CrossRef
Medline
MEDLINE
MEDLINE (Ovid)
MEDLINE
MEDLINE
PubMed
Biotechnology Research Abstracts
Computer and Information Systems Abstracts
Technology Research Database
Engineering Research Database
ProQuest Computer Science Collection
ProQuest Health & Medical Complete (Alumni)
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
Biotechnology and BioEngineering Abstracts
Genetics Abstracts
MEDLINE - Academic
Hyper Article en Ligne (HAL)
Hyper Article en Ligne (HAL) (Open Access)
PubMed Central (Full Participant titles)
DatabaseTitle CrossRef
MEDLINE
Medline Complete
MEDLINE with Full Text
PubMed
MEDLINE (Ovid)
Genetics Abstracts
Biotechnology Research Abstracts
Technology Research Database
Computer and Information Systems Abstracts – Academic
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
ProQuest Health & Medical Complete (Alumni)
Engineering Research Database
Advanced Technologies Database with Aerospace
Biotechnology and BioEngineering Abstracts
Computer and Information Systems Abstracts Professional
MEDLINE - Academic
DatabaseTitleList


MEDLINE - Academic
MEDLINE
CrossRef
Genetics Abstracts
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: TOX
  name: Oxford Journals Open Access Collection
  url: https://academic.oup.com/journals/
  sourceTypes: Publisher
– sequence: 3
  dbid: 7X8
  name: MEDLINE - Academic
  url: https://search.proquest.com/medline
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Biology
Computer Science
EISSN 1477-4054
EndPage 13
ExternalDocumentID PMC10591307
oai:HAL:hal-04255776v1
37870287
10_1093_bib_bbad375
10.1093/bib/bbad375
Genre Research Support, Non-U.S. Gov't
Journal Article
GroupedDBID ---
-E4
.2P
.I3
0R~
1TH
23N
2WC
36B
4.4
48X
53G
5GY
5VS
6J9
70D
8VB
AAGQS
AAHBH
AAIJN
AAIMJ
AAJKP
AAJQQ
AAMDB
AAMVS
AAOGV
AAPQZ
AAPXW
AARHZ
AAUQX
AAVAP
AAVLN
ABDBF
ABEJV
ABEUO
ABGNP
ABIXL
ABNKS
ABPQP
ABPTD
ABQLI
ABQTQ
ABWST
ABXVV
ABXZS
ABZBJ
ACGFO
ACGFS
ACGOD
ACIWK
ACPRK
ACUFI
ACUHS
ACUXJ
ACYTK
ADBBV
ADEYI
ADFTL
ADGKP
ADGZP
ADHKW
ADHZD
ADOCK
ADPDF
ADQBN
ADRDM
ADRTK
ADVEK
ADYVW
ADZTZ
ADZXQ
AECKG
AEGPL
AEGXH
AEJOX
AEKKA
AEKSI
AELWJ
AEMDU
AEMOZ
AENEX
AENZO
AEPUE
AETBJ
AEWNT
AFFZL
AFGWE
AFIYH
AFOFC
AFRAH
AGINJ
AGKEF
AGQXC
AGSYK
AHMBA
AHQJS
AHXPO
AIAGR
AIJHB
AJEEA
AJEUX
AKHUL
AKVCP
AKWXX
ALMA_UNASSIGNED_HOLDINGS
ALTZX
ALUQC
ALXQX
AMNDL
ANAKG
APIBT
APWMN
ARIXL
AXUDD
AYOIW
AZVOD
BAWUL
BAYMD
BEYMZ
BHONS
BQDIO
BQUQU
BSWAC
BTQHN
C1A
C45
CAG
CDBKE
COF
CS3
CZ4
DAKXR
DIK
DILTD
DU5
D~K
E3Z
EAD
EAP
EAS
EBA
EBC
EBD
EBR
EBS
EBU
EE~
EJD
EMB
EMK
EMOBN
EST
ESX
F5P
F9B
FHSFR
FLIZI
FLUFQ
FOEOM
FQBLK
GAUVT
GJXCC
GROUPED_DOAJ
GX1
H13
H5~
HAR
HW0
HZ~
IOX
J21
JXSIZ
K1G
KBUDW
KOP
KSI
KSN
M-Z
M49
MK~
ML0
N9A
NGC
NLBLG
NMDNZ
NOMLY
NU-
O0~
O9-
OAWHX
ODMLO
OJQWA
OK1
OVD
OVEED
P2P
PAFKI
PEELM
PQQKQ
Q1.
Q5Y
QWB
RD5
RPM
RUSNO
RW1
RXO
SV3
TEORI
TH9
TJP
TLC
TOX
TR2
TUS
W8F
WOQ
X7H
YAYTL
YKOAZ
YXANX
ZKX
ZL0
~91
77I
AAYXX
AHGBF
CITATION
ROX
CGR
CUY
CVF
ECM
EIF
NPM
7QO
7SC
8FD
FR3
JQ2
K9.
L7M
L~C
L~D
P64
RC3
7X8
1XC
VOOES
5PM
ID FETCH-LOGICAL-c475t-6f8c431c23515b877acb2019907d3c1b9ccee43c3d4727cefe60505987b4fbfe3
IEDL.DBID TOX
ISICitedReferencesCount 17
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001136371500043&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1467-5463
1477-4054
IngestDate Tue Nov 04 02:06:25 EST 2025
Tue Nov 18 06:21:19 EST 2025
Sat Sep 27 20:30:09 EDT 2025
Sun Nov 30 05:19:37 EST 2025
Mon Jul 21 05:59:44 EDT 2025
Sat Nov 29 05:43:39 EST 2025
Tue Nov 18 21:33:41 EST 2025
Wed Apr 02 07:05:42 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 6
Keywords research best practices
reproducibility
bioinformatics
computational research
Language English
License This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
https://creativecommons.org/licenses/by/4.0
The Author(s) 2023. Published by Oxford University Press.
licence_http://creativecommons.org/publicdomain/zero/: http://creativecommons.org/publicdomain/zero
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c475t-6f8c431c23515b877acb2019907d3c1b9ccee43c3d4727cefe60505987b4fbfe3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ObjectType-Literature Review-3
content type line 23
ORCID 0009-0006-2908-1352
0000-0003-4177-3619
0000-0002-7688-6974
OpenAccessLink https://dx.doi.org/10.1093/bib/bbad375
PMID 37870287
PQID 2955306833
PQPubID 26846
PageCount 13
ParticipantIDs pubmedcentral_primary_oai_pubmedcentral_nih_gov_10591307
hal_primary_oai_HAL_hal_04255776v1
proquest_miscellaneous_2880822552
proquest_journals_2955306833
pubmed_primary_37870287
crossref_citationtrail_10_1093_bib_bbad375
crossref_primary_10_1093_bib_bbad375
oup_primary_10_1093_bib_bbad375
PublicationCentury 2000
PublicationDate 2023-09-22
PublicationDateYYYYMMDD 2023-09-22
PublicationDate_xml – month: 09
  year: 2023
  text: 2023-09-22
  day: 22
PublicationDecade 2020
PublicationPlace England
PublicationPlace_xml – name: England
– name: Oxford
PublicationTitle Briefings in bioinformatics
PublicationTitleAlternate Brief Bioinform
PublicationYear 2023
Publisher Oxford University Press
Oxford Publishing Limited (England)
Oxford University Press (OUP)
Publisher_xml – name: Oxford University Press
– name: Oxford Publishing Limited (England)
– name: Oxford University Press (OUP)
References Mangul (2023102311221560200_ref94) 2019; 17
Miyakawa (2023102311221560200_ref107) 2020; 13
Tsitoara (2023102311221560200_ref82)
Kaelin (2023102311221560200_ref150) 2017; 545
Rosenblatt (2023102311221560200_ref153) 2016; 8
JOSS Editorial Board (2023102311221560200_ref74) 2018
Han (2023102311221560200_ref148) 2017; 12
Blischak (2023102311221560200_ref72) 2016; 12
Gymrek (2023102311221560200_ref118) 2013; 339
Dyer (2023102311221560200_ref12) 2015; 350
Gorgolewski (2023102311221560200_ref29) 2016; 14
Landau (2023102311221560200_ref49) 2021; 6
Simoneau (2023102311221560200_ref132) 2021; 22
Lasser (2023102311221560200_ref66) 2020; 3
Docker Inc (2023102311221560200_ref88) 2021
Office of The Director (2023102311221560200_ref112) 2023
Krafczyk (2023102311221560200_ref28) 2021; 379
Blischak (2023102311221560200_ref143) 2019; 8
Noor (2023102311221560200_ref139) 2022; 8
Heon (2023102311221560200_ref91) 2018
Baker (2023102311221560200_ref3) 2015; 521
Oransky (2023102311221560200_ref152) 2023; 1887
Wilson (2023102311221560200_ref156) 2014; 3
Wijesooriya (2023102311221560200_ref35) 2022; 18
Ziemann (2023102311221560200_ref135) 2023
Lewis (2023102311221560200_ref17) 2016; 10
Di Tommaso (2023102311221560200_ref89) 2015; 3
McKiernan (2023102311221560200_ref104) 2016; 5
Barba (2023102311221560200_ref137) 2016; 354
Ram (2023102311221560200_ref71) 2013; 8
Posit PBC. Quarto. 2022 [cited 2023 Aug 21]. Citations & footnotes (2023102311221560200_ref62)
Donoho (2023102311221560200_ref108) 2010; 11
Khatri (2023102311221560200_ref130) 2012; 8
Wilson (2023102311221560200_ref19) 2017; 13
Beaulieu-Jones (2023102311221560200_ref20) 2017; 35
Cheifet (2023102311221560200_ref115) 2020; 21
Abramatic (2023102311221560200_ref76) 2018; 61
Mellor (2023102311221560200_ref133) 2022
Kramer (2023102311221560200_ref34) 2016; 5
Sandve (2023102311221560200_ref15) 2013; 9
Reiter (2023102311221560200_ref46) 2021; 10
Cadwallader (2023102311221560200_ref69) 2022; 10
Potti (2023102311221560200_ref11) 2011; 17
Committee on Envisioning the Data Science Discipline: The Undergraduate Perspective, Computer Science and Telecommunications Board, Board on Mathematical Sciences and Analytics, Committee on Applied and Theoretical Statistics, Division on Engineering and Physical Sciences, Board on Science Education (2023102311221560200_ref154) 2018
Kim (2023102311221560200_ref22) 2018; 7
Nekrutenko (2023102311221560200_ref131) 2012; 13
Perkel (2023102311221560200_ref58) 2018; 563
Krekel (2023102311221560200_ref142) 2004
Brito (2023102311221560200_ref24) 2020; 9
Ge (2023102311221560200_ref39) 2020; 36
United Nations Educational, Scientific and Cultural Organization (UNESCO) recommendation on open science (2023102311221560200_ref105) 2021
Nosek (2023102311221560200_ref2) 2022; 73
McInnes (2023102311221560200_ref41) 2018
Perez-Riverol (2023102311221560200_ref73) 2016; 12
Piccolo (2023102311221560200_ref16) 2016; 5
Vallet (2023102311221560200_ref84) 2022; 9
Hennessey (2023102311221560200_ref128) 2013; 14
Chacon (2023102311221560200_ref70) 2023
Beer (2023102311221560200_ref81)
Packer (2023102311221560200_ref113) 2018; 360
The Turing Way Community (2023102311221560200_ref138) 2019
Zeeberg (2023102311221560200_ref31) 2004; 5
Cadwallader (2023102311221560200_ref67) 2022; 18
Da (2023102311221560200_ref95) 2017; 33
Grüning (2023102311221560200_ref21) 2018; 6
Kurtzer (2023102311221560200_ref92) 2017; 12
Knuth (2023102311221560200_ref53) 1984; 27
Grolemund (2023102311221560200_ref55) 2017
Williams (2023102311221560200_ref155) 2023
Bonaretti (2023102311221560200_ref124) 2019
Ziemann (2023102311221560200_ref32) 2016; 17
Suetake (2023102311221560200_ref145) 2022; 12
Crusoe (2023102311221560200_ref50) 2022; 65
GNU Project Volunteers (2023102311221560200_ref98) 2022
Gabelica (2023102311221560200_ref109) 2022; 150
Abeysooriya (2023102311221560200_ref33) 2021; 17
Munafò (2023102311221560200_ref146) 2017; 1
Archmiller (2023102311221560200_ref103) 2020; 84
Hanson (2023102311221560200_ref114) 2011; 331
Byrd (2023102311221560200_ref116) 2020; 21
Cosmo (2023102311221560200_ref77) 2020
Van Der (2023102311221560200_ref40) 2008; 9
Wilkinson (2023102311221560200_ref106) 2016; 3
Kluyver T, Ragan-Kelley B, Pérez F, Fernando G, Brian B, Matthias F, Jonathan K, Kyle H, Jessica G, Jason C, Sylvain I, Paul A, Damián A, Safia W (2023102311221560200_ref79)
Maitner (2023102311221560200_ref68) 2023
Leipzig (2023102311221560200_ref45) 2016; 18
Rehm (2023102311221560200_ref119) 2021; 1
Ioannidis (2023102311221560200_ref5) 2009; 41
Tan (2023102311221560200_ref13) 2010; 11
Becht (2023102311221560200_ref42) 2018; 37
Merkel (2023102311221560200_ref90) 2014; 2014
Akalin (2023102311221560200_ref23) 2018
Panko (2023102311221560200_ref30) 1996
Garijo (2023102311221560200_ref44) 2013; 8
Xie (2023102311221560200_ref63) 2018
Heeney (2023102311221560200_ref117) 2011; 14
Begley (2023102311221560200_ref4) 2012; 483
Bauer (2023102311221560200_ref60) 2023
Strijkers (2023102311221560200_ref64) 2011; 4
Balamuta (2023102311221560200_ref102) 2023
Thorogood (2023102311221560200_ref120) 2021; 1
Edgar (2023102311221560200_ref121) 2002; 30
Benureau (2023102311221560200_ref43) 2017; 11
Roche (2023102311221560200_ref127) 2015; 13
Huang (2023102311221560200_ref149) 2013; 14
Baggerly (2023102311221560200_ref10) 2010
Seyednasrollah (2023102311221560200_ref83) 2015; 16
Voss (2023102311221560200_ref51) 2017
Potti (2023102311221560200_ref9) 2006; 12
Wurmus (2023102311221560200_ref99) 2018; 7
Reich (2023102311221560200_ref37) 2006; 38
Perkel (2023102311221560200_ref61) 2022; 603
Zaringhalam (2023102311221560200_ref6) 2020
Hurley (2023102311221560200_ref85) 2015; 16
Caetano (2023102311221560200_ref129) 2014; 98
Courtes (2023102311221560200_ref97) 2022; 24
Di Tommaso (2023102311221560200_ref52) 2017; 35
Diaba-Nuhoho (2023102311221560200_ref147) 2021; 14
Trisovic (2023102311221560200_ref8) 2022; 9
Stodden (2023102311221560200_ref18) 2016; 354
Kluyver (2023102311221560200_ref56) 2016
Peikert (2023102311221560200_ref25) 2021; 1
Community (2023102311221560200_ref36); 50
Mangul (2023102311221560200_ref93) 2019; 20
Peng (2023102311221560200_ref14) 2011; 334
Köster (2023102311221560200_ref47) 2012; 28
Leisch (2023102311221560200_ref54) 2002; 2
Perkel (2023102311221560200_ref96) 2023; 617
eLife Editorial Board (2023102311221560200_ref65) 2017
Kodama (2023102311221560200_ref122) 2012; 40
Del Sole (2023102311221560200_ref80)
Di Cosmo (2023102311221560200_ref75) 2017
Goodman (2023102311221560200_ref1) 2016; 8
Mölder (2023102311221560200_ref48) 2021; 10
Diong (2023102311221560200_ref151) 2021; 6
Rule (2023102311221560200_ref134) 2019; 15
National Health and Medical Research Council (2023102311221560200_ref111) 2019
Patel (2023102311221560200_ref136) 2023; 10
Nüst (2023102311221560200_ref87) 2020; 16
Pasquier (2023102311221560200_ref86) 2018; 20
Wickham (2023102311221560200_ref141) 2011; 3
Pampel (2023102311221560200_ref126) 2013; 8
Samuel (2023102311221560200_ref7) 2022
Powell (2023102311221560200_ref38) 2019
Kluyver (2023102311221560200_ref78) 2016
Hassenrück (2023102311221560200_ref125) 2021
Granger (2023102311221560200_ref57) 2021; 23
Martens (2023102311221560200_ref123) 2005; 5
Tuloup (2023102311221560200_ref101) 2021
Sarma (2023102311221560200_ref140) 2016; 5
Wang (2023102311221560200_ref59) 2016; 5
Perkel (2023102311221560200_ref100) 2020; 584
DuPre (2023102311221560200_ref27) 2022; 18
Brazma (2023102311221560200_ref110) 2001; 29
Kidwell (2023102311221560200_ref144) 2016; 14
Rodrigues (2023102311221560200_ref26) 2023
References_xml – volume-title: JupyterLite: Jupyter WebAssembly python
  year: 2021
  ident: 2023102311221560200_ref101
– year: 2023
  ident: 2023102311221560200_ref112
– volume: 12
  start-page: e1004947
  year: 2016
  ident: 2023102311221560200_ref73
  article-title: Ten simple rules for taking advantage of Git and GitHub
  publication-title: PLOS Comput Biol
  doi: 10.1371/journal.pcbi.1004947
– volume: 13
  start-page: 24
  year: 2020
  ident: 2023102311221560200_ref107
  article-title: No raw data, no science: another possible source of the reproducibility crisis
  publication-title: Mol Brain
  doi: 10.1186/s13041-020-0552-2
– year: 2023
  ident: 2023102311221560200_ref68
  article-title: Code sharing increases citations, but remains uncommon
  publication-title: Res Sq
– volume: 331
  start-page: 649
  year: 2011
  ident: 2023102311221560200_ref114
  article-title: Making data maximally available
  publication-title: Science
  doi: 10.1126/science.1203354
– volume: 483
  start-page: 531
  year: 2012
  ident: 2023102311221560200_ref4
  article-title: Drug development: raise standards for preclinical cancer research: drug development
  publication-title: Nature
  doi: 10.1038/483531a
– volume: 350
  start-page: h2559
  year: 2015
  ident: 2023102311221560200_ref12
  article-title: Duke university settles lawsuits alleging that patients were harmed in chemotherapy trials
  publication-title: BMJ
  doi: 10.1136/bmj.h2559
– volume-title: Beginning Git and GitHub
  ident: 2023102311221560200_ref82
  article-title: Git GUI Tools
  doi: 10.1007/978-1-4842-5313-7_15
– volume: 14
  start-page: 391
  year: 2013
  ident: 2023102311221560200_ref149
  article-title: Comparability and reproducibility of biomedical data
  publication-title: Brief Bioinform
  doi: 10.1093/bib/bbs078
– volume-title: Scientific Data Analysis Pipelines and Reproducibility
  year: 2018
  ident: 2023102311221560200_ref23
– start-page: 87
  volume-title: Positioning and Power in Academic Publishing: Players, Agents and Agendas
  year: 2016
  ident: 2023102311221560200_ref56
– volume: 8
  start-page: e78080
  year: 2013
  ident: 2023102311221560200_ref126
  article-title: Making research data repositories visible: the re3data.org registry
  publication-title: PloS One
  doi: 10.1371/journal.pone.0078080
– volume: 5
  start-page: 5
  year: 2016
  ident: 2023102311221560200_ref104
  article-title: How open science helps researchers succeed
  publication-title: Elife
  doi: 10.7554/eLife.16800
– volume: 12
  start-page: 1294
  year: 2006
  ident: 2023102311221560200_ref9
  article-title: Genomic signatures to guide the use of chemotherapeutics
  publication-title: Nat Med
  doi: 10.1038/nm1491
– volume: 6
  start-page: 631
  year: 2018
  ident: 2023102311221560200_ref21
  article-title: Practical computational reproducibility in the life sciences
  publication-title: Cell Syst
  doi: 10.1016/j.cels.2018.03.014
– volume: 10
  start-page: 33
  year: 2021
  ident: 2023102311221560200_ref48
  article-title: Sustainable data analysis with snakemake
  publication-title: F1000Res
  doi: 10.12688/f1000research.29032.1
– volume: 4
  start-page: 707
  year: 2011
  ident: 2023102311221560200_ref64
  article-title: Toward executable scientific publications
  publication-title: Procedia Comput Sci
  doi: 10.1016/j.procs.2011.04.074
– volume: 379
  year: 2021
  ident: 2023102311221560200_ref28
  article-title: Learning from reproducing computational results: introducing three principles and the reproduction package
  publication-title: Philos Trans A Math Phys Eng Sci
– volume-title: Proceedings of the 14th International Conference on Digital Preservation, iPRES 2017, Japan
  year: 2017
  ident: 2023102311221560200_ref75
– volume: 50
  start-page: 8999
  issue: 50
  ident: 2023102311221560200_ref36
  article-title: The galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2022 update
  publication-title: Nucleic Acids Res
– volume: 35
  start-page: 316
  year: 2017
  ident: 2023102311221560200_ref52
  article-title: Nextflow enables reproducible computational workflows
  publication-title: Nat Biotechnol
  doi: 10.1038/nbt.3820
– volume: 22
  start-page: 140
  year: 2021
  ident: 2023102311221560200_ref132
  article-title: Current RNA-seq methodology reporting limits reproducibility
  publication-title: Brief Bioinform
  doi: 10.1093/bib/bbz124
– volume: 334
  start-page: 1226
  year: 2011
  ident: 2023102311221560200_ref14
  article-title: Reproducible research in computational science
  publication-title: Science
  doi: 10.1126/science.1213847
– volume: 9
  start-page: 60
  year: 2022
  ident: 2023102311221560200_ref8
  article-title: A large-scale study on research code quality and execution
  publication-title: Sci Data
  doi: 10.1038/s41597-022-01143-6
– volume: 27
  start-page: 97
  year: 1984
  ident: 2023102311221560200_ref53
  article-title: Literate programming
  publication-title: Comput J
  doi: 10.1093/comjnl/27.2.97
– volume-title: Positioning and Power in Academic Publishing: Players, Agents and Agendas
  ident: 2023102311221560200_ref79
  article-title: Jupyter Notebooks – a publishing format for reproducible computational workflows
– volume-title: Pytest 7.4
  year: 2004
  ident: 2023102311221560200_ref142
– ident: 2023102311221560200_ref62
– year: 2021
  ident: 2023102311221560200_ref105
– volume: 18
  start-page: e1009935
  year: 2022
  ident: 2023102311221560200_ref35
  article-title: Urgent need for consistent standards in functional enrichment analysis
  publication-title: PLoS Comput Biol
  doi: 10.1371/journal.pcbi.1009935
– volume: 1
  start-page: 100029
  year: 2021
  ident: 2023102311221560200_ref119
  article-title: GA4GH: international policies and standards for data sharing across genomic research and healthcare
  publication-title: Cell Genom
  doi: 10.1016/j.xgen.2021.100029
– volume: 12
  start-page: e0177459
  year: 2017
  ident: 2023102311221560200_ref92
  article-title: Singularity: scientific containers for mobility of compute
  publication-title: PloS One
  doi: 10.1371/journal.pone.0177459
– volume: 360
  start-page: k510
  year: 2018
  ident: 2023102311221560200_ref113
  article-title: Data sharing in medical research
  publication-title: BMJ
  doi: 10.1136/bmj.k510
– volume-title: What is a Container?
  year: 2021
  ident: 2023102311221560200_ref88
– start-page: 425
  volume-title: R for Data Science
  year: 2017
  ident: 2023102311221560200_ref55
– volume: 14
  year: 2021
  ident: 2023102311221560200_ref147
  article-title: Reproducibility and research integrity: the role of scientists and institutions
  publication-title: BMC Res Notes
  doi: 10.1186/s13104-021-05875-3
– volume: 9
  start-page: 2579
  year: 2008
  ident: 2023102311221560200_ref40
  article-title: Visualizing data using t-SNE
  publication-title: J Mach Learn Res
– year: 2023
  ident: 2023102311221560200_ref60
  article-title: Writing a reproducible paper with RStudio and quarto
  publication-title: OSF Preprints
– volume: 6
  start-page: 2959
  year: 2021
  ident: 2023102311221560200_ref49
  article-title: The targets r package: a dynamic make-like function-oriented pipeline toolkit for reproducibility and high-performance computing
  publication-title: J Open Source Softw
  doi: 10.21105/joss.02959
– volume: 12
  year: 2016
  ident: 2023102311221560200_ref72
  article-title: A quick introduction to version control with git and GitHub
  publication-title: PLoS Comput Biol
  doi: 10.1371/journal.pcbi.1004668
– volume: 5
  start-page: 3537
  year: 2005
  ident: 2023102311221560200_ref123
  article-title: PRIDE: the proteomics identifications database
  publication-title: Proteomics
  doi: 10.1002/pmic.200401303
– volume: 3
  start-page: 1
  year: 2020
  ident: 2023102311221560200_ref66
  article-title: Creating an executable paper is a journey through open science
  publication-title: Commun Phys
  doi: 10.1038/s42005-020-00403-4
– volume: 38
  start-page: 500
  year: 2006
  ident: 2023102311221560200_ref37
  article-title: GenePattern 2.0
  publication-title: Nat Genet
  doi: 10.1038/ng0506-500
– volume: 28
  start-page: 2520
  year: 2012
  ident: 2023102311221560200_ref47
  article-title: Snakemake—a scalable bioinformatics workflow engine
  publication-title: Bioinformatics
  doi: 10.1093/bioinformatics/bts480
– volume: 3
  start-page: 5
  year: 2011
  ident: 2023102311221560200_ref141
  article-title: Testthat: get started with testing
  publication-title: The R Journal
  doi: 10.32614/RJ-2011-002
– volume: 6
  start-page: 11
  year: 2021
  ident: 2023102311221560200_ref151
  article-title: Strengthening the incentives for responsible research practices in Australian health and medical research funding
  publication-title: Res Integr Peer Rev
  doi: 10.1186/s41073-021-00113-7
– volume: 8
  start-page: e80278
  year: 2013
  ident: 2023102311221560200_ref44
  article-title: Quantifying reproducibility in computational biology: the case of the tuberculosis drugome
  publication-title: PloS One
  doi: 10.1371/journal.pone.0080278
– volume: 1887
  year: 2023
  ident: 2023102311221560200_ref152
  article-title: How bibliometrics and school rankings reward unreliable science
  publication-title: BMJ
– volume: 17
  start-page: e1008984
  year: 2021
  ident: 2023102311221560200_ref33
  article-title: Gene name errors: lessons not learned
  publication-title: PLoS Comput Biol
  doi: 10.1371/journal.pcbi.1008984
– year: 2022
  ident: 2023102311221560200_ref7
  article-title: Computational reproducibility of Jupyter notebooks from biomedical publications
– volume: 10
  start-page: e13933
  year: 2022
  ident: 2023102311221560200_ref69
  article-title: A survey of researchers’ code sharing and code reuse practices, and assessment of interactive notebook prototypes
  publication-title: PeerJ
  doi: 10.7717/peerj.13933
– year: 2018
  ident: 2023102311221560200_ref41
  article-title: UMAP: uniform manifold approximation and projection for dimension reduction
– volume: 8
  start-page: e839
  year: 2022
  ident: 2023102311221560200_ref139
  article-title: Improving bioinformatics software quality through incorporation of software engineering practices
  publication-title: PeerJ Comput Sci
  doi: 10.7717/peerj-cs.839
– volume: 17
  start-page: 177
  year: 2016
  ident: 2023102311221560200_ref32
  article-title: Gene name errors are widespread in the scientific literature
  publication-title: Genome Biol
  doi: 10.1186/s13059-016-1044-7
– volume: 3
  start-page: 62
  year: 2014
  ident: 2023102311221560200_ref156
  article-title: Software carpentry: lessons learned
  publication-title: F1000Res
  doi: 10.12688/f1000research.3-62.v1
– volume: 10
  start-page: 52
  year: 2016
  ident: 2023102311221560200_ref17
  article-title: Where next for the reproducibility agenda in computational biology?
  publication-title: BMC Syst Biol
  doi: 10.1186/s12918-016-0288-x
– volume: 11
  start-page: 69
  year: 2017
  ident: 2023102311221560200_ref43
  article-title: Re-run, repeat, reproduce, reuse, replicate: transforming code into scientific contributions
  publication-title: Front Neuroinform
  doi: 10.3389/fninf.2017.00069
– volume-title: Visual Studio Code Distilled
  ident: 2023102311221560200_ref80
  article-title: Introducing Visual Studio Code
– volume: 13
  start-page: e1005510
  year: 2017
  ident: 2023102311221560200_ref19
  article-title: Good enough practices in scientific computing
  publication-title: PLoS Comput Biol
  doi: 10.1371/journal.pcbi.1005510
– volume: 20
  start-page: 47
  year: 2019
  ident: 2023102311221560200_ref93
  article-title: Improving the usability and archival stability of bioinformatics software
  publication-title: Genome Biol
  doi: 10.1186/s13059-019-1649-8
– volume: 98
  start-page: 1
  year: 2014
  ident: 2023102311221560200_ref129
  article-title: Forgotten treasures: the fate of data in animal behaviour studies
  publication-title: Anim Behav
  doi: 10.1016/j.anbehav.2014.09.025
– volume: 14
  year: 2016
  ident: 2023102311221560200_ref29
  article-title: A practical guide for improving transparency and reproducibility in neuroimaging research
  publication-title: PLoS Biol
  doi: 10.1371/journal.pbio.1002506
– volume: 18
  start-page: e1010193
  year: 2022
  ident: 2023102311221560200_ref67
  article-title: Advancing code sharing in the computational biology community
  publication-title: PLoS Comput Biol
  doi: 10.1371/journal.pcbi.1010193
– volume-title: Editorial guide
  year: 2018
  ident: 2023102311221560200_ref74
– volume: 3
  year: 2015
  ident: 2023102311221560200_ref89
  article-title: The impact of docker containers on the performance of genomic pipelines
  publication-title: PeerJ
  doi: 10.7717/peerj.1273
– volume: 41
  start-page: 149
  year: 2009
  ident: 2023102311221560200_ref5
  article-title: Repeatability of published microarray gene expression analyses
  publication-title: Nat Genet
  doi: 10.1038/ng.295
– volume: 33
  start-page: 2580
  year: 2017
  ident: 2023102311221560200_ref95
  article-title: BioContainers: an open-source and community-driven framework for software standardization
  publication-title: Bioinformatics
  doi: 10.1093/bioinformatics/btx192
– volume: 5
  start-page: 80
  year: 2004
  ident: 2023102311221560200_ref31
  article-title: Mistaken identifiers: gene name errors can be introduced inadvertently when using excel in bioinformatics
  publication-title: BMC Bioinformatics
  doi: 10.1186/1471-2105-5-80
– year: 2023
  ident: 2023102311221560200_ref26
  article-title: Building reproducible analytical pipelines with R
  publication-title: Leanpub
– start-page: 19
  volume-title: Pro git
  year: 2023
  ident: 2023102311221560200_ref70
– volume: 30
  start-page: 207
  year: 2002
  ident: 2023102311221560200_ref121
  article-title: Gene expression omnibus: NCBI gene expression and hybridization array data repository
  publication-title: Nucleic Acids Res
  doi: 10.1093/nar/30.1.207
– volume: 12
  start-page: 12
  year: 2022
  ident: 2023102311221560200_ref145
  article-title: A workflow reproducibility scale for automatic validation of biological interpretation results
  publication-title: Gigascience
  doi: 10.1093/gigascience/giad031
– volume: 24
  start-page: 77
  year: 2022
  ident: 2023102311221560200_ref97
  article-title: Reproducibility and performance: why choose?
  publication-title: Comput Sci Eng
  doi: 10.1109/MCSE.2022.3165626
– volume: 584
  start-page: 656
  year: 2020
  ident: 2023102311221560200_ref100
  article-title: Challenge to scientists: does your ten-year-old code still run?
  publication-title: Nature
  doi: 10.1038/d41586-020-02462-7
– volume: 11
  start-page: 385
  year: 2010
  ident: 2023102311221560200_ref108
  article-title: An invitation to reproducible computational research
  publication-title: Biostatistics
  doi: 10.1093/biostatistics/kxq028
– volume: 354
  start-page: 142
  year: 2016
  ident: 2023102311221560200_ref137
  article-title: The hard road to reproducibility
  publication-title: Science
  doi: 10.1126/science.354.6308.142
– volume: 18
  start-page: bbw020
  year: 2016
  ident: 2023102311221560200_ref45
  article-title: A review of bioinformatic pipeline frameworks
  publication-title: Brief Bioinform
  doi: 10.1093/bib/bbw020
– volume: 5
  start-page: 1946
  year: 2016
  ident: 2023102311221560200_ref140
  article-title: Unit testing, model validation, and biological simulation
  publication-title: F1000Res
  doi: 10.12688/f1000research.9315.1
– volume: 8
  start-page: 1749
  year: 2019
  ident: 2023102311221560200_ref143
  article-title: Creating and sharing reproducible research code the workflowr way
  publication-title: F1000Res
  doi: 10.12688/f1000research.20843.1
– volume: 37
  start-page: 38
  year: 2018
  ident: 2023102311221560200_ref42
  article-title: Dimensionality reduction for visualizing single-cell data using UMAP
  publication-title: Nat Biotechnol
  doi: 10.1038/nbt.4314
– volume: 36
  start-page: 2628
  year: 2020
  ident: 2023102311221560200_ref39
  article-title: ShinyGO: a graphical gene-set enrichment tool for animals and plants
  publication-title: Bioinformatics
  doi: 10.1093/bioinformatics/btz931
– volume: 9
  start-page: 597
  year: 2022
  ident: 2023102311221560200_ref84
  article-title: Toward practical transparent verifiable and long-term reproducible research using Guix
  publication-title: Sci Data.
  doi: 10.1038/s41597-022-01720-9
– volume: 9
  start-page: e1003285
  year: 2013
  ident: 2023102311221560200_ref15
  article-title: Ten simple rules for reproducible computational research
  publication-title: PLoS Comput Biol
  doi: 10.1371/journal.pcbi.1003285
– volume: 23
  start-page: 7
  year: 2021
  ident: 2023102311221560200_ref57
  article-title: Jupyter: thinking and storytelling with code and data
  publication-title: Comput Sci Eng
  doi: 10.1109/MCSE.2021.3059263
– volume-title: Drpowell/degust 4.1.1
  year: 2019
  ident: 2023102311221560200_ref38
– volume: 563
  start-page: 145
  year: 2018
  ident: 2023102311221560200_ref58
  article-title: Why Jupyter is data scientists’ computational notebook of choice
  publication-title: Nature
  doi: 10.1038/d41586-018-07196-1
– volume: 21
  start-page: 96
  year: 2020
  ident: 2023102311221560200_ref115
  article-title: Open data in a deeply connected world
  publication-title: Genome Biol
  doi: 10.1186/s13059-020-02010-6
– volume: 8
  start-page: 8
  year: 2016
  ident: 2023102311221560200_ref153
  article-title: An incentive-based approach for improving data reproducibility
  publication-title: Sci Transl Med
  doi: 10.1126/scitranslmed.aaf5003
– volume: 29
  start-page: 365
  year: 2001
  ident: 2023102311221560200_ref110
  article-title: Minimum information about a microarray experiment (MIAME)-toward standards for microarray data
  publication-title: Nat Genet
  doi: 10.1038/ng1201-365
– volume: 8
  start-page: 1
  year: 2016
  ident: 2023102311221560200_ref1
  article-title: What does research reproducibility mean?
  publication-title: Sci Transl Med
  doi: 10.1126/scitranslmed.aaf5027
– volume: 5
  start-page: 692
  year: 2016
  ident: 2023102311221560200_ref34
  article-title: Innovations in scholarly communication—global survey on research tool usage
  publication-title: F1000Res
  doi: 10.12688/f1000research.8414.1
– volume: 13
  start-page: e1002295
  year: 2015
  ident: 2023102311221560200_ref127
  article-title: Public data archiving in ecology and evolution: how well are we doing?
  publication-title: PLoS Biol
  doi: 10.1371/journal.pbio.1002295
– year: 2023
  ident: 2023102311221560200_ref155
  article-title: Optimizing short-format training: an international consensus on effective, inclusive, and career-spanning professional development in the life sciences and beyond
  publication-title: bioRxiv
– volume: 1
  start-page: 100032
  year: 2021
  ident: 2023102311221560200_ref120
  article-title: International federation of genomic medicine databases using GA4GH standards
  publication-title: Cell Genom
  doi: 10.1016/j.xgen.2021.100032
– volume: 5
  start-page: 30
  year: 2016
  ident: 2023102311221560200_ref16
  article-title: Tools and techniques for computational reproducibility
  publication-title: Gigascience.
  doi: 10.1186/s13742-016-0135-4
– year: 2022
  ident: 2023102311221560200_ref133
  article-title: MDAR: towards minimum reporting standards for life scientists
  publication-title: Open Science Framework
– volume: 10
  start-page: 557
  year: 2023
  ident: 2023102311221560200_ref136
  article-title: Making biomedical research software FAIR: actionable step-by-step guidelines with a user-support tool
  publication-title: Sci Data
  doi: 10.1038/s41597-023-02463-x
– volume: 603
  start-page: 191
  year: 2022
  ident: 2023102311221560200_ref61
  article-title: Cut the tyranny of copy-and-paste with these coding tools
  publication-title: Nature
  doi: 10.1038/d41586-022-00563-z
– volume: 17
  start-page: e3000333
  year: 2019
  ident: 2023102311221560200_ref94
  article-title: Challenges and recommendations to improve the installability and archival stability of omics computational tools
  publication-title: PLoS Biol
  doi: 10.1371/journal.pbio.3000333
– year: 2019
  ident: 2023102311221560200_ref138
  article-title: The Turing Way: a handbook for reproducible data science
  publication-title: Zenodo
– volume: 61
  start-page: 29
  year: 2018
  ident: 2023102311221560200_ref76
  article-title: Building the universal archive of source code. ACM, editor
  publication-title: Commun ACM
  doi: 10.1145/3183558
– volume: 16
  year: 2020
  ident: 2023102311221560200_ref87
  article-title: Ten simple rules for writing dockerfiles for reproducible data science
  publication-title: PLoS Comput Biol
  doi: 10.1371/journal.pcbi.1008316
– volume: 339
  start-page: 321
  year: 2013
  ident: 2023102311221560200_ref118
  article-title: Identifying personal genomes by surname inference
  publication-title: Science
  doi: 10.1126/science.1229566
– volume: 7
  start-page: giy123
  year: 2018
  ident: 2023102311221560200_ref99
  article-title: PiGx: reproducible genomics analysis pipelines with GNU guix
  publication-title: Gigascience.
  doi: 10.1093/gigascience/giy123
– volume: 8
  start-page: e1002375
  year: 2012
  ident: 2023102311221560200_ref130
  article-title: Ten years of pathway analysis: current approaches and outstanding challenges
  publication-title: PLoS Comput Biol
  doi: 10.1371/journal.pcbi.1002375
– volume: 35
  start-page: 342
  year: 2017
  ident: 2023102311221560200_ref20
  article-title: Reproducibility of computational workflows is automated using continuous analysis
  publication-title: Nat Biotechnol
  doi: 10.1038/nbt.3780
– volume: 8
  start-page: 7
  year: 2013
  ident: 2023102311221560200_ref71
  article-title: Git can facilitate greater reproducibility and increased transparency in science
  publication-title: Source Code Biol Med
  doi: 10.1186/1751-0473-8-7
– year: 2019
  ident: 2023102311221560200_ref124
  publication-title: Two real use cases of FAIR maturity indicators in the life sciences
– volume: 5
  start-page: 1574
  year: 2016
  ident: 2023102311221560200_ref59
  article-title: An open RNA-Seq data analysis pipeline tutorial with an example of reprocessing data from a recent Zika virus study
  publication-title: F1000Research
  doi: 10.12688/f1000research.9110.1
– volume: 1
  start-page: e3763
  year: 2021
  ident: 2023102311221560200_ref25
  article-title: A reproducible data analysis workflow with r markdown, git, make, and docker
  publication-title: Quant Comput Methods Behav Sci
  doi: 10.5964/qcmb.3763
– volume-title: Quarto-webr: Quarto Extension to Embed WebR for HTML Documents
  year: 2023
  ident: 2023102311221560200_ref102
– year: 2018
  ident: 2023102311221560200_ref91
  article-title: Podman—A tool for managing OCI containers and pods
  publication-title: Zenodo
– volume: 16
  start-page: 59
  year: 2015
  ident: 2023102311221560200_ref83
  article-title: Comparison of software packages for detecting differential expression in RNA-seq studies
  publication-title: Brief Bioinform
  doi: 10.1093/bib/bbt086
– volume: 14
  start-page: 17
  year: 2011
  ident: 2023102311221560200_ref117
  article-title: Assessing the privacy risks of data sharing in genomics
  publication-title: Public Health Genomics
  doi: 10.1159/000294150
– volume: 545
  start-page: 387
  year: 2017
  ident: 2023102311221560200_ref150
  article-title: Publish houses of brick, not mansions of straw
  publication-title: Nature
  doi: 10.1038/545387a
– volume: 354
  start-page: 1240
  year: 2016
  ident: 2023102311221560200_ref18
  article-title: Enhancing reproducibility for computational methods
  publication-title: Science
  doi: 10.1126/science.aah6168
– volume: 2014
  start-page: 2
  year: 2014
  ident: 2023102311221560200_ref90
  article-title: Docker: lightweight linux containers for consistent development and deployment
  publication-title: Linux J
– volume: 14
  start-page: e1002456
  year: 2016
  ident: 2023102311221560200_ref144
  article-title: Badges to acknowledge open practices: A simple, low-cost, effective method for increasing transparency
  publication-title: PLoS Biol
  doi: 10.1371/journal.pbio.1002456
– volume: 1
  year: 2017
  ident: 2023102311221560200_ref146
  article-title: A manifesto for reproducible science
  publication-title: Nat Hum Behav
  doi: 10.1038/s41562-016-0021
– volume: 17
  start-page: 135
  year: 2011
  ident: 2023102311221560200_ref11
  article-title: Retraction note: genomic signatures to guide the use of chemotherapeutics
  publication-title: Nat Med
  doi: 10.1038/nm0111-135
– volume: 14
  start-page: S5
  issue: Suppl 14
  year: 2013
  ident: 2023102311221560200_ref128
  article-title: A cross disciplinary study of link decay and the effectiveness of mitigation techniques
  publication-title: BMC Bioinformatics
  doi: 10.1186/1471-2105-14-S14-S5
– volume: 12
  start-page: e0183591
  year: 2017
  ident: 2023102311221560200_ref148
  article-title: A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review
  publication-title: PloS One
  doi: 10.1371/journal.pone.0183591
– volume: 521
  start-page: 17
  year: 2015
  ident: 2023102311221560200_ref3
  article-title: First results from psychology’s largest reproducibility test
  publication-title: Nature
– year: 2017
  ident: 2023102311221560200_ref51
  article-title: Full-stack genomics pipelining with GATK4+ WDL+ Cromwell
  publication-title: F1000Research
– volume: 84
  start-page: 1012
  year: 2020
  ident: 2023102311221560200_ref103
  article-title: Computational reproducibility in the wildlife society’s flagship journals
  publication-title: J Wildl Manage
  doi: 10.1002/jwmg.21855
– ident: 2023102311221560200_ref81
  article-title: Introducing Github: A Non-Technical Guide
– volume: 16
  start-page: 901
  year: 2015
  ident: 2023102311221560200_ref85
  article-title: Virtual reference environments: a simple way to make research reproducible
  publication-title: Brief Bioinform
  doi: 10.1093/bib/bbu043
– start-page: 362
  volume-title: Archiving and Referencing Source Code with Software Heritage
  year: 2020
  ident: 2023102311221560200_ref77
– start-page: 326
  volume-title: Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences
  year: 1996
  ident: 2023102311221560200_ref30
  doi: 10.1109/HICSS.1996.495416
– volume: 65
  start-page: 54
  year: 2022
  ident: 2023102311221560200_ref50
  article-title: Methods included: standardizing computational reuse and portability with the common workflow language
  publication-title: Commun ACM
  doi: 10.1145/3486897
– volume: 9
  start-page: 9
  year: 2020
  ident: 2023102311221560200_ref24
  article-title: Recommendations to enhance rigor and reproducibility in biomedical research
  publication-title: Gigascience
– volume: 20
  start-page: 111
  year: 2018
  ident: 2023102311221560200_ref86
  article-title: Sharing and preserving computational analyses for posterity with encapsulator
  publication-title: Comput Sci Eng
  doi: 10.1109/MCSE.2018.042781334
– volume: 21
  start-page: 615
  year: 2020
  ident: 2023102311221560200_ref116
  article-title: Responsible, practical genomic data sharing that accelerates research
  publication-title: Nat Rev Genet
  doi: 10.1038/s41576-020-0257-5
– volume: 10
  year: 2021
  ident: 2023102311221560200_ref46
  article-title: Streamlining data-intensive biology with workflow systems
  publication-title: Gigascience
  doi: 10.1093/gigascience/giaa140
– volume: 13
  start-page: 667
  year: 2012
  ident: 2023102311221560200_ref131
  article-title: Next-generation sequencing data interpretation: enhancing reproducibility and accessibility
  publication-title: Nat Rev Genet
  doi: 10.1038/nrg3305
– volume-title: Data Science for Undergraduates: Opportunities and Options
  year: 2018
  ident: 2023102311221560200_ref154
  doi: 10.17226/25104
– volume: 73
  start-page: 719
  year: 2022
  ident: 2023102311221560200_ref2
  article-title: Replicability, robustness, and reproducibility in psychological science
  publication-title: Annu Rev Psychol
  doi: 10.1146/annurev-psych-020821-114157
– start-page: 1309
  volume-title: Ann Appl Stat
  year: 2010
  ident: 2023102311221560200_ref10
  article-title: Deriving chemosensitivity from cell lines: forensic bioinformatics and reproducible research in high-throughput biology
– volume: 150
  start-page: 33
  year: 2022
  ident: 2023102311221560200_ref109
  article-title: Many researchers were not compliant with their published data sharing statement: A mixed-methods study
  publication-title: J Clin Epidemiol
  doi: 10.1016/j.jclinepi.2022.05.019
– volume: 11
  start-page: S27
  issue: Suppl 4
  year: 2010
  ident: 2023102311221560200_ref13
  article-title: Advancing standards for bioinformatics activities: persistence, reproducibility, disambiguation and minimum information about a bioinformatics investigation (MIABi)
  publication-title: BMC Genomics
  doi: 10.1186/1471-2164-11-S4-S27
– volume: 3
  year: 2016
  ident: 2023102311221560200_ref106
  article-title: The FAIR guiding principles for scientific data management and stewardship
  publication-title: Sci Data
  doi: 10.1038/sdata.2016.18
– volume-title: Management of Data and Information in Research: A Guide Supporting the Australian Code for the Responsible Conduct of Research
  year: 2019
  ident: 2023102311221560200_ref111
– year: 2020
  ident: 2023102311221560200_ref6
  article-title: Data and code for reproducible research: lessons learned from the NLM reproducibility workshop
  publication-title: Zenodo
– start-page: 87
  volume-title: Jupyter Notebooks—A Publishing Format for Reproducible Computational Workflows
  year: 2016
  ident: 2023102311221560200_ref78
– year: 2021
  ident: 2023102311221560200_ref125
  article-title: FAIR enough? A perspective on the status of nucleotide sequence data and metadata on public archives
  publication-title: bioRxiv
– volume: 18
  start-page: e1009651
  year: 2022
  ident: 2023102311221560200_ref27
  article-title: Beyond advertising: new infrastructures for publishing integrated research objects
  publication-title: PLoS Comput Biol
  doi: 10.1371/journal.pcbi.1009651
– volume: 40
  start-page: D54
  year: 2012
  ident: 2023102311221560200_ref122
  article-title: International Nucleotide Sequence Database Collaboration. The sequence read archive: explosive growth of sequencing data
  publication-title: Nucleic Acids Res
  doi: 10.1093/nar/gkr854
– volume: 2
  start-page: 28
  year: 2002
  ident: 2023102311221560200_ref54
  article-title: The R journal: Sweave, part i: mixing R and LaTeX
  publication-title: R News
– year: 2017
  ident: 2023102311221560200_ref65
  article-title: Innovation: collaborating to devise the future executable article
  publication-title: eLife
– volume-title: A Recipe for Extremely Reproducible Enrichment Analysis v2. protocols.io
  year: 2023
  ident: 2023102311221560200_ref135
– volume: 7
  year: 2018
  ident: 2023102311221560200_ref22
  article-title: Experimenting with reproducibility: a case study of robustness in bioinformatics
  publication-title: Gigascience
  doi: 10.1093/gigascience/giy077
– volume-title: GNU Guix Reference Manual
  year: 2022
  ident: 2023102311221560200_ref98
– volume: 617
  start-page: 212
  year: 2023
  ident: 2023102311221560200_ref96
  article-title: The sleight-of-hand trick that can simplify scientific computing
  publication-title: Nature
  doi: 10.1038/d41586-023-01469-0
– volume: 15
  start-page: e1007007
  year: 2019
  ident: 2023102311221560200_ref134
  article-title: Ten simple rules for writing and sharing computational analyses in Jupyter notebooks
  publication-title: PLoS Comput Biol
  doi: 10.1371/journal.pcbi.1007007
– volume-title: R Markdown: The Definitive Guide
  year: 2018
  ident: 2023102311221560200_ref63
  doi: 10.1201/9781138359444
SSID ssj0020781
Score 2.4780602
SecondaryResourceType review_article
Snippet Abstract Computational reproducibility is a simple premise in theory, but is difficult to achieve in practice. Building upon past efforts and proposals to...
Computational reproducibility is a simple premise in theory, but is difficult to achieve in practice. Building upon past efforts and proposals to maximize...
Abstract Computational reproducibility is a simple premise in theory, but is difficult to achieve in practice. Building upon past efforts and proposals to...
SourceID pubmedcentral
hal
proquest
pubmed
crossref
oup
SourceType Open Access Repository
Aggregation Database
Index Database
Enrichment Source
Publisher
StartPage 1
SubjectTerms Bioinformatics
Computational Biology
Computer applications
Computer Science
Information Dissemination
Reproducibility
Reproducibility of Results
Review
Software
Version control
Title The five pillars of computational reproducibility: bioinformatics and beyond
URI https://www.ncbi.nlm.nih.gov/pubmed/37870287
https://www.proquest.com/docview/2955306833
https://www.proquest.com/docview/2880822552
https://hal.science/hal-04255776
https://pubmed.ncbi.nlm.nih.gov/PMC10591307
Volume 24
WOSCitedRecordID wos001136371500043&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVASL
  databaseName: Oxford Journals Open Access Collection
  customDbUrl:
  eissn: 1477-4054
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0020781
  issn: 1467-5463
  databaseCode: TOX
  dateStart: 20000101
  isFulltext: true
  titleUrlDefault: https://academic.oup.com/journals/
  providerName: Oxford University Press
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LS8QwEB5UFLz4ftRnFE9C0SZp03gTUTws6kFlb6VJG1yQruyugv_emTYWK6Je27QNyWTmSzPzfQBHwupI8lMXah2rUFoVhdrlJnSCE1sUT3id8v_YUzc3ab-v73yC7PiHI3wtTszAnBiTF0JRLXkUpyRUcH_bb_dVxFfTFBGpkNjdfRnet2c7gWf6idIeOyVtX5Dl9wTJLxHnavG_fV2CBY8p2XljBMswVVYrMNeoTL6vQg9NgTn0auyFJIZGYzZ0zNZqDv5PICNqS2J-bVJl38-YGQw9pSrROLO8Kpipa13W4OHq8v7iOvQiCqGVKp6EiUstggTLBSIXkyqVW4NBH4OQKoSNjLYYJqWwopAIZWzpyoTE7XSqjHTGlWIdZqphVW4C04qbUkkiyI-lEAlCHdytcVcYKd2plQEcf45wZj3DOAldPGfNSbfIcHgyPzwBHLWNXxpijZ-bHeJUtS2IDPv6vJfRNXI3sVLJWxTAPs7k76_Z-ZzlzC_SccY1aSYlqRABHLS3cXnRmUlelcNXbIP-DTFUHPMANhqjaL8jyNnhjjOAtGMund5271SDp5rCm1Atoge19WfPt2GeFO4pRYXzHZiZjF7LXZi1b5PBeLQH06qf7tXL4QPZsAWE
linkProvider Oxford University Press
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+five+pillars+of+computational+reproducibility%3A+bioinformatics+and+beyond&rft.jtitle=Briefings+in+bioinformatics&rft.au=Ziemann%2C+Mark&rft.au=Poulain%2C+Pierre&rft.au=Bora%2C+Anusuiya&rft.date=2023-09-22&rft.pub=Oxford+University+Press+%28OUP%29&rft.issn=1467-5463&rft.eissn=1477-4054&rft.volume=24&rft.issue=6&rft.spage=1&rft.epage=13&rft_id=info:doi/10.1093%2Fbib%2Fbbad375&rft.externalDBID=HAS_PDF_LINK&rft.externalDocID=oai%3AHAL%3Ahal-04255776v1
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1467-5463&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1467-5463&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1467-5463&client=summon