Repeatability of published microarray gene expression analyses - Ioannidis, John P A; Allison, David B; Ball, Catherine A; Coulibaly, Issa; Cui, Xiangqin; Culhane, Aedín C; Falchi, Mario; Furlanello, Cesare; Game, Laurence; Jurman, Giuseppe; Mangion, Jon; Mehta, Tapan; et al. (Nature genetics, 2009)
Four teams of analysts attempted exact reproduction of results of 18 microarray experiments published in the journal in 2005–2006 using the data and analytical methods detailed in the original publications. In addition to MIAME criteria, the authors recommend publication of an explicit record of the analytical protocols used. Given the complexity of microarray-based gene expression studies, guidelines encourage transparent design and public data availability. Several journals require public data deposition and several public databases exist. However, not all data are publicly available, and even when available, it is unknown whether the published results are reproducible by independent scientists. Here we evaluated the replication of data analyses in 18 articles on microarray-based gene expression profiling published in Nature Genetics in 2005–2006. One table or figure from each article was independently evaluated by two teams of analysts. We reproduced two analyses in principle and six partially or with some discrepancies; ten could not be reproduced. The main reason for failure to reproduce was data unavailability, and discrepancies were mostly due to incomplete data annotation or specification of data processing and analysis. Repeatability of published microarray studies is apparently limited. More strict publication rules enforcing public data availability and explicit description of data processing and analysis should be considered.