Cross-validation failure: Small sample sizes lead to large error bars

Predictive models ground many state-of-the-art developments in statistical brain image analysis: decoding, MVPA, searchlight, or extraction of biomarkers. The principled approach to establish their validity and usefulness is cross-validation, testing prediction on unseen data. Here, I would like to...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:NeuroImage (Orlando, Fla.) Jg. 180; H. Pt A; S. 68 - 77
1. Verfasser: Varoquaux, Gaël
Format: Journal Article
Sprache:Englisch
Veröffentlicht: United States Elsevier Inc 15.10.2018
Elsevier Limited
Elsevier
Schlagworte:
ISSN:1053-8119, 1095-9572, 1095-9572
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract Predictive models ground many state-of-the-art developments in statistical brain image analysis: decoding, MVPA, searchlight, or extraction of biomarkers. The principled approach to establish their validity and usefulness is cross-validation, testing prediction on unseen data. Here, I would like to raise awareness on error bars of cross-validation, which are often underestimated. Simple experiments show that sample sizes of many neuroimaging studies inherently lead to large error bars, eg±10% for 100 samples. The standard error across folds strongly underestimates them. These large error bars compromise the reliability of conclusions drawn with predictive models, such as biomarkers or methods developments where, unlike with cognitive neuroimaging MVPA approaches, more samples cannot be acquired by repeating the experiment across many subjects. Solutions to increase sample size must be investigated, tackling possible increases in heterogeneity of the data.
AbstractList Predictive models ground many state-of-the-art developments in statistical brain image analysis: decoding, MVPA, searchlight, or extraction of biomarkers. The principled approach to establish their validity and usefulness is cross-validation, testing prediction on unseen data. Here, I would like to raise awareness on error bars of cross-validation, which are often underestimated. Simple experiments show that sample sizes of many neuroimaging studies inherently lead to large error bars, eg±10% for 100 samples. The standard error across folds strongly underestimates them. These large error bars compromise the reliability of conclusions drawn with predictive models, such as biomarkers or methods developments where, unlike with cognitive neuroimaging MVPA approaches, more samples cannot be acquired by repeating the experiment across many subjects. Solutions to increase sample size must be investigated, tackling possible increases in heterogeneity of the data.
Predictive models ground many state-of-the-art developments in statistical brain image analysis: decoding, MVPA, searchlight, or extraction of biomarkers. The principled approach to establish their validity and usefulness is cross-validation, testing prediction on unseen data. Here, I would like to raise awareness on error bars of cross-validation, which are often underestimated. Simple experiments show that sample sizes of many neuroimaging studies inherently lead to large error bars, eg ±10% for 100 samples. The standard error across folds strongly underestimates them. These large error bars compromise the reliability of conclusions drawn with predictive models, such as biomarkers or methods developments where, unlike with cognitive neuroimaging MVPA approaches, more samples cannot be acquired by repeating the experiment across many subjects. Solutions to increase sample size must be investigated, tackling possible increases in heterogeneity of the data.
Predictive models ground many state-of-the-art developments in statistical brain image analysis: decoding, MVPA, searchlight, or extraction of biomarkers. The principled approach to establish their validity and usefulness is cross-validation, testing prediction on unseen data. Here, I would like to raise awareness on error bars of cross-validation, which are often underestimated. Simple experiments show that sample sizes of many neuroimaging studies inherently lead to large error bars, eg±10% for 100 samples. The standard error across folds strongly underestimates them. These large error bars compromise the reliability of conclusions drawn with predictive models, such as biomarkers or methods developments where, unlike with cognitive neuroimaging MVPA approaches, more samples cannot be acquired by repeating the experiment across many subjects. Solutions to increase sample size must be investigated, tackling possible increases in heterogeneity of the data.Predictive models ground many state-of-the-art developments in statistical brain image analysis: decoding, MVPA, searchlight, or extraction of biomarkers. The principled approach to establish their validity and usefulness is cross-validation, testing prediction on unseen data. Here, I would like to raise awareness on error bars of cross-validation, which are often underestimated. Simple experiments show that sample sizes of many neuroimaging studies inherently lead to large error bars, eg±10% for 100 samples. The standard error across folds strongly underestimates them. These large error bars compromise the reliability of conclusions drawn with predictive models, such as biomarkers or methods developments where, unlike with cognitive neuroimaging MVPA approaches, more samples cannot be acquired by repeating the experiment across many subjects. Solutions to increase sample size must be investigated, tackling possible increases in heterogeneity of the data.
Author Varoquaux, Gaël
Author_xml – sequence: 1
  givenname: Gaël
  surname: Varoquaux
  fullname: Varoquaux, Gaël
  email: gael.varoquaux@inria.fr
  organization: Parietal Project-team, INRIA Saclay-île de France, France
BackLink https://www.ncbi.nlm.nih.gov/pubmed/28655633$$D View this record in MEDLINE/PubMed
https://inria.hal.science/hal-01545002$$DView record in HAL
BookMark eNqNkU9r3DAQxUVJaP40X6EIemkP3o5ky5Z7KEmXpAks9NDmLGR5nGorW1vJXkg_feVs0sKeFgYkxG_eaN47I0eDH5AQymDBgJUf14sBp-Btrx9wwYFVCyhTsVfklEEtslpU_Gi-izyTjNUn5CzGNQDUrJCvyQmXpRBlnp-S62XwMWZb7WyrR-sH2mnrpoCf6PdeO0ej7jcOabR_MFKHuqWjp06HB6QYgg-00SG-IceddhEvns9zcn9z_WN5m62-fb1bXq0yI4p8zLgpOOYCSo2s7gCMFI0A0xo0vJG6aVvomppD03EseYEdypoXvCu4rhqu8_ycfNjp_tRObULaPzwqr626vVqp-Q2YKAQA37LEvt-xm-B_TxhH1dto0Dk9oJ-iYskLUckaZELf7aFrP4UhbaI41JVIPpY8UW-fqanpsf03_8XMBHzeAWb2NGCnjB2fTB1DMlUxUHN6aq3-p6fm9BSUqeYvyz2BlxkHtH7ZtWKyf2sxqGgsDgZbG9CMqvX2EJHLPRHj7GCNdr_w8TCJvy54z1M
CitedBy_id crossref_primary_10_3758_s13415_025_01268_2
crossref_primary_10_1007_s10548_023_00950_3
crossref_primary_10_1016_j_cexr_2023_100016
crossref_primary_10_1016_j_cageo_2023_105457
crossref_primary_10_1093_gigascience_giaf091
crossref_primary_10_3389_fpsyt_2022_804440
crossref_primary_10_3390_diagnostics15020216
crossref_primary_10_1016_j_cpr_2022_102193
crossref_primary_10_1016_j_nicl_2022_103120
crossref_primary_10_1140_epjc_s10052_019_6767_7
crossref_primary_10_1016_j_yebeh_2020_107525
crossref_primary_10_1038_s41598_024_60507_7
crossref_primary_10_1016_j_eswa_2023_119879
crossref_primary_10_1016_j_nicl_2021_102790
crossref_primary_10_1038_s41586_022_05118_w
crossref_primary_10_1371_journal_pone_0256152
crossref_primary_10_1016_j_euroneuro_2020_03_008
crossref_primary_10_1016_j_schres_2023_02_024
crossref_primary_10_1002_pne2_12065
crossref_primary_10_3389_fpsyt_2024_1436006
crossref_primary_10_1016_j_neuroimage_2025_121334
crossref_primary_10_1038_s41366_025_01761_1
crossref_primary_10_1016_j_neuroimage_2023_120246
crossref_primary_10_1002_hbm_26631
crossref_primary_10_3389_fninf_2021_577451
crossref_primary_10_1038_s41398_019_0663_7
crossref_primary_10_1097_j_pain_0000000000002958
crossref_primary_10_3389_fpsyt_2021_680811
crossref_primary_10_1109_TBME_2023_3330087
crossref_primary_10_1186_s40537_019_0181_8
crossref_primary_10_1038_s41380_022_01635_2
crossref_primary_10_1007_s41811_020_00089_0
crossref_primary_10_3390_microorganisms10081658
crossref_primary_10_1038_s41598_022_24541_7
crossref_primary_10_1038_s41597_022_01226_4
crossref_primary_10_1192_bjp_2019_245
crossref_primary_10_1016_j_neuroimage_2021_118044
crossref_primary_10_1093_brain_awac224
crossref_primary_10_1186_s12859_020_03773_2
crossref_primary_10_1038_s41598_022_23327_1
crossref_primary_10_1038_s41746_020_00336_w
crossref_primary_10_3389_fnagi_2018_00290
crossref_primary_10_1109_TMI_2017_2789181
crossref_primary_10_1038_s41398_022_02214_3
crossref_primary_10_1093_iob_obae010
crossref_primary_10_1523_JNEUROSCI_0884_18_2018
crossref_primary_10_1177_20539517231155060
crossref_primary_10_1080_2331186X_2024_2408836
crossref_primary_10_1038_s41598_023_39738_7
crossref_primary_10_1088_1741_2552_acad2b
crossref_primary_10_1523_JNEUROSCI_1312_22_2022
crossref_primary_10_1002_alz_12948
crossref_primary_10_1016_j_bpsc_2019_05_018
crossref_primary_10_1016_j_neuroimage_2023_119947
crossref_primary_10_1002_ccd_31689
crossref_primary_10_1016_j_ejrad_2020_108871
crossref_primary_10_1016_j_inffus_2022_11_007
crossref_primary_10_1093_comjnl_bxac033
crossref_primary_10_1016_j_eplepsyres_2024_107357
crossref_primary_10_1016_j_media_2025_103565
crossref_primary_10_1016_j_mran_2022_100201
crossref_primary_10_1016_j_watres_2020_116004
crossref_primary_10_1002_hbm_70336
crossref_primary_10_1109_JTEHM_2018_2874887
crossref_primary_10_1155_da_5512539
crossref_primary_10_1051_0004_6361_202038040
crossref_primary_10_1016_j_neuroimage_2021_118253
crossref_primary_10_1016_j_brat_2019_103530
crossref_primary_10_1038_s41598_022_06459_2
crossref_primary_10_1093_ajcn_nqy344
crossref_primary_10_1007_s00415_022_11148_1
crossref_primary_10_1093_brain_awab002
crossref_primary_10_1038_s41386_020_0666_3
crossref_primary_10_1016_j_neuroimage_2019_116388
crossref_primary_10_1016_j_jtbi_2020_110268
crossref_primary_10_1016_j_biopsych_2021_08_024
crossref_primary_10_1016_j_nicl_2021_102898
crossref_primary_10_1016_j_mri_2019_05_031
crossref_primary_10_1029_2023WR034933
crossref_primary_10_1176_appi_ajp_2021_21040370
crossref_primary_10_1016_j_psychres_2023_115378
crossref_primary_10_1038_s43587_025_00928_9
crossref_primary_10_1016_j_nicl_2022_103135
crossref_primary_10_3389_fnagi_2021_624731
crossref_primary_10_1016_j_compbiomed_2021_104324
crossref_primary_10_1109_TCDS_2021_3101643
crossref_primary_10_3389_fpsyg_2020_00220
crossref_primary_10_1016_j_neubiorev_2020_09_036
crossref_primary_10_1016_j_sleep_2025_106645
crossref_primary_10_1093_schbul_sby189
crossref_primary_10_1002_hbm_25436
crossref_primary_10_1109_ACCESS_2020_3030787
crossref_primary_10_1038_s41398_020_01013_y
crossref_primary_10_1111_add_16047
crossref_primary_10_1016_j_matchar_2025_115148
crossref_primary_10_1155_2023_8594273
crossref_primary_10_7554_eLife_53498
crossref_primary_10_3390_s20164629
crossref_primary_10_1038_s41598_021_01361_9
crossref_primary_10_1109_JBHI_2021_3101662
crossref_primary_10_1093_schbul_sbab112
crossref_primary_10_1162_netn_a_00402
crossref_primary_10_1038_s41380_019_0520_3
crossref_primary_10_1515_revneuro_2023_0117
crossref_primary_10_1016_j_nicl_2020_102195
crossref_primary_10_1080_10494820_2025_2553118
crossref_primary_10_1002_mds_28517
crossref_primary_10_1007_s00439_022_02439_8
crossref_primary_10_1016_j_applthermaleng_2019_03_111
crossref_primary_10_1016_j_neucom_2020_04_152
crossref_primary_10_1161_STROKEAHA_121_036749
crossref_primary_10_1007_s00429_024_02792_6
crossref_primary_10_1038_s41467_024_54022_6
crossref_primary_10_1038_s44220_024_00281_7
crossref_primary_10_1371_journal_pone_0224365
crossref_primary_10_1016_j_bpsc_2021_06_010
crossref_primary_10_1016_j_biopsych_2022_07_025
crossref_primary_10_1515_revneuro_2022_0122
crossref_primary_10_2147_OPTH_S367722
crossref_primary_10_1016_j_neuroimage_2022_119275
crossref_primary_10_1016_j_bpsc_2018_06_003
crossref_primary_10_1016_j_neuroimage_2024_120665
crossref_primary_10_1093_brain_awaa095
crossref_primary_10_3389_fnins_2019_01165
crossref_primary_10_1016_j_psychres_2020_112938
crossref_primary_10_3390_brainsci11060722
crossref_primary_10_1016_j_ijforecast_2021_04_002
crossref_primary_10_1038_s41598_019_50881_y
crossref_primary_10_1002_ldr_3922
crossref_primary_10_1017_S0033291724003118
crossref_primary_10_1038_s41398_020_01193_7
crossref_primary_10_1016_j_csbj_2020_10_011
crossref_primary_10_1111_deci_12445
crossref_primary_10_1093_gigascience_giab071
crossref_primary_10_1109_TCYB_2022_3158697
crossref_primary_10_1117_1_JBO_24_5_056007
crossref_primary_10_1080_10503307_2020_1839140
crossref_primary_10_1038_s41386_024_01960_w
crossref_primary_10_1038_s44184_023_00035_w
crossref_primary_10_2196_18331
crossref_primary_10_1016_j_janxdis_2021_102448
crossref_primary_10_1038_s41390_025_04131_9
crossref_primary_10_1038_s41746_022_00592_y
crossref_primary_10_52294_51f2e656_d4da_457e_851e_139131a68f14
crossref_primary_10_3390_diagnostics10070444
crossref_primary_10_1002_hbm_24886
crossref_primary_10_1016_j_foodcont_2021_108353
crossref_primary_10_1093_plphys_kiab322
crossref_primary_10_1016_j_phrs_2023_106984
crossref_primary_10_1186_s12905_025_03584_8
crossref_primary_10_1016_j_foodchem_2025_143465
crossref_primary_10_1038_s41598_020_79170_9
crossref_primary_10_1016_j_bpsgos_2025_100589
crossref_primary_10_1016_j_media_2023_102913
crossref_primary_10_2196_58572
crossref_primary_10_1016_j_neuroimage_2023_120213
crossref_primary_10_1002_brb3_70323
crossref_primary_10_1002_advs_202000675
crossref_primary_10_1016_j_nicl_2019_101782
crossref_primary_10_1002_jmri_28544
crossref_primary_10_1007_s00216_021_03665_1
crossref_primary_10_1016_j_pnpbp_2020_110136
crossref_primary_10_1016_j_tics_2022_07_003
crossref_primary_10_1371_journal_pntd_0012782
crossref_primary_10_1016_j_jneumeth_2025_110406
crossref_primary_10_1007_s40336_024_00617_4
crossref_primary_10_1002_jclp_23763
crossref_primary_10_1007_s11357_023_00924_0
crossref_primary_10_1016_j_jtice_2021_11_001
crossref_primary_10_1371_journal_pcbi_1011849
crossref_primary_10_3389_fonc_2022_843436
crossref_primary_10_3758_s13428_022_01793_9
crossref_primary_10_1016_j_expneurol_2021_113608
crossref_primary_10_1016_j_jad_2024_09_066
crossref_primary_10_3390_app15105495
crossref_primary_10_1002_hbm_24554
crossref_primary_10_1016_j_neucom_2021_05_048
crossref_primary_10_1146_annurev_clinpsy_072220_014550
crossref_primary_10_1097_PR9_0000000000000751
crossref_primary_10_1007_s00429_020_02113_7
crossref_primary_10_1371_journal_pone_0282402
crossref_primary_10_1038_s41562_023_01642_5
crossref_primary_10_1016_j_zemedi_2025_03_003
crossref_primary_10_1109_ACCESS_2020_3025165
crossref_primary_10_1002_eat_23650
crossref_primary_10_1038_s41598_020_80751_x
crossref_primary_10_1016_j_jpsychires_2022_03_063
crossref_primary_10_1016_j_bpsc_2019_10_006
crossref_primary_10_1177_20531680231211640
crossref_primary_10_1371_journal_pcbi_1008795
crossref_primary_10_3389_fpubh_2024_1373883
crossref_primary_10_1016_j_neuroimage_2024_120639
crossref_primary_10_1016_j_ynirp_2023_100170
crossref_primary_10_1016_j_neuroimage_2022_119171
crossref_primary_10_1016_j_seizure_2021_07_025
crossref_primary_10_1016_j_dscb_2021_100005
crossref_primary_10_1016_j_jpsychires_2020_08_004
crossref_primary_10_1016_j_eswa_2021_115549
crossref_primary_10_1016_j_intell_2022_101654
crossref_primary_10_1016_j_media_2020_101848
crossref_primary_10_1007_s11682_019_00191_8
crossref_primary_10_1016_j_inffus_2020_09_008
crossref_primary_10_1038_s41386_021_01104_4
crossref_primary_10_1038_s41467_024_46508_0
crossref_primary_10_1088_1361_6560_ac9d1e
crossref_primary_10_1007_s10822_019_00274_0
crossref_primary_10_1038_s41467_021_21732_0
crossref_primary_10_1080_10503307_2021_1916120
crossref_primary_10_3233_JND_230075
crossref_primary_10_1038_s41586_023_05746_w
crossref_primary_10_1002_hbm_25023
crossref_primary_10_1038_s41597_020_00680_2
crossref_primary_10_1002_hbm_25020
crossref_primary_10_1177_23328584241303495
crossref_primary_10_1016_j_neuroimage_2023_120089
crossref_primary_10_3390_land14091883
crossref_primary_10_1111_ejn_14954
crossref_primary_10_1016_j_biopsych_2022_08_024
crossref_primary_10_1016_j_jhydrol_2025_132697
crossref_primary_10_1088_1741_2552_adbfbd
crossref_primary_10_1088_1741_2552_ac28d4
crossref_primary_10_1038_s41598_022_04854_3
crossref_primary_10_1088_1741_2552_abcefe
crossref_primary_10_1007_s00330_023_09519_x
crossref_primary_10_3389_fgene_2022_821832
crossref_primary_10_1186_s11689_021_09405_x
crossref_primary_10_1111_bdi_12895
crossref_primary_10_1038_s41398_021_01286_x
crossref_primary_10_1016_j_procs_2021_08_128
crossref_primary_10_1038_s41386_024_01893_4
crossref_primary_10_2118_221475_PA
crossref_primary_10_1177_17470218211039502
crossref_primary_10_1038_s41562_022_01301_1
crossref_primary_10_1109_TBME_2019_2898871
crossref_primary_10_1001_jamaneurol_2025_2351
crossref_primary_10_1016_j_cej_2024_158322
crossref_primary_10_1007_s11682_021_00501_z
crossref_primary_10_1038_s41598_023_50050_2
crossref_primary_10_1111_acps_13233
crossref_primary_10_1038_s41582_020_0362_2
crossref_primary_10_3390_app12073673
crossref_primary_10_1016_j_bja_2020_10_034
crossref_primary_10_1038_s42256_019_0069_5
crossref_primary_10_1088_2632_2153_acac01
crossref_primary_10_1186_s12916_023_02941_4
crossref_primary_10_1038_s41598_023_44322_0
crossref_primary_10_1038_s41386_021_01020_7
crossref_primary_10_1038_s42003_024_05869_4
crossref_primary_10_1016_j_neubiorev_2024_105695
crossref_primary_10_1162_imag_a_00222
crossref_primary_10_1016_j_neuroimage_2021_118096
crossref_primary_10_1016_j_cogpsych_2020_101305
crossref_primary_10_1016_j_jad_2020_05_048
crossref_primary_10_1038_s41380_020_0771_z
crossref_primary_10_1016_j_ajp_2025_104695
crossref_primary_10_3390_s22176361
crossref_primary_10_1038_s42003_025_08132_6
crossref_primary_10_1038_s41562_021_01086_9
crossref_primary_10_1038_s41598_025_12026_2
crossref_primary_10_1017_S003329172400223X
crossref_primary_10_1109_ACCESS_2021_3117891
crossref_primary_10_52294_001c_117311
crossref_primary_10_1038_s41398_024_02954_4
crossref_primary_10_3389_fnins_2018_00737
crossref_primary_10_1093_brain_awz384
crossref_primary_10_1155_2020_8871712
crossref_primary_10_1016_j_scib_2020_04_003
crossref_primary_10_1038_s41398_022_02162_y
crossref_primary_10_1016_j_neucom_2022_09_001
crossref_primary_10_1016_j_brat_2022_104116
crossref_primary_10_1093_pm_pnaa178
crossref_primary_10_1016_j_bpsc_2019_11_007
crossref_primary_10_1016_j_neurobiolaging_2022_06_008
crossref_primary_10_1055_a_2142_9325
crossref_primary_10_1016_j_eswa_2024_126295
crossref_primary_10_1016_j_neuroimage_2023_120412
crossref_primary_10_3390_s23094199
crossref_primary_10_1111_brv_12752
crossref_primary_10_3389_fnins_2023_1174080
crossref_primary_10_1016_j_dcn_2021_100966
crossref_primary_10_1016_j_scib_2019_04_034
crossref_primary_10_1371_journal_pone_0292047
crossref_primary_10_1002_hbm_26683
crossref_primary_10_1002_hbm_70280
crossref_primary_10_53464_JMTE_01_2024_04
crossref_primary_10_1038_s41598_020_65384_4
crossref_primary_10_1016_j_neuroimage_2023_119960
crossref_primary_10_1089_brain_2021_0186
crossref_primary_10_1038_s44220_023_00173_2
crossref_primary_10_1016_j_jvcir_2025_104436
crossref_primary_10_1016_j_schres_2023_01_014
crossref_primary_10_1038_s41586_022_04492_9
crossref_primary_10_1111_psyp_14762
crossref_primary_10_1080_03610926_2021_2021240
crossref_primary_10_1016_j_inffus_2025_103482
crossref_primary_10_1016_j_eswa_2019_04_063
crossref_primary_10_1038_s41467_024_53743_y
crossref_primary_10_1002_hbm_25907
crossref_primary_10_1001_jamanetworkopen_2023_36094
crossref_primary_10_1016_j_neuroimage_2021_118756
crossref_primary_10_1016_j_biopsych_2022_03_007
crossref_primary_10_1016_j_bpsc_2022_12_006
crossref_primary_10_7554_eLife_56929
crossref_primary_10_3389_fneur_2022_923988
crossref_primary_10_1016_j_chemosphere_2019_03_007
crossref_primary_10_1109_TII_2020_3036159
crossref_primary_10_3390_su15097086
crossref_primary_10_1016_j_neuroimage_2023_120162
crossref_primary_10_1016_j_saa_2024_124189
crossref_primary_10_1162_imag_a_00219
crossref_primary_10_7554_eLife_54055
crossref_primary_10_1016_j_cmpb_2021_106549
crossref_primary_10_1038_s42003_024_06574_y
crossref_primary_10_1016_j_jare_2025_04_026
crossref_primary_10_1016_j_nsa_2023_103931
crossref_primary_10_1111_ejn_15842
crossref_primary_10_1590_1516_3180_2020_0474_r2_03052021
crossref_primary_10_1162_netn_a_00233
crossref_primary_10_1016_j_neuroimage_2021_118648
crossref_primary_10_7717_peerj_18490
crossref_primary_10_1016_j_seppur_2024_130497
crossref_primary_10_2478_amns_2024_3085
crossref_primary_10_1093_cercor_bhac309
crossref_primary_10_1093_gigascience_giaf036
crossref_primary_10_1007_s12021_021_09519_6
crossref_primary_10_1016_j_biopsych_2022_09_024
crossref_primary_10_1038_s41386_020_0785_x
crossref_primary_10_1016_j_neubiorev_2024_105640
crossref_primary_10_1038_s41586_025_09250_1
crossref_primary_10_1038_s41467_021_25492_9
crossref_primary_10_1002_hbm_24802
crossref_primary_10_1007_s00406_020_01201_3
crossref_primary_10_1093_bib_bbac040
crossref_primary_10_1016_j_neubiorev_2022_104552
crossref_primary_10_1016_j_nic_2019_09_005
crossref_primary_10_1038_s41598_024_64487_6
crossref_primary_10_1016_j_cortex_2019_01_018
crossref_primary_10_3389_fpsyg_2025_1520186
crossref_primary_10_1016_j_neuroimage_2023_119986
crossref_primary_10_1001_jamanetworkopen_2023_1671
crossref_primary_10_1016_j_nicl_2019_101821
crossref_primary_10_1016_j_pnpbp_2022_110574
crossref_primary_10_3390_jpm11100957
crossref_primary_10_1016_j_mri_2020_11_006
crossref_primary_10_1111_epi_16853
crossref_primary_10_3389_fgene_2022_848205
crossref_primary_10_1016_j_ejrad_2024_111731
crossref_primary_10_1016_j_neuropsychologia_2021_107893
crossref_primary_10_1016_j_bionps_2024_100102
crossref_primary_10_7554_eLife_39324
crossref_primary_10_1007_s11920_022_01385_6
crossref_primary_10_1002_trc2_12303
crossref_primary_10_1111_iej_70034
crossref_primary_10_1002_hbm_25803
crossref_primary_10_1016_j_jpsychires_2020_11_014
crossref_primary_10_1111_jcpp_13545
crossref_primary_10_3233_IDA_230220
crossref_primary_10_1162_netn_a_00338
crossref_primary_10_1016_j_biopsych_2022_10_014
crossref_primary_10_1002_per_2235
crossref_primary_10_1038_s41598_021_03785_9
crossref_primary_10_1080_14737167_2023_2279107
crossref_primary_10_1016_j_biombioe_2025_107956
crossref_primary_10_1038_s41562_021_01085_w
crossref_primary_10_1111_pcn_13612
crossref_primary_10_1093_brain_awy251
crossref_primary_10_3389_fnins_2018_00149
crossref_primary_10_1192_j_eurpsy_2021_2248
crossref_primary_10_1038_s41598_023_37034_y
crossref_primary_10_1016_j_neuroimage_2021_118623
crossref_primary_10_1016_j_artmed_2021_102039
crossref_primary_10_1111_2041_210X_13785
crossref_primary_10_1016_j_neuroimage_2018_06_064
crossref_primary_10_1016_j_dcn_2025_101555
crossref_primary_10_1007_s00115_021_01197_8
crossref_primary_10_3389_fneur_2024_1425490
Cites_doi 10.1016/j.neuroimage.2008.11.007
10.3389/fninf.2015.00008
10.3389/fninf.2013.00012
10.1016/j.cub.2011.08.031
10.1002/hbm.20136
10.1016/j.neuroimage.2016.02.079
10.1016/j.neuron.2011.08.026
10.1038/nrn.2016.167
10.1177/0956797611417632
10.1186/2047-217X-3-28
10.1016/j.neuroimage.2010.05.026
10.1186/1471-2202-8-91
10.1056/NEJMoa1204471
10.1371/journal.pmed.0020124
10.1214/09-SS054
10.1016/j.neuroimage.2012.07.004
10.1037/0033-2909.86.3.638
10.1038/mp.2013.78
10.1038/nn.4393
10.1038/nature06713
10.1038/nrn1931
10.1016/j.neuroimage.2017.01.072
10.1016/j.tics.2006.07.005
10.1016/j.patcog.2011.04.006
10.1126/science.1063736
10.1016/j.neuroimage.2013.05.041
10.1097/EDE.0b013e31818131e7
10.1038/nn.4478
10.1016/j.neuroimage.2016.10.045
10.1016/j.neuroimage.2014.04.018
10.1038/nrn3475
10.1111/j.1467-9280.2009.02460.x
10.1038/nn.4125
10.1016/j.neuroimage.2005.12.062
10.3389/neuro.11.033.2009
10.1038/nn.3818
10.1111/j.1745-6924.2009.01125.x
10.1016/j.neubiorev.2015.08.001
10.1126/science.aaa9375
10.1016/j.neuroimage.2016.10.038
10.1073/pnas.0600244103
10.1016/j.neuroimage.2012.09.063
10.1093/bioinformatics/btg419
10.1007/s11682-013-9269-5
10.1073/pnas.0911855107
ContentType Journal Article
Copyright 2017 Elsevier Inc.
Copyright © 2017 Elsevier Inc. All rights reserved.
2017. Elsevier Inc.
Distributed under a Creative Commons Attribution 4.0 International License
Copyright_xml – notice: 2017 Elsevier Inc.
– notice: Copyright © 2017 Elsevier Inc. All rights reserved.
– notice: 2017. Elsevier Inc.
– notice: Distributed under a Creative Commons Attribution 4.0 International License
DBID AAYXX
CITATION
CGR
CUY
CVF
ECM
EIF
NPM
3V.
7TK
7X7
7XB
88E
88G
8AO
8FD
8FE
8FH
8FI
8FJ
8FK
ABUWG
AFKRA
AZQEC
BBNVY
BENPR
BHPHI
CCPQU
DWQXO
FR3
FYUFA
GHDGH
GNUQQ
HCIFZ
K9.
LK8
M0S
M1P
M2M
M7P
P64
PHGZM
PHGZT
PJZUB
PKEHL
PPXIY
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PSYQQ
Q9U
RC3
7X8
1XC
VOOES
DOI 10.1016/j.neuroimage.2017.06.061
DatabaseName CrossRef
Medline
MEDLINE
MEDLINE (Ovid)
MEDLINE
MEDLINE
PubMed
ProQuest Central (Corporate)
Neurosciences Abstracts
Health & Medical Collection (ProQuest)
ProQuest Central (purchase pre-March 2016)
Medical Database (Alumni Edition)
Psychology Database (Alumni)
ProQuest Pharma Collection
Technology Research Database
ProQuest SciTech Collection
ProQuest Natural Science Collection
ProQuest Hospital Collection
Hospital Premium Collection (Alumni Edition)
ProQuest Central (Alumni) (purchase pre-March 2016)
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
ProQuest Central Essentials
Biological Science Collection
ProQuest Central
Natural Science Collection
ProQuest One
ProQuest Central Korea
Engineering Research Database
ProQuest Health & Medical Collection
Health Research Premium Collection (Alumni)
ProQuest Central Student
SciTech Premium Collection
ProQuest Health & Medical Complete (Alumni)
Biological Sciences
ProQuest Health & Medical Collection
Medical Database
Psychology Database
Biological Science Database (ProQuest)
Biotechnology and BioEngineering Abstracts
ProQuest Central Premium
ProQuest One Academic
ProQuest Health & Medical Research Collection
ProQuest One Academic Middle East (New)
ProQuest One Health & Nursing
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
ProQuest One Psychology
ProQuest Central Basic
Genetics Abstracts
MEDLINE - Academic
Hyper Article en Ligne (HAL)
Hyper Article en Ligne (HAL) (Open Access)
DatabaseTitle CrossRef
MEDLINE
Medline Complete
MEDLINE with Full Text
PubMed
MEDLINE (Ovid)
ProQuest One Psychology
ProQuest Central Student
Technology Research Database
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest Health & Medical Complete (Alumni)
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest One Health & Nursing
ProQuest Natural Science Collection
ProQuest Pharma Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Health & Medical Research Collection
Genetics Abstracts
Health Research Premium Collection
Health and Medicine Complete (Alumni Edition)
Natural Science Collection
ProQuest Central Korea
Health & Medical Research Collection
Biological Science Collection
ProQuest Central (New)
ProQuest Medical Library (Alumni)
ProQuest Biological Science Collection
ProQuest Central Basic
ProQuest One Academic Eastern Edition
ProQuest Hospital Collection
Health Research Premium Collection (Alumni)
ProQuest Psychology Journals (Alumni)
Biological Science Database
ProQuest SciTech Collection
Neurosciences Abstracts
ProQuest Hospital Collection (Alumni)
Biotechnology and BioEngineering Abstracts
ProQuest Health & Medical Complete
ProQuest Medical Library
ProQuest Psychology Journals
ProQuest One Academic UKI Edition
Engineering Research Database
ProQuest One Academic
ProQuest One Academic (New)
ProQuest Central (Alumni)
MEDLINE - Academic
DatabaseTitleList

ProQuest One Psychology
MEDLINE

MEDLINE - Academic
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: BENPR
  name: ProQuest Central
  url: https://www.proquest.com/central
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Medicine
Statistics
Psychology
Computer Science
EISSN 1095-9572
EndPage 77
ExternalDocumentID oai:HAL:hal-01545002v1
28655633
10_1016_j_neuroimage_2017_06_061
S1053811917305311
Genre Research Support, Non-U.S. Gov't
Journal Article
GroupedDBID ---
--K
--M
.1-
.FO
.~1
0R~
123
1B1
1RT
1~.
1~5
4.4
457
4G.
5RE
5VS
7-5
71M
7X7
88E
8AO
8FE
8FH
8FI
8FJ
8P~
9JM
AABNK
AAEDT
AAEDW
AAIKJ
AAKOC
AALRI
AAOAW
AAQFI
AATTM
AAXKI
AAXLA
AAXUO
AAYWO
ABBQC
ABCQJ
ABFNM
ABFRF
ABIVO
ABJNI
ABMAC
ABMZM
ABUWG
ACDAQ
ACGFO
ACGFS
ACIEU
ACLOT
ACPRK
ACRLP
ACVFH
ADBBV
ADCNI
ADEZE
ADFRT
AEBSH
AEFWE
AEIPS
AEKER
AENEX
AEUPX
AFJKZ
AFKRA
AFPUW
AFRHN
AFTJW
AFXIZ
AGUBO
AGWIK
AGYEJ
AHHHB
AHMBA
AIEXJ
AIIUN
AIKHN
AITUG
AJRQY
AJUYK
AKBMS
AKRWK
AKYEP
ALMA_UNASSIGNED_HOLDINGS
AMRAJ
ANKPU
ANZVX
APXCP
AXJTR
AZQEC
BBNVY
BENPR
BHPHI
BKOJK
BLXMC
BNPGV
BPHCQ
BVXVI
CCPQU
CS3
DM4
DU5
DWQXO
EBS
EFBJH
EFKBS
EFLBG
EJD
EO8
EO9
EP2
EP3
F5P
FDB
FIRID
FNPLU
FYGXN
FYUFA
G-Q
GBLVA
GNUQQ
GROUPED_DOAJ
HCIFZ
HMCUK
IHE
J1W
KOM
LG5
LK8
LX8
M1P
M29
M2M
M2V
M41
M7P
MO0
MOBAO
N9A
O-L
O9-
OAUVE
OVD
OZT
P-8
P-9
P2P
PC.
PHGZM
PHGZT
PJZUB
PPXIY
PQGLB
PQQKQ
PROAC
PSQYO
PSYQQ
Q38
ROL
RPZ
SAE
SCC
SDF
SDG
SDP
SES
SSH
SSN
SSZ
T5K
TEORI
UKHRP
UV1
YK3
Z5R
ZU3
~G-
~HD
3V.
AACTN
AADPK
AAIAV
ABLVK
ABYKQ
AFKWA
AJOXV
AMFUW
C45
HMQ
LCYCR
RIG
SNS
ZA5
29N
53G
9DU
AAFWJ
AAQXK
AAYXX
ABXDB
ACRPL
ADFGL
ADMUD
ADNMO
ADVLN
ADXHL
AFFHD
AFPKN
AGHFR
AGQPQ
AIGII
AKRLJ
ASPBG
AVWKF
AZFZN
CAG
CITATION
COF
FEDTE
FGOYB
G-2
HDW
HEI
HMK
HMO
HVGLF
HZ~
OK1
R2-
SEW
WUQ
XPP
ZMT
0SF
ALIPV
CGR
CUY
CVF
ECM
EIF
NPM
7TK
7XB
8FD
8FK
FR3
K9.
P64
PKEHL
PQEST
PQUKI
PRINS
Q9U
RC3
7X8
PUEGO
1XC
VOOES
ID FETCH-LOGICAL-c543t-2c42e3506ae19f00c85b50cdcec2b8abdd0fb920bf2e624efe89242f42a7b2a33
IEDL.DBID M7P
ISICitedReferencesCount 417
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000443268900006&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1053-8119
1095-9572
IngestDate Tue Oct 14 20:46:26 EDT 2025
Thu Oct 02 10:16:52 EDT 2025
Sat Nov 29 15:02:37 EST 2025
Wed Feb 19 02:33:28 EST 2025
Tue Nov 18 22:35:00 EST 2025
Sat Nov 29 07:05:57 EST 2025
Fri Feb 23 02:36:54 EST 2024
Tue Oct 14 19:31:18 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue Pt A
Keywords fMRI
Model selection
Cross-validation
Biomarkers
Decoding
Statistics
MVPA
biomarkers
model selection
Comments and Controversies
cross-validation
decoding
statistics
Language English
License Copyright © 2017 Elsevier Inc. All rights reserved.
Distributed under a Creative Commons Attribution 4.0 International License: http://creativecommons.org/licenses/by/4.0
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c543t-2c42e3506ae19f00c85b50cdcec2b8abdd0fb920bf2e624efe89242f42a7b2a33
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ORCID 0000-0003-1076-5122
OpenAccessLink https://inria.hal.science/hal-01545002
PMID 28655633
PQID 2097510562
PQPubID 2031077
PageCount 10
ParticipantIDs hal_primary_oai_HAL_hal_01545002v1
proquest_miscellaneous_1914578908
proquest_journals_2097510562
pubmed_primary_28655633
crossref_citationtrail_10_1016_j_neuroimage_2017_06_061
crossref_primary_10_1016_j_neuroimage_2017_06_061
elsevier_sciencedirect_doi_10_1016_j_neuroimage_2017_06_061
elsevier_clinicalkey_doi_10_1016_j_neuroimage_2017_06_061
PublicationCentury 2000
PublicationDate 2018-10-15
PublicationDateYYYYMMDD 2018-10-15
PublicationDate_xml – month: 10
  year: 2018
  text: 2018-10-15
  day: 15
PublicationDecade 2010
PublicationPlace United States
PublicationPlace_xml – name: United States
– name: Amsterdam
PublicationTitle NeuroImage (Orlando, Fla.)
PublicationTitleAlternate Neuroimage
PublicationYear 2018
Publisher Elsevier Inc
Elsevier Limited
Elsevier
Publisher_xml – name: Elsevier Inc
– name: Elsevier Limited
– name: Elsevier
References Hoyos-Idrobo, Varoquaux, Kahn, Thirion (bib20) 2016
Poldrack, Gorgolewski (bib37) 2014; 17
Van Essen, Smith, Barch, Behrens, Yacoub, Ugurbil, Consortium, W.M.H (bib49) 2013; 80
Hastie, Tibshirani, Friedman (bib16) 2009
Costafreda (bib10) 2009; 3
Pereira, Botvinick (bib32) 2011; 56
Saxe, Brett, Kanwisher (bib41) 2006; 30
Dansereau, Benhajali, Risterucci, Pich, Orban, Arnold, Bellec (bib11) 2017; 149
Rosenthal (bib39) 1979; 86
Thompson, Stein, Medland, Hibar, Vasquez, Renteria, Toro, Jahanshad, Schumann, Franke (bib48) 2014; 8
Brown, Hamarneh (bib7) 2016
Woo, Chang, Lindquist, Wager (bib56) 2017; 20
Biswal, Mennes, Zuo, Gohel, Kelly, Smith, Beckmann (bib5) 2010; 107
Haxby, Gobbini, Furey (bib17) 2001; 293
Dwork, Feldman, Hardt, Pitassi, Reingold, Roth (bib14) 2015; 349
Carp (bib9) 2012; 63
Skocik, Collins, Callahan-Flintoft, Bowman, Wyble (bib44) 2016
Norman, Polyn, Detre, Haxby (bib30) 2006; 10
Poldrack, Barch, Mitchell, Wager, Wagner, Devlin, Cumba, Koyejo, Milham (bib36) 2013; 7
Ziegler, Ridgway, Dahnke, Gaser, Initiative (bib59) 2014; 97
Di Martino, Yan, Li, Denio, Castellanos, Alaerts, Anderson, Assaf, Bookheimer, Dapretto (bib13) 2014; 19
Kriegeskorte, Goebel, Bandettini (bib24) 2006; 103
Arbabshirani, Plis, Sui, Calhoun (bib2) 2017; 145
Miller, Alfaro-Almagro, Bangerter, Thomas, Yacoub, Xu, Bartsch, Jbabdi, Sotiropoulos, Andersson (bib28) 2016; 19
Yarkoni, Westfall (bib58) 2016
Kay, Naselaris, Prenger, Gallant (bib23) 2008; 452
Little, Varoquaux, Saeb, Lonini, Jayaraman, Mohr, Kording (bib26) 2017; 6
Szucs, Ioannidis (bib47) 2016
Ioannidis (bib21) 2005; 2
Silva, Castro, Gupta, Cetin, Arbabshirani, Potluru, Plis, Calhoun (bib42) 2014
Saeb, Lonini, Jayaraman, Mohr, David, Kording (bib40) 2017
Button, Ioannidis, Mokrysz, Nosek, Flint, Robinson, Munafò (bib8) 2013; 14
Haynes, Rees (bib19) 2006; 7
Wager, Atlas, Lindquist, Roy, Woo, Kross (bib54) 2013; 368
Varoquaux, Gramfort, Thirion (bib50) 2012
Wolfers, Buitelaar, Beckmann, Franke, Marquand (bib55) 2015; 57
Demšar (bib12) 2006; 7
Haxby, Guntupalli, Connolly, Halchenko, Conroy, Gobbini, Hanke, Ramadge (bib18) 2011; 72
Penny, Friston, Ashburner, Kiebel, Nichols (bib31) 2007
Pinel, Thirion, Meriaux, Jobert, Serres, Le Bihan, Poline, Dehaene (bib34) 2007; 8
Smith, Nichols, Vidaurre, Winkler, Behrens, Glasser, Ugurbil, Barch, Van Essen, Miller (bib45) 2015; 18
Abraham, Milham, Di Martino, Craddock, Samaras, Thirion, Varoquaux (bib1) 2017; 147
Stelzer, Chen, Turner (bib46) 2013; 65
Nishimoto, Vu, Naselaris, Benjamini, Yu, Gallant (bib29) 2011; 21
Gorgolewski, Varoquaux, Rivera, Schwarz, Ghosh, Maumet, Sochat, Nichols, Poldrack, Poline (bib15) 2015; 9
Bengio, Grandvalet (bib4) 2004; 5
Ioannidis (bib22) 2008; 19
Yahata, Morimoto, Hashimoto, Lisi, Shibata, Kawakubo, Kuwabara, Kuroda, Yamada, Megumi (bib57) 2016; 7
Poldrack, Baker, Durnez, Gorgolewski, Matthews, Munafo, Nichols, Poline, Vul, Yarkoni (bib35) 2017; 18
Braga-Neto, Dougherty (bib6) 2004; 20
Arlot, Celisse (bib3) 2010; 4
Simmons, Nelson, Simonsohn (bib43) 2011; 22
Laird, Fox, Price, Glahn, Uecker, Lancaster, Turkeltaub, Kochunov, Fox (bib25) 2005; 25
Michel, Gramfort, Varoquaux, Eger, Keribin, Thirion (bib27) 2012; 45
Poldrack, Halchenko, Hanson (bib38) 2009; 20
Pereira, Mitchell, Botvinick (bib33) 2009; 45
Vul, Harris, Winkielman, Pashler (bib53) 2009; 4
Varoquaux, Raamana, Engemann, Hoyos-Idrobo, Schwartz, Thirion (bib51) 2017; 145
Varoquaux, Thirion (bib52) 2014; 3
Penny (10.1016/j.neuroimage.2017.06.061_bib31) 2007
Ioannidis (10.1016/j.neuroimage.2017.06.061_bib21) 2005; 2
Wager (10.1016/j.neuroimage.2017.06.061_bib54) 2013; 368
Silva (10.1016/j.neuroimage.2017.06.061_bib42) 2014
Di Martino (10.1016/j.neuroimage.2017.06.061_bib13) 2014; 19
Biswal (10.1016/j.neuroimage.2017.06.061_bib5) 2010; 107
Ioannidis (10.1016/j.neuroimage.2017.06.061_bib22) 2008; 19
Hoyos-Idrobo (10.1016/j.neuroimage.2017.06.061_bib20) 2016
Poldrack (10.1016/j.neuroimage.2017.06.061_bib38) 2009; 20
Gorgolewski (10.1016/j.neuroimage.2017.06.061_bib15) 2015; 9
Nishimoto (10.1016/j.neuroimage.2017.06.061_bib29) 2011; 21
Woo (10.1016/j.neuroimage.2017.06.061_bib56) 2017; 20
Wolfers (10.1016/j.neuroimage.2017.06.061_bib55) 2015; 57
Ziegler (10.1016/j.neuroimage.2017.06.061_bib59) 2014; 97
Carp (10.1016/j.neuroimage.2017.06.061_bib9) 2012; 63
Braga-Neto (10.1016/j.neuroimage.2017.06.061_bib6) 2004; 20
Poldrack (10.1016/j.neuroimage.2017.06.061_bib35) 2017; 18
Abraham (10.1016/j.neuroimage.2017.06.061_bib1) 2017; 147
Miller (10.1016/j.neuroimage.2017.06.061_bib28) 2016; 19
Saeb (10.1016/j.neuroimage.2017.06.061_bib40) 2017
Brown (10.1016/j.neuroimage.2017.06.061_bib7) 2016
Vul (10.1016/j.neuroimage.2017.06.061_bib53) 2009; 4
Haxby (10.1016/j.neuroimage.2017.06.061_bib18) 2011; 72
Arlot (10.1016/j.neuroimage.2017.06.061_bib3) 2010; 4
Varoquaux (10.1016/j.neuroimage.2017.06.061_bib52) 2014; 3
Arbabshirani (10.1016/j.neuroimage.2017.06.061_bib2) 2017; 145
Thompson (10.1016/j.neuroimage.2017.06.061_bib48) 2014; 8
Poldrack (10.1016/j.neuroimage.2017.06.061_bib37) 2014; 17
Demšar (10.1016/j.neuroimage.2017.06.061_bib12) 2006; 7
Simmons (10.1016/j.neuroimage.2017.06.061_bib43) 2011; 22
Laird (10.1016/j.neuroimage.2017.06.061_bib25) 2005; 25
Varoquaux (10.1016/j.neuroimage.2017.06.061_bib51) 2017; 145
Dansereau (10.1016/j.neuroimage.2017.06.061_bib11) 2017; 149
Pinel (10.1016/j.neuroimage.2017.06.061_bib34) 2007; 8
Haynes (10.1016/j.neuroimage.2017.06.061_bib19) 2006; 7
Pereira (10.1016/j.neuroimage.2017.06.061_bib33) 2009; 45
Bengio (10.1016/j.neuroimage.2017.06.061_bib4) 2004; 5
Yahata (10.1016/j.neuroimage.2017.06.061_bib57) 2016; 7
Norman (10.1016/j.neuroimage.2017.06.061_bib30) 2006; 10
Michel (10.1016/j.neuroimage.2017.06.061_bib27) 2012; 45
Smith (10.1016/j.neuroimage.2017.06.061_bib45) 2015; 18
Little (10.1016/j.neuroimage.2017.06.061_bib26) 2017; 6
Skocik (10.1016/j.neuroimage.2017.06.061_bib44) 2016
Varoquaux (10.1016/j.neuroimage.2017.06.061_bib50) 2012
Saxe (10.1016/j.neuroimage.2017.06.061_bib41) 2006; 30
Costafreda (10.1016/j.neuroimage.2017.06.061_bib10) 2009; 3
Button (10.1016/j.neuroimage.2017.06.061_bib8) 2013; 14
Yarkoni (10.1016/j.neuroimage.2017.06.061_bib58) 2016
Kay (10.1016/j.neuroimage.2017.06.061_bib23) 2008; 452
Pereira (10.1016/j.neuroimage.2017.06.061_bib32) 2011; 56
Haxby (10.1016/j.neuroimage.2017.06.061_bib17) 2001; 293
Van Essen (10.1016/j.neuroimage.2017.06.061_bib49) 2013; 80
Dwork (10.1016/j.neuroimage.2017.06.061_bib14) 2015; 349
Stelzer (10.1016/j.neuroimage.2017.06.061_bib46) 2013; 65
Kriegeskorte (10.1016/j.neuroimage.2017.06.061_bib24) 2006; 103
Poldrack (10.1016/j.neuroimage.2017.06.061_bib36) 2013; 7
Hastie (10.1016/j.neuroimage.2017.06.061_bib16) 2009
Szucs (10.1016/j.neuroimage.2017.06.061_bib47) 2016
Rosenthal (10.1016/j.neuroimage.2017.06.061_bib39) 1979; 86
References_xml – volume: 63
  start-page: 289
  year: 2012
  end-page: 300
  ident: bib9
  article-title: The secret lives of experiments: methods reporting in the fmri literature
  publication-title: Neuroimage
– volume: 30
  start-page: 1088
  year: 2006
  end-page: 1096
  ident: bib41
  article-title: Divide and conquer: a defense of functional localizers
  publication-title: Neuroimage
– volume: 21
  start-page: 1641
  year: 2011
  ident: bib29
  article-title: Reconstructing visual experiences from brain activity evoked by natural movies
  publication-title: Curr. Biol.
– volume: 57
  start-page: 328
  year: 2015
  ident: bib55
  article-title: From estimating activation locality to predicting disorder: a review of pattern recognition for neuroimaging-based psychiatric diagnostics
  publication-title: Neurosci. Biobehav. Rev.
– volume: 4
  start-page: 40
  year: 2010
  ident: bib3
  article-title: A survey of cross-validation procedures for model selection
  publication-title: Stat. Surv.
– year: 2016
  ident: bib44
  article-title: I Tried a Bunch of Things: the Dangers of Unexpected Overfitting in Classification
– volume: 149
  start-page: 220
  year: 2017
  end-page: 232
  ident: bib11
  article-title: Statistical power and prediction accuracy in multisite resting-state fmri connectivity
  publication-title: NeuroImage
– volume: 65
  start-page: 69
  year: 2013
  end-page: 82
  ident: bib46
  article-title: Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (mvpa): random permutations and cluster size control
  publication-title: Neuroimage
– volume: 80
  start-page: 62
  year: 2013
  end-page: 79
  ident: bib49
  article-title: The wu-minn human connectome project: an overview
  publication-title: Neuroimage
– year: 2016
  ident: bib58
  article-title: Choosing Prediction over Explanation in Psychology: Lessons from machine Learning
– volume: 7
  start-page: 12
  year: 2013
  ident: bib36
  article-title: Toward open sharing of task-based fmri data: the openfmri project
  publication-title: Front. Neuroinform.
– volume: 97
  start-page: 333
  year: 2014
  end-page: 348
  ident: bib59
  article-title: Individualized gaussian process-based prediction and detection of local and global gray matter abnormalities in elderly subjects
  publication-title: NeuroImage
– volume: 5
  start-page: 1089
  year: 2004
  ident: bib4
  article-title: No unbiased estimator of the variance of k-fold cross-validation
  publication-title: J. Mach. Learn. Res.
– volume: 349
  start-page: 636
  year: 2015
  ident: bib14
  article-title: The reusable holdout: preserving validity in adaptive data analysis
  publication-title: Science
– volume: 19
  start-page: 640
  year: 2008
  end-page: 648
  ident: bib22
  article-title: Why most discovered true associations are inflated
  publication-title: Epidemiology
– volume: 45
  start-page: S199
  year: 2009
  ident: bib33
  article-title: Machine learning classifiers and fMRI: a tutorial overview
  publication-title: Neuroimage
– volume: 147
  start-page: 736
  year: 2017
  end-page: 745
  ident: bib1
  article-title: Deriving reproducible biomarkers from multi-site resting-state data: an autism-based example
  publication-title: NeuroImage
– volume: 6
  start-page: 1
  year: 2017
  end-page: 6
  ident: bib26
  article-title: Using and understanding cross-validation strategies. perspectives on Saeb et al
– volume: 86
  start-page: 638
  year: 1979
  ident: bib39
  article-title: The file drawer problem and tolerance for null results
  publication-title: Psychol. Bull.
– volume: 10
  start-page: 424
  year: 2006
  ident: bib30
  article-title: Beyond mind-reading: multi-voxel pattern analysis of fMRI data
  publication-title: Trends Cogn. Sci.
– volume: 7
  start-page: 1
  year: 2016
  ident: bib57
  article-title: A small number of abnormal brain connections predicts adult autism spectrum disorder
  publication-title: NATURE
– volume: 293
  start-page: 2425
  year: 2001
  ident: bib17
  article-title: Distributed and overlapping representations of faces and objects in ventral temporal cortex
  publication-title: Science
– volume: 20
  start-page: 374
  year: 2004
  end-page: 380
  ident: bib6
  article-title: Is cross-validation valid for small-sample microarray classification?
  publication-title: Bioinformatics
– volume: 56
  start-page: 476
  year: 2011
  ident: bib32
  article-title: Information mapping with pattern classifiers: a comparative study
  publication-title: Neuroimage
– volume: 7
  start-page: 523
  year: 2006
  ident: bib19
  article-title: Decoding mental states from brain activity in humans
  publication-title: Nat. Rev. Neurosci.
– volume: 19
  start-page: 659
  year: 2014
  end-page: 667
  ident: bib13
  article-title: The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism
  publication-title: Mol. psychiatry
– volume: 368
  start-page: 1388
  year: 2013
  ident: bib54
  article-title: An fMRI-based neurologic signature of physical pain
  publication-title: N. Engl. J. Med.
– year: 2007
  ident: bib31
  article-title: Statistical Parametric Mapping: the Analysis of Functional Brain Images
– volume: 8
  start-page: 153
  year: 2014
  end-page: 182
  ident: bib48
  article-title: The enigma consortium: large-scale collaborative analyses of neuroimaging and genetic data
  publication-title: Brain Imaging Behav.
– year: 2012
  ident: bib50
  article-title: Small-sample brain mapping: sparse recovery on spatially correlated designs with randomization and clustering
  publication-title: ICML
– volume: 2
  start-page: e124
  year: 2005
  ident: bib21
  article-title: Why most published research findings are false
  publication-title: PLOS Med.
– volume: 20
  start-page: 1364
  year: 2009
  ident: bib38
  article-title: Decoding the large-scale structure of brain function by classifying mental states across individuals
  publication-title: Psychol. Sci.
– volume: 7
  start-page: 1
  year: 2006
  end-page: 30
  ident: bib12
  article-title: Statistical comparisons of classifiers over multiple data sets
  publication-title: J. Mach. Learn. Res.
– volume: 452
  start-page: 352
  year: 2008
  end-page: 355
  ident: bib23
  article-title: Identifying natural images from human brain activity
  publication-title: Nature
– volume: 4
  start-page: 274
  year: 2009
  ident: bib53
  article-title: Puzzlingly high correlations in fmri studies of emotion, personality, and social cognition
  publication-title: Perspect. Psychol. Sci.
– volume: 19
  start-page: 1523
  year: 2016
  end-page: 1536
  ident: bib28
  article-title: Multimodal population brain imaging in the UK biobank prospective epidemiological study
  publication-title: Nat. Neurosci.
– volume: 22
  start-page: 1359
  year: 2011
  ident: bib43
  article-title: False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant
  publication-title: Psychol. Sci.
– volume: 3
  start-page: 33
  year: 2009
  ident: bib10
  article-title: Pooling fmri data: meta-analysis, mega-analysis and multi-center studies
  publication-title: Front. Neuroinformatics
– volume: 17
  start-page: 1510
  year: 2014
  end-page: 1517
  ident: bib37
  article-title: Making big data open: data sharing in neuroimaging
  publication-title: Nat. Neurosci.
– year: 2017
  ident: bib40
  article-title: The Need to Approximate the Use-case in Clinical machine Learning
– volume: 9
  start-page: 8
  year: 2015
  ident: bib15
  article-title: Neurovault. org: a web-based repository for collecting and sharing unthresholded statistical maps of the human brain
  publication-title: Front. Neuroinform.
– volume: 8
  start-page: 91
  year: 2007
  ident: bib34
  article-title: Fast reproducible identification and large-scale databasing of individual functional cognitive networks
  publication-title: BMC Neurosci.
– volume: 145
  start-page: 137
  year: 2017
  end-page: 165
  ident: bib2
  article-title: Single subject prediction of brain disorders in neuroimaging: promises and pitfalls
  publication-title: NeuroImage
– volume: 107
  start-page: 4734
  year: 2010
  ident: bib5
  article-title: Toward discovery science of human brain function
  publication-title: Proc. Ntl Acad. Sci.
– volume: 18
  start-page: 115
  year: 2017
  ident: bib35
  article-title: Scanning the horizon: future challenges for neuroimaging research
  publication-title: Nat. Rev. Neurosci.
– volume: 20
  start-page: 365
  year: 2017
  end-page: 377
  ident: bib56
  article-title: Building better biomarkers: brain models in translational neuroimaging
  publication-title: Nat. Neurosci.
– year: 2016
  ident: bib47
  article-title: Empirical Assessment of Published Effect Sizes and Power in the Recent Cognitive Neuroscience and Psychology Literature
– volume: 72
  start-page: 404
  year: 2011
  end-page: 416
  ident: bib18
  article-title: A common, high-dimensional model of the representational space in human ventral temporal cortex
  publication-title: Neuron
– volume: 14
  start-page: 365
  year: 2013
  end-page: 376
  ident: bib8
  article-title: Power failure: why small sample size undermines the reliability of neuroscience
  publication-title: Nat. Rev. Neurosci.
– volume: 25
  start-page: 155
  year: 2005
  end-page: 164
  ident: bib25
  article-title: Ale meta-analysis: controlling the false discovery rate and performing statistical contrasts
  publication-title: Hum. Brain Mapp.
– volume: 18
  start-page: 1565
  year: 2015
  end-page: 1567
  ident: bib45
  article-title: A positive-negative mode of population covariation links brain connectivity, demographics and behavior
  publication-title: Nat. Neurosci.
– volume: 3
  start-page: 28
  year: 2014
  ident: bib52
  article-title: How machine learning is shaping cognitive neuroimaging
  publication-title: GigaScience
– volume: 103
  start-page: 3863
  year: 2006
  ident: bib24
  article-title: Information-based functional brain mapping
  publication-title: Proc. Natl. Acad. Sci. U. S. A.
– volume: 45
  start-page: 2041
  year: 2012
  end-page: 2049
  ident: bib27
  article-title: A supervised clustering approach for fmri-based inference of brain states
  publication-title: Pattern Recognit.
– start-page: 1
  year: 2014
  end-page: 6
  ident: bib42
  article-title: The tenth annual mlsp competition: schizophrenia classification challenge
  publication-title: Machine Learning for Signal Processing (MLSP)
– year: 2016
  ident: bib7
  article-title: Machine Learning on Human Connectome Data from mri
– year: 2016
  ident: bib20
  article-title: Recursive Nearest Agglomeration (Rena): Fast Clustering for Approximation of Structured Signals
– year: 2009
  ident: bib16
  article-title: The Elements of Statistical Learning
– volume: 145
  start-page: 166
  year: 2017
  end-page: 179
  ident: bib51
  article-title: Assessing and tuning brain decoders: cross-validation, caveats, and guidelines
  publication-title: NeuroImage
– volume: 45
  start-page: S199
  year: 2009
  ident: 10.1016/j.neuroimage.2017.06.061_bib33
  article-title: Machine learning classifiers and fMRI: a tutorial overview
  publication-title: Neuroimage
  doi: 10.1016/j.neuroimage.2008.11.007
– volume: 9
  start-page: 8
  year: 2015
  ident: 10.1016/j.neuroimage.2017.06.061_bib15
  article-title: Neurovault. org: a web-based repository for collecting and sharing unthresholded statistical maps of the human brain
  publication-title: Front. Neuroinform.
  doi: 10.3389/fninf.2015.00008
– year: 2007
  ident: 10.1016/j.neuroimage.2017.06.061_bib31
– volume: 7
  start-page: 12
  year: 2013
  ident: 10.1016/j.neuroimage.2017.06.061_bib36
  article-title: Toward open sharing of task-based fmri data: the openfmri project
  publication-title: Front. Neuroinform.
  doi: 10.3389/fninf.2013.00012
– year: 2016
  ident: 10.1016/j.neuroimage.2017.06.061_bib58
– volume: 21
  start-page: 1641
  year: 2011
  ident: 10.1016/j.neuroimage.2017.06.061_bib29
  article-title: Reconstructing visual experiences from brain activity evoked by natural movies
  publication-title: Curr. Biol.
  doi: 10.1016/j.cub.2011.08.031
– year: 2017
  ident: 10.1016/j.neuroimage.2017.06.061_bib40
– volume: 25
  start-page: 155
  year: 2005
  ident: 10.1016/j.neuroimage.2017.06.061_bib25
  article-title: Ale meta-analysis: controlling the false discovery rate and performing statistical contrasts
  publication-title: Hum. Brain Mapp.
  doi: 10.1002/hbm.20136
– start-page: 1
  year: 2014
  ident: 10.1016/j.neuroimage.2017.06.061_bib42
  article-title: The tenth annual mlsp competition: schizophrenia classification challenge
– volume: 145
  start-page: 137
  year: 2017
  ident: 10.1016/j.neuroimage.2017.06.061_bib2
  article-title: Single subject prediction of brain disorders in neuroimaging: promises and pitfalls
  publication-title: NeuroImage
  doi: 10.1016/j.neuroimage.2016.02.079
– volume: 72
  start-page: 404
  year: 2011
  ident: 10.1016/j.neuroimage.2017.06.061_bib18
  article-title: A common, high-dimensional model of the representational space in human ventral temporal cortex
  publication-title: Neuron
  doi: 10.1016/j.neuron.2011.08.026
– volume: 18
  start-page: 115
  year: 2017
  ident: 10.1016/j.neuroimage.2017.06.061_bib35
  article-title: Scanning the horizon: future challenges for neuroimaging research
  publication-title: Nat. Rev. Neurosci.
  doi: 10.1038/nrn.2016.167
– volume: 22
  start-page: 1359
  year: 2011
  ident: 10.1016/j.neuroimage.2017.06.061_bib43
  article-title: False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant
  publication-title: Psychol. Sci.
  doi: 10.1177/0956797611417632
– volume: 3
  start-page: 28
  year: 2014
  ident: 10.1016/j.neuroimage.2017.06.061_bib52
  article-title: How machine learning is shaping cognitive neuroimaging
  publication-title: GigaScience
  doi: 10.1186/2047-217X-3-28
– volume: 56
  start-page: 476
  year: 2011
  ident: 10.1016/j.neuroimage.2017.06.061_bib32
  article-title: Information mapping with pattern classifiers: a comparative study
  publication-title: Neuroimage
  doi: 10.1016/j.neuroimage.2010.05.026
– volume: 8
  start-page: 91
  year: 2007
  ident: 10.1016/j.neuroimage.2017.06.061_bib34
  article-title: Fast reproducible identification and large-scale databasing of individual functional cognitive networks
  publication-title: BMC Neurosci.
  doi: 10.1186/1471-2202-8-91
– volume: 368
  start-page: 1388
  year: 2013
  ident: 10.1016/j.neuroimage.2017.06.061_bib54
  article-title: An fMRI-based neurologic signature of physical pain
  publication-title: N. Engl. J. Med.
  doi: 10.1056/NEJMoa1204471
– volume: 2
  start-page: e124
  year: 2005
  ident: 10.1016/j.neuroimage.2017.06.061_bib21
  article-title: Why most published research findings are false
  publication-title: PLOS Med.
  doi: 10.1371/journal.pmed.0020124
– volume: 5
  start-page: 1089
  year: 2004
  ident: 10.1016/j.neuroimage.2017.06.061_bib4
  article-title: No unbiased estimator of the variance of k-fold cross-validation
  publication-title: J. Mach. Learn. Res.
– volume: 4
  start-page: 40
  year: 2010
  ident: 10.1016/j.neuroimage.2017.06.061_bib3
  article-title: A survey of cross-validation procedures for model selection
  publication-title: Stat. Surv.
  doi: 10.1214/09-SS054
– volume: 63
  start-page: 289
  year: 2012
  ident: 10.1016/j.neuroimage.2017.06.061_bib9
  article-title: The secret lives of experiments: methods reporting in the fmri literature
  publication-title: Neuroimage
  doi: 10.1016/j.neuroimage.2012.07.004
– volume: 86
  start-page: 638
  year: 1979
  ident: 10.1016/j.neuroimage.2017.06.061_bib39
  article-title: The file drawer problem and tolerance for null results
  publication-title: Psychol. Bull.
  doi: 10.1037/0033-2909.86.3.638
– volume: 19
  start-page: 659
  year: 2014
  ident: 10.1016/j.neuroimage.2017.06.061_bib13
  article-title: The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism
  publication-title: Mol. psychiatry
  doi: 10.1038/mp.2013.78
– volume: 19
  start-page: 1523
  issue: 11
  year: 2016
  ident: 10.1016/j.neuroimage.2017.06.061_bib28
  article-title: Multimodal population brain imaging in the UK biobank prospective epidemiological study
  publication-title: Nat. Neurosci.
  doi: 10.1038/nn.4393
– volume: 452
  start-page: 352
  year: 2008
  ident: 10.1016/j.neuroimage.2017.06.061_bib23
  article-title: Identifying natural images from human brain activity
  publication-title: Nature
  doi: 10.1038/nature06713
– volume: 7
  start-page: 1
  year: 2006
  ident: 10.1016/j.neuroimage.2017.06.061_bib12
  article-title: Statistical comparisons of classifiers over multiple data sets
  publication-title: J. Mach. Learn. Res.
– volume: 7
  start-page: 523
  year: 2006
  ident: 10.1016/j.neuroimage.2017.06.061_bib19
  article-title: Decoding mental states from brain activity in humans
  publication-title: Nat. Rev. Neurosci.
  doi: 10.1038/nrn1931
– volume: 149
  start-page: 220
  year: 2017
  ident: 10.1016/j.neuroimage.2017.06.061_bib11
  article-title: Statistical power and prediction accuracy in multisite resting-state fmri connectivity
  publication-title: NeuroImage
  doi: 10.1016/j.neuroimage.2017.01.072
– volume: 10
  start-page: 424
  year: 2006
  ident: 10.1016/j.neuroimage.2017.06.061_bib30
  article-title: Beyond mind-reading: multi-voxel pattern analysis of fMRI data
  publication-title: Trends Cogn. Sci.
  doi: 10.1016/j.tics.2006.07.005
– year: 2016
  ident: 10.1016/j.neuroimage.2017.06.061_bib44
– volume: 45
  start-page: 2041
  year: 2012
  ident: 10.1016/j.neuroimage.2017.06.061_bib27
  article-title: A supervised clustering approach for fmri-based inference of brain states
  publication-title: Pattern Recognit.
  doi: 10.1016/j.patcog.2011.04.006
– volume: 293
  start-page: 2425
  year: 2001
  ident: 10.1016/j.neuroimage.2017.06.061_bib17
  article-title: Distributed and overlapping representations of faces and objects in ventral temporal cortex
  publication-title: Science
  doi: 10.1126/science.1063736
– volume: 6
  start-page: 1
  year: 2017
  ident: 10.1016/j.neuroimage.2017.06.061_bib26
– year: 2012
  ident: 10.1016/j.neuroimage.2017.06.061_bib50
  article-title: Small-sample brain mapping: sparse recovery on spatially correlated designs with randomization and clustering
– volume: 80
  start-page: 62
  year: 2013
  ident: 10.1016/j.neuroimage.2017.06.061_bib49
  article-title: The wu-minn human connectome project: an overview
  publication-title: Neuroimage
  doi: 10.1016/j.neuroimage.2013.05.041
– volume: 19
  start-page: 640
  year: 2008
  ident: 10.1016/j.neuroimage.2017.06.061_bib22
  article-title: Why most discovered true associations are inflated
  publication-title: Epidemiology
  doi: 10.1097/EDE.0b013e31818131e7
– volume: 20
  start-page: 365
  year: 2017
  ident: 10.1016/j.neuroimage.2017.06.061_bib56
  article-title: Building better biomarkers: brain models in translational neuroimaging
  publication-title: Nat. Neurosci.
  doi: 10.1038/nn.4478
– year: 2016
  ident: 10.1016/j.neuroimage.2017.06.061_bib47
– volume: 147
  start-page: 736
  year: 2017
  ident: 10.1016/j.neuroimage.2017.06.061_bib1
  article-title: Deriving reproducible biomarkers from multi-site resting-state data: an autism-based example
  publication-title: NeuroImage
  doi: 10.1016/j.neuroimage.2016.10.045
– volume: 97
  start-page: 333
  year: 2014
  ident: 10.1016/j.neuroimage.2017.06.061_bib59
  article-title: Individualized gaussian process-based prediction and detection of local and global gray matter abnormalities in elderly subjects
  publication-title: NeuroImage
  doi: 10.1016/j.neuroimage.2014.04.018
– volume: 14
  start-page: 365
  year: 2013
  ident: 10.1016/j.neuroimage.2017.06.061_bib8
  article-title: Power failure: why small sample size undermines the reliability of neuroscience
  publication-title: Nat. Rev. Neurosci.
  doi: 10.1038/nrn3475
– volume: 20
  start-page: 1364
  year: 2009
  ident: 10.1016/j.neuroimage.2017.06.061_bib38
  article-title: Decoding the large-scale structure of brain function by classifying mental states across individuals
  publication-title: Psychol. Sci.
  doi: 10.1111/j.1467-9280.2009.02460.x
– volume: 18
  start-page: 1565
  year: 2015
  ident: 10.1016/j.neuroimage.2017.06.061_bib45
  article-title: A positive-negative mode of population covariation links brain connectivity, demographics and behavior
  publication-title: Nat. Neurosci.
  doi: 10.1038/nn.4125
– volume: 30
  start-page: 1088
  year: 2006
  ident: 10.1016/j.neuroimage.2017.06.061_bib41
  article-title: Divide and conquer: a defense of functional localizers
  publication-title: Neuroimage
  doi: 10.1016/j.neuroimage.2005.12.062
– volume: 7
  start-page: 1
  year: 2016
  ident: 10.1016/j.neuroimage.2017.06.061_bib57
  article-title: A small number of abnormal brain connections predicts adult autism spectrum disorder
  publication-title: NATURE
– volume: 3
  start-page: 33
  year: 2009
  ident: 10.1016/j.neuroimage.2017.06.061_bib10
  article-title: Pooling fmri data: meta-analysis, mega-analysis and multi-center studies
  publication-title: Front. Neuroinformatics
  doi: 10.3389/neuro.11.033.2009
– volume: 17
  start-page: 1510
  year: 2014
  ident: 10.1016/j.neuroimage.2017.06.061_bib37
  article-title: Making big data open: data sharing in neuroimaging
  publication-title: Nat. Neurosci.
  doi: 10.1038/nn.3818
– volume: 4
  start-page: 274
  year: 2009
  ident: 10.1016/j.neuroimage.2017.06.061_bib53
  article-title: Puzzlingly high correlations in fmri studies of emotion, personality, and social cognition
  publication-title: Perspect. Psychol. Sci.
  doi: 10.1111/j.1745-6924.2009.01125.x
– volume: 57
  start-page: 328
  year: 2015
  ident: 10.1016/j.neuroimage.2017.06.061_bib55
  article-title: From estimating activation locality to predicting disorder: a review of pattern recognition for neuroimaging-based psychiatric diagnostics
  publication-title: Neurosci. Biobehav. Rev.
  doi: 10.1016/j.neubiorev.2015.08.001
– volume: 349
  start-page: 636
  year: 2015
  ident: 10.1016/j.neuroimage.2017.06.061_bib14
  article-title: The reusable holdout: preserving validity in adaptive data analysis
  publication-title: Science
  doi: 10.1126/science.aaa9375
– volume: 145
  start-page: 166
  year: 2017
  ident: 10.1016/j.neuroimage.2017.06.061_bib51
  article-title: Assessing and tuning brain decoders: cross-validation, caveats, and guidelines
  publication-title: NeuroImage
  doi: 10.1016/j.neuroimage.2016.10.038
– volume: 103
  start-page: 3863
  year: 2006
  ident: 10.1016/j.neuroimage.2017.06.061_bib24
  article-title: Information-based functional brain mapping
  publication-title: Proc. Natl. Acad. Sci. U. S. A.
  doi: 10.1073/pnas.0600244103
– year: 2016
  ident: 10.1016/j.neuroimage.2017.06.061_bib20
– volume: 65
  start-page: 69
  year: 2013
  ident: 10.1016/j.neuroimage.2017.06.061_bib46
  article-title: Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (mvpa): random permutations and cluster size control
  publication-title: Neuroimage
  doi: 10.1016/j.neuroimage.2012.09.063
– volume: 20
  start-page: 374
  year: 2004
  ident: 10.1016/j.neuroimage.2017.06.061_bib6
  article-title: Is cross-validation valid for small-sample microarray classification?
  publication-title: Bioinformatics
  doi: 10.1093/bioinformatics/btg419
– year: 2016
  ident: 10.1016/j.neuroimage.2017.06.061_bib7
– volume: 8
  start-page: 153
  year: 2014
  ident: 10.1016/j.neuroimage.2017.06.061_bib48
  article-title: The enigma consortium: large-scale collaborative analyses of neuroimaging and genetic data
  publication-title: Brain Imaging Behav.
  doi: 10.1007/s11682-013-9269-5
– volume: 107
  start-page: 4734
  year: 2010
  ident: 10.1016/j.neuroimage.2017.06.061_bib5
  article-title: Toward discovery science of human brain function
  publication-title: Proc. Ntl Acad. Sci.
  doi: 10.1073/pnas.0911855107
– year: 2009
  ident: 10.1016/j.neuroimage.2017.06.061_bib16
SSID ssj0009148
Score 2.6924677
Snippet Predictive models ground many state-of-the-art developments in statistical brain image analysis: decoding, MVPA, searchlight, or extraction of biomarkers. The...
SourceID hal
proquest
pubmed
crossref
elsevier
SourceType Open Access Repository
Aggregation Database
Index Database
Enrichment Source
Publisher
StartPage 68
SubjectTerms Accuracy
Bioinformatics
Biomarkers
Brain Mapping - methods
Brain Mapping - standards
Cognitive ability
Cognitive science
Computer Science
Cross-validation
Decoding
Experiments
fMRI
Humans
Image processing
Image Processing, Computer-Assisted - methods
Image Processing, Computer-Assisted - standards
Machine Learning
Magnetic Resonance Imaging - methods
Magnetic Resonance Imaging - standards
Medical imaging
Methodology
Model selection
MVPA
Neuroimaging
Neuroscience
Psychology
Reproducibility of Results
Sample Size
Simulation
Statistical analysis
Statistics
Title Cross-validation failure: Small sample sizes lead to large error bars
URI https://www.clinicalkey.com/#!/content/1-s2.0-S1053811917305311
https://dx.doi.org/10.1016/j.neuroimage.2017.06.061
https://www.ncbi.nlm.nih.gov/pubmed/28655633
https://www.proquest.com/docview/2097510562
https://www.proquest.com/docview/1914578908
https://inria.hal.science/hal-01545002
Volume 180
WOSCitedRecordID wos000443268900006&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVESC
  databaseName: Elsevier SD Freedom Collection Journals 2021
  customDbUrl:
  eissn: 1095-9572
  dateEnd: 20191231
  omitProxy: false
  ssIdentifier: ssj0009148
  issn: 1053-8119
  databaseCode: AIEXJ
  dateStart: 19950301
  isFulltext: true
  titleUrlDefault: https://www.sciencedirect.com
  providerName: Elsevier
– providerCode: PRVPQU
  databaseName: Biological Science Database
  customDbUrl:
  eissn: 1095-9572
  dateEnd: 20251009
  omitProxy: false
  ssIdentifier: ssj0009148
  issn: 1053-8119
  databaseCode: M7P
  dateStart: 19980501
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/biologicalscijournals
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Health & Medical Collection
  customDbUrl:
  eissn: 1095-9572
  dateEnd: 20251009
  omitProxy: false
  ssIdentifier: ssj0009148
  issn: 1053-8119
  databaseCode: 7X7
  dateStart: 20020801
  isFulltext: true
  titleUrlDefault: https://search.proquest.com/healthcomplete
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: ProQuest Central
  customDbUrl:
  eissn: 1095-9572
  dateEnd: 20251009
  omitProxy: false
  ssIdentifier: ssj0009148
  issn: 1053-8119
  databaseCode: BENPR
  dateStart: 19980501
  isFulltext: true
  titleUrlDefault: https://www.proquest.com/central
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Psychology Database
  customDbUrl:
  eissn: 1095-9572
  dateEnd: 20251009
  omitProxy: false
  ssIdentifier: ssj0009148
  issn: 1053-8119
  databaseCode: M2M
  dateStart: 20020801
  isFulltext: true
  titleUrlDefault: https://www.proquest.com/psychology
  providerName: ProQuest
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1Lj9MwELboLkJceD8KS2UQ10iJnYcDB1RWXS2IVhUPqTfLdiYiKNssSXcP_Ho8jpOeFlXikkhOxnHssWdsf_6GkLcKjFAZg4DnymAIMx3oVECQCWBaiUIpHbpgE9lqJTabfO0X3DoPqxzGRDdQF43BNXI7Sc8zdAZS9uHyd4BRo3B31YfQmJBjZEngDrq33pPuRnF_FC7hgYii3CN5enyX44usLmyvRYBX5lg80-gm8zT5iTjJm5xQZ4zO7v_vbzwg97wbSue93jwkt2D7iNxZ-o32x2RxisULrBpWfdAlWqoKEezv6LcLVde0U0grTLvqD3S0topCdw2tEVZOoW2blmo7Y35Cfpwtvp-eBz7iQmCSmO8CZmIGPAlTBVFehqERiU5CUxgwTAuliyIsdc5CXTJIWQwlCDt_Y2XMVKaZ4vwpOdo2W3hOaM4xHlGRGtBZDJCrgtv8IgAVQpKnYkqyoaKl8XTkGBWjlgPu7JfcN5HEJpIIwUujKYlGycuekuMAmXxoSzkcObWDpLR24wDZ96Osd0t6d-NA6TdWdcaCIpv3-fyLxDTnvlqLdG1fOhm0RfoxpJN7VZmS1-Nj2_txS0dtobnqJLLzJXiW2dbns14jx0-5M8cp5y_-nflLctcW13H8RskJOdq1V_CK3DbXu6prZ2SSbTJ3FTNyPP-02Hy294-L1fqrTV2y5cx1ub97rS-3
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1bb9MwFLbGQGwv3McKAwyCR0uJc7NBCE1jU6e1FRJD2puxnRMtU9aMpBuCH8VvxCe3Pg31ZQ-8tj1p4pybk-98HyFvNVihEw4skNqihJlhJhbAEgHcaJFqbbxGbCKZzcTJifyyRv70szAIq-xzYpOo09LiM3K3SZcJNgMx_3Txg6FqFL5d7SU0Wrc4gl8_3Zat_nj42d3fd5wf7B_vjVmnKsBsFAYLxm3IIYi8WIMvM8-zIjKRZ1MLlhuhTZp6mZHcMxmHmIeQgXB7FJ6FXCeGa3wA6lL-7dDthFAqYsqnS5JfP2xH76KACd-XHXKoxZM1_JT5ucsSCChLGtbQ2L-uHN46RVzmdU1vU_wO7v9vy_aA3OvabLrbxsVDsgbzR-TutAMSPCb7e7gczIVZ3opK0UzniNB_T7-e66KgtUbaZFrnv6GmhQsEuihpgbB5ClVVVtToqn5Cvt3IRWyR9Xk5h21CZYB6S2lswSQhgNRp4I7nA2gPIhmLEUn6G6tsR7eOqh-F6nF1Z2rpEgpdQiHEMPZHxB8sL1rKkRVsZO87qh-pdUVAubq4gu2HwbZru9p2akXrN85VhxNFtvLx7kThZ0177irulfvRTu-dqsuRtVq65oi8Hr522Q1fWek5lJe1QvbBCGe13Xo-bSNg-KtmpjoOgmf_PvgrsjE-nk7U5HB29JxsulNv-Iz9aIesL6pLeEHu2KtFXlcvm2Cm5PtNh8FfDoCIzw
linkToPdf http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9QwELZKQVUvlDdLCxgEx4jEedkghKq2q1Ytq5UAqTfXdiYiVbopybYIflp_XT3OY09Fe-mBa5JxEnteTr75hpB3CgxXKQMvFMpgCzPt6YSDl3JgWvFMKe27ZhPpZMKPj8V0hVz1tTAIq-x9onPUWWXwG7ndpIsUk4GEfcg7WMR0d_zl_JeHHaTwT2vfTqNVkUP489tu35rPB7t2rd8zNt77vrPvdR0GPBNH4dxjJmIQxn6iIBC57xse69g3mQHDNFc6y_xcC-brnEHCIsiB2_0KyyOmUs0Ufgy17v9uiqTlDjY4XRD-BlFbhheHHg8C0aGIWmyZ46oszqzHQHBZ6hhEk-Cm0HjnJ2I0b0qAXSAcb_zPU_iA3O_Sb7rd2stDsgKzR2TtawcweEz2dnBqPGt-RdtsiuaqQOT-R_rtTJUlbRTSKdOm-AsNLa2B0HlFS4TTU6jrqqZa1c0T8uNWXuIpWZ1VM3hOqAixD1OWGNBpBCBUFtrxAgDlQywSPiJpv8jSdDTs2A2klD3e7lQu1EOiekiEHibBiASD5HlLRbKEjOj1SPaltjY4SBsvl5D9NMh26VibZi0p_daq7fCgyGK-v30k8ZhL220kvrQXbfWaKjvf2ciFmo7Im-G09Xr4K0vNoLpoJLISxljDbefzWWsNw61crXUShi_-Pfhrsma1Xx4dTA43ybp9ckdzHMRbZHVeX8BLcs9czoumfuXsmpKT27aCa46PkaA
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cross-validation+failure%3A+Small+sample+sizes+lead+to+large+error+bars&rft.jtitle=NeuroImage+%28Orlando%2C+Fla.%29&rft.au=Varoquaux%2C+Ga%C3%ABl&rft.date=2018-10-15&rft.issn=1053-8119&rft.volume=180&rft.spage=68&rft.epage=77&rft_id=info:doi/10.1016%2Fj.neuroimage.2017.06.061&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_neuroimage_2017_06_061
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1053-8119&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1053-8119&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1053-8119&client=summon