Machine learning reduced workload with minimal risk of missing studies: development and evaluation of a randomized controlled trial classifier for Cochrane Reviews
This study developed, calibrated, and evaluated a machine learning classifier designed to reduce study identification workload in Cochrane for producing systematic reviews. A machine learning classifier for retrieving randomized controlled trials (RCTs) was developed (the “Cochrane RCT Classifier”),...
Uložené v:
| Vydané v: | Journal of clinical epidemiology Ročník 133; s. 140 - 151 |
|---|---|
| Hlavní autori: | , , , , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
United States
Elsevier Inc
01.05.2021
Elsevier Limited Elsevier |
| Predmet: | |
| ISSN: | 0895-4356, 1878-5921, 1878-5921 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | This study developed, calibrated, and evaluated a machine learning classifier designed to reduce study identification workload in Cochrane for producing systematic reviews.
A machine learning classifier for retrieving randomized controlled trials (RCTs) was developed (the “Cochrane RCT Classifier”), with the algorithm trained using a data set of title–abstract records from Embase, manually labeled by the Cochrane Crowd. The classifier was then calibrated using a further data set of similar records manually labeled by the Clinical Hedges team, aiming for 99% recall. Finally, the recall of the calibrated classifier was evaluated using records of RCTs included in Cochrane Reviews that had abstracts of sufficient length to allow machine classification.
The Cochrane RCT Classifier was trained using 280,620 records (20,454 of which reported RCTs). A classification threshold was set using 49,025 calibration records (1,587 of which reported RCTs), and our bootstrap validation found the classifier had recall of 0.99 (95% confidence interval 0.98–0.99) and precision of 0.08 (95% confidence interval 0.06–0.12) in this data set. The final, calibrated RCT classifier correctly retrieved 43,783 (99.5%) of 44,007 RCTs included in Cochrane Reviews but missed 224 (0.5%). Older records were more likely to be missed than those more recently published.
The Cochrane RCT Classifier can reduce manual study identification workload for Cochrane Reviews, with a very low and acceptable risk of missing eligible RCTs. This classifier now forms part of the Evidence Pipeline, an integrated workflow deployed within Cochrane to help improve the efficiency of the study identification processes that support systematic review production.
•Systematic review processes need to become more efficient.•Machine learning is sufficiently mature for real-world use.•A machine learning classifier was built using data from Cochrane Crowd.•It was calibrated to achieve very high recall.•It is now live and in use in Cochrane review production systems. |
|---|---|
| AbstractList | This study developed, calibrated, and evaluated a machine learning classifier designed to reduce study identification workload in Cochrane for producing systematic reviews.
A machine learning classifier for retrieving randomized controlled trials (RCTs) was developed (the “Cochrane RCT Classifier”), with the algorithm trained using a data set of title–abstract records from Embase, manually labeled by the Cochrane Crowd. The classifier was then calibrated using a further data set of similar records manually labeled by the Clinical Hedges team, aiming for 99% recall. Finally, the recall of the calibrated classifier was evaluated using records of RCTs included in Cochrane Reviews that had abstracts of sufficient length to allow machine classification.
The Cochrane RCT Classifier was trained using 280,620 records (20,454 of which reported RCTs). A classification threshold was set using 49,025 calibration records (1,587 of which reported RCTs), and our bootstrap validation found the classifier had recall of 0.99 (95% confidence interval 0.98–0.99) and precision of 0.08 (95% confidence interval 0.06–0.12) in this data set. The final, calibrated RCT classifier correctly retrieved 43,783 (99.5%) of 44,007 RCTs included in Cochrane Reviews but missed 224 (0.5%). Older records were more likely to be missed than those more recently published.
The Cochrane RCT Classifier can reduce manual study identification workload for Cochrane Reviews, with a very low and acceptable risk of missing eligible RCTs. This classifier now forms part of the Evidence Pipeline, an integrated workflow deployed within Cochrane to help improve the efficiency of the study identification processes that support systematic review production.
•Systematic review processes need to become more efficient.•Machine learning is sufficiently mature for real-world use.•A machine learning classifier was built using data from Cochrane Crowd.•It was calibrated to achieve very high recall.•It is now live and in use in Cochrane review production systems. • Systematic review processes need to become more efficient. • Machine learning is sufficiently mature for real-world use. • A machine learning classifier was built using data from Cochrane Crowd. • It was calibrated to achieve very high recall. • It is now live and in use in Cochrane review production systems. This study developed, calibrated, and evaluated a machine learning classifier designed to reduce study identification workload in Cochrane for producing systematic reviews. A machine learning classifier for retrieving randomized controlled trials (RCTs) was developed (the "Cochrane RCT Classifier"), with the algorithm trained using a data set of title-abstract records from Embase, manually labeled by the Cochrane Crowd. The classifier was then calibrated using a further data set of similar records manually labeled by the Clinical Hedges team, aiming for 99% recall. Finally, the recall of the calibrated classifier was evaluated using records of RCTs included in Cochrane Reviews that had abstracts of sufficient length to allow machine classification. The Cochrane RCT Classifier was trained using 280,620 records (20,454 of which reported RCTs). A classification threshold was set using 49,025 calibration records (1,587 of which reported RCTs), and our bootstrap validation found the classifier had recall of 0.99 (95% confidence interval 0.98-0.99) and precision of 0.08 (95% confidence interval 0.06-0.12) in this data set. The final, calibrated RCT classifier correctly retrieved 43,783 (99.5%) of 44,007 RCTs included in Cochrane Reviews but missed 224 (0.5%). Older records were more likely to be missed than those more recently published. The Cochrane RCT Classifier can reduce manual study identification workload for Cochrane Reviews, with a very low and acceptable risk of missing eligible RCTs. This classifier now forms part of the Evidence Pipeline, an integrated workflow deployed within Cochrane to help improve the efficiency of the study identification processes that support systematic review production. This study developed, calibrated, and evaluated a machine learning classifier designed to reduce study identification workload in Cochrane for producing systematic reviews.OBJECTIVESThis study developed, calibrated, and evaluated a machine learning classifier designed to reduce study identification workload in Cochrane for producing systematic reviews.A machine learning classifier for retrieving randomized controlled trials (RCTs) was developed (the "Cochrane RCT Classifier"), with the algorithm trained using a data set of title-abstract records from Embase, manually labeled by the Cochrane Crowd. The classifier was then calibrated using a further data set of similar records manually labeled by the Clinical Hedges team, aiming for 99% recall. Finally, the recall of the calibrated classifier was evaluated using records of RCTs included in Cochrane Reviews that had abstracts of sufficient length to allow machine classification.METHODSA machine learning classifier for retrieving randomized controlled trials (RCTs) was developed (the "Cochrane RCT Classifier"), with the algorithm trained using a data set of title-abstract records from Embase, manually labeled by the Cochrane Crowd. The classifier was then calibrated using a further data set of similar records manually labeled by the Clinical Hedges team, aiming for 99% recall. Finally, the recall of the calibrated classifier was evaluated using records of RCTs included in Cochrane Reviews that had abstracts of sufficient length to allow machine classification.The Cochrane RCT Classifier was trained using 280,620 records (20,454 of which reported RCTs). A classification threshold was set using 49,025 calibration records (1,587 of which reported RCTs), and our bootstrap validation found the classifier had recall of 0.99 (95% confidence interval 0.98-0.99) and precision of 0.08 (95% confidence interval 0.06-0.12) in this data set. The final, calibrated RCT classifier correctly retrieved 43,783 (99.5%) of 44,007 RCTs included in Cochrane Reviews but missed 224 (0.5%). Older records were more likely to be missed than those more recently published.RESULTSThe Cochrane RCT Classifier was trained using 280,620 records (20,454 of which reported RCTs). A classification threshold was set using 49,025 calibration records (1,587 of which reported RCTs), and our bootstrap validation found the classifier had recall of 0.99 (95% confidence interval 0.98-0.99) and precision of 0.08 (95% confidence interval 0.06-0.12) in this data set. The final, calibrated RCT classifier correctly retrieved 43,783 (99.5%) of 44,007 RCTs included in Cochrane Reviews but missed 224 (0.5%). Older records were more likely to be missed than those more recently published.The Cochrane RCT Classifier can reduce manual study identification workload for Cochrane Reviews, with a very low and acceptable risk of missing eligible RCTs. This classifier now forms part of the Evidence Pipeline, an integrated workflow deployed within Cochrane to help improve the efficiency of the study identification processes that support systematic review production.CONCLUSIONSThe Cochrane RCT Classifier can reduce manual study identification workload for Cochrane Reviews, with a very low and acceptable risk of missing eligible RCTs. This classifier now forms part of the Evidence Pipeline, an integrated workflow deployed within Cochrane to help improve the efficiency of the study identification processes that support systematic review production. ObjectivesThis study developed, calibrated, and evaluated a machine learning classifier designed to reduce study identification workload in Cochrane for producing systematic reviews.MethodsA machine learning classifier for retrieving randomized controlled trials (RCTs) was developed (the “Cochrane RCT Classifier”), with the algorithm trained using a data set of title–abstract records from Embase, manually labeled by the Cochrane Crowd. The classifier was then calibrated using a further data set of similar records manually labeled by the Clinical Hedges team, aiming for 99% recall. Finally, the recall of the calibrated classifier was evaluated using records of RCTs included in Cochrane Reviews that had abstracts of sufficient length to allow machine classification.ResultsThe Cochrane RCT Classifier was trained using 280,620 records (20,454 of which reported RCTs). A classification threshold was set using 49,025 calibration records (1,587 of which reported RCTs), and our bootstrap validation found the classifier had recall of 0.99 (95% confidence interval 0.98–0.99) and precision of 0.08 (95% confidence interval 0.06–0.12) in this data set. The final, calibrated RCT classifier correctly retrieved 43,783 (99.5%) of 44,007 RCTs included in Cochrane Reviews but missed 224 (0.5%). Older records were more likely to be missed than those more recently published.ConclusionsThe Cochrane RCT Classifier can reduce manual study identification workload for Cochrane Reviews, with a very low and acceptable risk of missing eligible RCTs. This classifier now forms part of the Evidence Pipeline, an integrated workflow deployed within Cochrane to help improve the efficiency of the study identification processes that support systematic review production. AbstractObjectivesThis study developed, calibrated, and evaluated of a machine learning classifier designed to reduce study identification workload in Cochrane for producing systematic reviews. MethodsA machine learning classifier for retrieving randomized controlled trials (RCTs) was developed (the “Cochrane RCT Classifier”), with the algorithm trained using a data set of title–abstract records from Embase, manually labeled by the Cochrane Crowd. The classifier was then calibrated using a further data set of similar records manually labeled by the Clinical Hedges team, aiming for 99% recall. Finally, the recall of the calibrated classifier was evaluated using records of RCTs included in Cochrane Reviews that had abstracts of sufficient length to allow machine classification. ResultsThe Cochrane RCT Classifier was trained using 280,620 records (20,454 of which reported RCTs). A classification threshold was set using 49,025 calibration records (1,587 of which reported RCTs), and our bootstrap validation found the classifier had recall of 0.99 (95% confidence interval 0.98–0.99) and precision of 0.08 (95% confidence interval 0.06–0.12) in this data set. The final, calibrated RCT classifier correctly retrieved 43,783 (99.5%) of 44,007 RCTs included in Cochrane Reviews but missed 224 (0.5%). Older records were more likely to be missed than those more recently published. ConclusionsThe Cochrane RCT Classifier can reduce manual study identification workload for Cochrane Reviews, with a very low and acceptable risk of missing eligible RCTs. This classifier now forms part of the Evidence Pipeline, an integrated workflow deployed within Cochrane to help improve the efficiency of the study identification processes that support systematic review production. |
| Author | Marshall, Iain J. Elliott, Julian Mavergames, Chris Shemilt, Ian Thomas, James McDonald, Steve Noel-Storr, Anna |
| Author_xml | – sequence: 1 givenname: James surname: Thomas fullname: Thomas, James email: james.thomas@ucl.ac.uk organization: EPPI-Centre, UCL Social Research Institute, University College London, London, UK – sequence: 2 givenname: Steve surname: McDonald fullname: McDonald, Steve organization: Cochrane Australia, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia – sequence: 3 givenname: Anna surname: Noel-Storr fullname: Noel-Storr, Anna organization: Radcliffe Department of Medicine, University of Oxford, Oxford, UK – sequence: 4 givenname: Ian surname: Shemilt fullname: Shemilt, Ian organization: EPPI-Centre, UCL Social Research Institute, University College London, London, UK – sequence: 5 givenname: Julian orcidid: 0000-0002-9165-5875 surname: Elliott fullname: Elliott, Julian organization: Department of Infectious Diseases, Monash University and Alfred Hospital, Melbourne, Australia – sequence: 6 givenname: Chris orcidid: 0000-0003-2762-6196 surname: Mavergames fullname: Mavergames, Chris organization: Cochrane, London, UK – sequence: 7 givenname: Iain J. surname: Marshall fullname: Marshall, Iain J. organization: School of Population Health & Environmental Sciences, Kings College London, London, UK |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/33171275$$D View this record in MEDLINE/PubMed |
| BookMark | eNqNkl1v0zAUhiM0xLrBX5gsccNNi-3E-UBoAlUMkIaQ-Li2HPtkdevaxU46jb_DH90J3SroBeMmsZ3nvDmvz3uSHfngIcvOGJ0xysqXy9lSO-thY2eccjxkM0rzR9mE1VU9FQ1nR9mE1o2YFrkoj7OTlJaUsopW4kl2nOesYrwSk-zXJ6UXqEMcqOitvyIRzKDBkOsQVy4oXNh-QdbW27VyJNq0IqHDfUojnfrBWEiviIEtuLBZg--J8obAVrlB9Tb4EVck4mFY25-orIPvY3AOl320KKqdQrXOQiRdiGQe9AJxIF9ga-E6Pc0ed8oleHb3Ps2-X7z7Nv8wvfz8_uP87eVUl7Tpp0bXNe94qzkrRJFXbddAUTBKdaGLtjJt0SramlJURqmuFUxUbWPAGGhF2YDJT7Pzne5maNdgNFqJyslNROfxRgZl5d9fvF3Iq7CVNSvx1zUKvLgTiOHHAKmXeE0anEMzYUiSF6IpeY5NIfr8AF2GIXq0J7ngTc1LwUbq7M-O9q3czw-BcgfoGFKK0O0RRuUYFLmU90GRY1AkYxKDgoWvDwq17X-PC51Z93D5m1054DxwSFEmbcFjbmwE3UsT7MMS5wcSI2W1ciu4gbS_DiYTl1R-HcM8ZpnTnLEKn_8U-J8ObgHkbA3b |
| CitedBy_id | crossref_primary_10_1136_bmjebm_2022_112019 crossref_primary_10_1002_14651858_CD001072_pub2 crossref_primary_10_1093_cid_ciae476 crossref_primary_10_7326_M24_0877 crossref_primary_10_1186_s13643_023_02171_y crossref_primary_10_1002_14651858_CD013758_pub2 crossref_primary_10_1016_j_mhp_2024_200349 crossref_primary_10_1016_j_zefq_2022_11_008 crossref_primary_10_1038_d41586_021_03690_1 crossref_primary_10_1007_s10462_024_10902_3 crossref_primary_10_1002_14651858_CD006182_pub3 crossref_primary_10_1016_j_compind_2024_104191 crossref_primary_10_1371_journal_pone_0307418 crossref_primary_10_1186_s12961_023_01085_4 crossref_primary_10_1002_clt2_70045 crossref_primary_10_11124_JBIES_23_00139 crossref_primary_10_1002_14651858_CD000143_pub2 crossref_primary_10_1186_s13643_021_01880_6 crossref_primary_10_11124_JBIES_23_00377 crossref_primary_10_3390_make7020028 crossref_primary_10_1002_14651858_CD015952 crossref_primary_10_1002_14651858_CD003281_pub5 crossref_primary_10_1002_14651858_CD012178_pub2 crossref_primary_10_1371_journal_pone_0330288 crossref_primary_10_1038_s41746_022_00730_6 crossref_primary_10_1002_jrsm_1731 crossref_primary_10_1002_14651858_CD015064_pub2 crossref_primary_10_1136_bmjebm_2022_112152 crossref_primary_10_1002_cesm_12021 crossref_primary_10_1027_2151_2604_a000509 crossref_primary_10_1002_14651858_CD015786_pub2 crossref_primary_10_1016_j_jclinepi_2022_04_027 crossref_primary_10_1002_asi_24851 crossref_primary_10_1002_14651858_CD016179 crossref_primary_10_1002_14651858_CD016178 crossref_primary_10_1002_cesm_70030 crossref_primary_10_1080_13678868_2025_2560486 crossref_primary_10_1002_14651858_CD016171 crossref_primary_10_1002_14651858_CD016170 crossref_primary_10_1002_jrsm_1589 crossref_primary_10_1016_j_jbi_2023_104389 crossref_primary_10_1016_j_ajpc_2025_101082 crossref_primary_10_1177_23969873221100032 crossref_primary_10_1017_rsm_2025_10023 crossref_primary_10_1002_cesm_70046 crossref_primary_10_1093_ejo_cjae037 crossref_primary_10_1017_rsm_2025_10028 crossref_primary_10_1111_aas_14295 crossref_primary_10_1186_s13750_025_00362_9 crossref_primary_10_1186_s13643_024_02592_3 crossref_primary_10_1002_14651858_CD016180 crossref_primary_10_1002_14651858_MR000054_pub2 crossref_primary_10_1111_hir_12515 crossref_primary_10_1002_14651858_CD008349_pub5 crossref_primary_10_1186_s12874_024_02320_4 crossref_primary_10_1186_s12874_021_01271_4 crossref_primary_10_1038_s41390_024_03197_1 crossref_primary_10_2106_JBJS_RVW_23_00077 crossref_primary_10_1002_14651858_CD014895_pub2 crossref_primary_10_1002_14651858_CD015983 crossref_primary_10_1002_cesm_70012 crossref_primary_10_1186_s13643_023_02255_9 crossref_primary_10_1136_bmj_r1105 crossref_primary_10_3390_electronics14061110 crossref_primary_10_12688_wellcomeopenres_22551_1 crossref_primary_10_1002_14651858_CD005495_pub5 crossref_primary_10_1017_rsm_2025_3 crossref_primary_10_1111_jebm_12594 crossref_primary_10_1001_jamainternmed_2025_3832 crossref_primary_10_1002_14651858_CD006475_pub3 crossref_primary_10_1186_s12874_021_01451_2 crossref_primary_10_1186_s13643_023_02334_x crossref_primary_10_1002_14651858_CD013396_pub2 crossref_primary_10_1002_14651858_CD016207 crossref_primary_10_1186_s12978_025_01947_w crossref_primary_10_1111_jgs_19457 crossref_primary_10_1016_j_jclinepi_2023_10_007 crossref_primary_10_1038_s41598_025_03979_5 crossref_primary_10_1016_j_jclinepi_2022_08_013 crossref_primary_10_1186_s12874_022_01649_y crossref_primary_10_1186_s12879_023_08304_x crossref_primary_10_1016_j_jclinepi_2022_03_019 crossref_primary_10_1002_14651858_CD015968 crossref_primary_10_1007_s00402_024_05359_6 crossref_primary_10_1111_bph_16100 crossref_primary_10_1016_j_jor_2024_03_012 crossref_primary_10_1002_oby_24172 crossref_primary_10_1002_14651858_CD015894_pub2 crossref_primary_10_3390_ijerph20116020 crossref_primary_10_1093_ageing_afaf171 crossref_primary_10_1002_jrsm_1664 crossref_primary_10_1016_j_jestch_2023_101433 crossref_primary_10_1002_14651858_CD013757_pub3 crossref_primary_10_1002_14651858_CD011677_pub4 crossref_primary_10_1016_j_cgh_2025_01_030 crossref_primary_10_1371_journal_pone_0283342 crossref_primary_10_3310_UDIR6682 crossref_primary_10_1016_j_jclinepi_2023_09_007 crossref_primary_10_1002_cl2_1363 crossref_primary_10_1007_s00607_023_01181_x |
| Cites_doi | 10.1093/jamia/ocx053 10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2 10.1016/j.jclinepi.2017.08.011 10.1016/S0895-4356(01)00341-9 10.7326/0003-4819-147-4-200708210-00179 10.1016/S0140-6736(13)62329-6 10.1186/2046-4053-1-60 10.1371/journal.pmed.1002708 10.1111/j.1471-1842.2008.00827.x 10.1186/2046-4053-4-5 10.1002/jrsm.1287 10.1371/journal.pmed.1000326 10.1136/bmj.f139 10.1016/j.jclinepi.2020.08.008 |
| ContentType | Journal Article |
| Copyright | 2020 The Authors The Authors Copyright © 2020 The Authors. Published by Elsevier Inc. All rights reserved. 2020. The Authors 2020 The Authors 2020 |
| Copyright_xml | – notice: 2020 The Authors – notice: The Authors – notice: Copyright © 2020 The Authors. Published by Elsevier Inc. All rights reserved. – notice: 2020. The Authors – notice: 2020 The Authors 2020 |
| DBID | 6I. AAFTH AAYXX CITATION CGR CUY CVF ECM EIF NPM 3V. 7QL 7QP 7RV 7T2 7T7 7TK 7U7 7U9 7X7 7XB 88C 88E 8AO 8C1 8FD 8FI 8FJ 8FK 8G5 ABUWG AEUYN AFKRA AZQEC BENPR C1K CCPQU DWQXO FR3 FYUFA GHDGH GNUQQ GUQSH H94 K9. KB0 M0S M0T M1P M2O M7N MBDVC NAPCQ P64 PHGZM PHGZT PJZUB PKEHL PPXIY PQEST PQQKQ PQUKI PRINS Q9U 7X8 5PM |
| DOI | 10.1016/j.jclinepi.2020.11.003 |
| DatabaseName | ScienceDirect Open Access Titles Elsevier:ScienceDirect:Open Access CrossRef Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed ProQuest Central (Corporate) Bacteriology Abstracts (Microbiology B) Calcium & Calcified Tissue Abstracts ProQuest Nursing and Allied Health Journals - PSU access expires 11/30/25. Health and Safety Science Abstracts (Full archive) Industrial and Applied Microbiology Abstracts (Microbiology A) Neurosciences Abstracts Toxicology Abstracts Virology and AIDS Abstracts Health & Medical Collection ProQuest Central (purchase pre-March 2016) Healthcare Administration Database (Alumni) Medical Database (Alumni Edition) ProQuest Pharma Collection Public Health Database Technology Research Database Hospital Premium Collection Hospital Premium Collection (Alumni Edition) ProQuest Central (Alumni) (purchase pre-March 2016) Research Library (Alumni Edition) ProQuest Central (Alumni) ProQuest One Sustainability (subscription) ProQuest Central UK/Ireland ProQuest Central Essentials ProQuest Central Environmental Sciences and Pollution Management ProQuest One Community College ProQuest Central Engineering Research Database Health Research Premium Collection Health Research Premium Collection (Alumni) ProQuest Central Student Research Library Prep AIDS and Cancer Research Abstracts ProQuest Health & Medical Complete (Alumni) Nursing & Allied Health Database (Alumni Edition) Health & Medical Collection (Alumni Edition) Healthcare Administration Database PML(ProQuest Medical Library) Research Library (subscription) Algology Mycology and Protozoology Abstracts (Microbiology C) Research Library (Corporate) Nursing & Allied Health Premium Biotechnology and BioEngineering Abstracts ProQuest Central Premium ProQuest One Academic (New) ProQuest Health & Medical Research Collection ProQuest One Academic Middle East (New) ProQuest One Health & Nursing ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China ProQuest Central Basic MEDLINE - Academic PubMed Central (Full Participant titles) |
| DatabaseTitle | CrossRef MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) Research Library Prep ProQuest Central Student ProQuest Central Essentials ProQuest Central China Environmental Sciences and Pollution Management ProQuest One Sustainability Health Research Premium Collection Health & Medical Research Collection Industrial and Applied Microbiology Abstracts (Microbiology A) ProQuest Central (New) ProQuest Medical Library (Alumni) Virology and AIDS Abstracts ProQuest One Academic Eastern Edition ProQuest Hospital Collection Health Research Premium Collection (Alumni) Neurosciences Abstracts ProQuest Hospital Collection (Alumni) Biotechnology and BioEngineering Abstracts Nursing & Allied Health Premium ProQuest Health & Medical Complete ProQuest One Academic UKI Edition ProQuest Health Management (Alumni Edition) ProQuest Nursing & Allied Health Source (Alumni) Engineering Research Database ProQuest One Academic Calcium & Calcified Tissue Abstracts ProQuest One Academic (New) Technology Research Database ProQuest One Academic Middle East (New) ProQuest Health & Medical Complete (Alumni) ProQuest Central (Alumni Edition) ProQuest One Community College ProQuest One Health & Nursing Research Library (Alumni Edition) ProQuest Pharma Collection ProQuest Central ProQuest Health & Medical Research Collection Health and Medicine Complete (Alumni Edition) ProQuest Central Korea Bacteriology Abstracts (Microbiology B) Algology Mycology and Protozoology Abstracts (Microbiology C) AIDS and Cancer Research Abstracts ProQuest Research Library Health & Safety Science Abstracts ProQuest Public Health ProQuest Central Basic Toxicology Abstracts ProQuest Health Management ProQuest Nursing & Allied Health Source ProQuest Medical Library ProQuest Central (Alumni) MEDLINE - Academic |
| DatabaseTitleList | MEDLINE MEDLINE - Academic Research Library Prep |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Medicine |
| EISSN | 1878-5921 |
| EndPage | 151 |
| ExternalDocumentID | PMC8168828 33171275 10_1016_j_jclinepi_2020_11_003 S0895435620311720 1_s2_0_S0895435620311720 |
| Genre | Research Support, Non-U.S. Gov't Journal Article Research Support, N.I.H., Extramural |
| GrantInformation_xml | – fundername: U.S. National Library of Medicine grantid: 2R01-LM012086-05 funderid: https://doi.org/10.13039/100000092 – fundername: Medical Research Council grantid: MR/N015185/1 funderid: https://doi.org/10.13039/501100000265 – fundername: National Health & Medical Research Council grantid: APP1114605 funderid: https://doi.org/10.13039/501100000925 – fundername: National Institute for Health Research funderid: https://doi.org/10.13039/501100000272 – fundername: Department of Health – fundername: Medical Research Council grantid: MR/J005037/1 – fundername: NLM NIH HHS grantid: R01 LM012086 – fundername: Medical Research Council grantid: MR/N015185/1 |
| GroupedDBID | --- --K --M -~X .1- .55 .FO .GJ .~1 0R~ 1B1 1P~ 1RT 1~. 1~5 29K 4.4 457 4CK 4G. 53G 5GY 5RE 5VS 7-5 71M 7RV 7X7 88E 8AO 8C1 8FI 8FJ 8G5 8P~ 9JM 9JO AABNK AAEDT AAEDW AAFJI AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AATTM AAWTL AAXKI AAXUO AAYJJ AAYWO ABBQC ABFNM ABIVO ABJNI ABLJU ABMAC ABMMH ABMZM ABOCM ABUWG ABWVN ABXDB ACDAQ ACGFS ACIEU ACIUM ACLOT ACPRK ACRLP ACRPL ACVFH ADBBV ADCNI ADEZE ADMUD ADNMO AEBSH AEIPS AEKER AENEX AEUPX AEUYN AEVXI AFFNX AFJKZ AFKRA AFPUW AFRAH AFRHN AFTJW AFXIZ AGHFR AGQPQ AGUBO AGYEJ AHHHB AHMBA AIEXJ AIGII AIIUN AIKHN AITUG AJRQY AJUYK AKBMS AKRWK AKYEP ALMA_UNASSIGNED_HOLDINGS AMRAJ ANKPU ANZVX AOMHK APXCP AQUVI ASPBG AVARZ AVWKF AXJTR AZFZN AZQEC BENPR BKEYQ BKOJK BLXMC BNPGV BPHCQ BVXVI CCPQU CS3 D-I DU5 DWQXO EBS EFJIC EFKBS EFLBG EJD EMOBN EO8 EO9 EP2 EP3 EX3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN FYUFA G-2 G-Q GBLVA GNUQQ GUQSH HEH HMCUK HMK HMO HVGLF HZ~ IHE J1W KOM L7B M0T M1P M29 M2O M3W M41 MO0 N9A NAPCQ O-L O9- OAUVE OD~ OHT OO0 OZT P-8 P-9 P2P PC. PHGZM PHGZT PJZUB PPXIY PQQKQ PRBVW PROAC PSQYO Q38 R2- ROL RPZ SAE SCC SDF SDG SDP SEL SES SEW SPCBC SSB SSH SSO SSZ SV3 T5K UAP UKHRP WOW WUQ X7M XPP YHZ Z5R ZGI ~G- ~HD 3V. AACTN AFCTW AFKWA AJOXV ALIPV AMFUW RIG 6I. AAFTH AAIAV ABLVK ABYKQ AHPSJ AJBFU AKYCK F3I LCYCR ZA5 9DU AAYXX AFFHD CITATION AGCQF AGRNS CGR CUY CVF ECM EIF NPM 7QL 7QP 7T2 7T7 7TK 7U7 7U9 7XB 8FD 8FK C1K FR3 H94 K9. M7N MBDVC P64 PKEHL PQEST PQUKI PRINS Q9U 7X8 PUEGO 5PM |
| ID | FETCH-LOGICAL-c609t-dc882f2bc2145437bf9e44100c4c4b7db4ba0bd657daafb5157b9deddeb569ed3 |
| IEDL.DBID | 7RV |
| ISICitedReferencesCount | 118 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000653651200016&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0895-4356 1878-5921 |
| IngestDate | Tue Nov 04 01:47:32 EST 2025 Thu Oct 02 04:10:46 EDT 2025 Sat Nov 29 14:55:54 EST 2025 Mon Jul 21 06:05:00 EDT 2025 Tue Nov 18 22:14:29 EST 2025 Sat Nov 29 07:14:19 EST 2025 Fri Feb 23 02:45:58 EST 2024 Tue Feb 25 20:07:18 EST 2025 Tue Oct 14 19:36:03 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Crowdsourcing Methods/methodology Automation Searching Cochrane Library Machine learning Systematic reviews Study classifiers Information retrieval Randomized controlled trials |
| Language | English |
| License | This is an open access article under the CC BY license. Copyright © 2020 The Authors. Published by Elsevier Inc. All rights reserved. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c609t-dc882f2bc2145437bf9e44100c4c4b7db4ba0bd657daafb5157b9deddeb569ed3 |
| Notes | ObjectType-Article-2 SourceType-Scholarly Journals-1 content type line 14 ObjectType-Feature-3 ObjectType-Evidence Based Healthcare-1 ObjectType-Article-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0003-2762-6196 0000-0002-9165-5875 |
| OpenAccessLink | https://pubmed.ncbi.nlm.nih.gov/PMC8168828 |
| PMID | 33171275 |
| PQID | 2529826511 |
| PQPubID | 105585 |
| PageCount | 12 |
| ParticipantIDs | pubmedcentral_primary_oai_pubmedcentral_nih_gov_8168828 proquest_miscellaneous_2459623441 proquest_journals_2529826511 pubmed_primary_33171275 crossref_primary_10_1016_j_jclinepi_2020_11_003 crossref_citationtrail_10_1016_j_jclinepi_2020_11_003 elsevier_sciencedirect_doi_10_1016_j_jclinepi_2020_11_003 elsevier_clinicalkeyesjournals_1_s2_0_S0895435620311720 elsevier_clinicalkey_doi_10_1016_j_jclinepi_2020_11_003 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-05-01 |
| PublicationDateYYYYMMDD | 2021-05-01 |
| PublicationDate_xml | – month: 05 year: 2021 text: 2021-05-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States – name: Elmsford |
| PublicationTitle | Journal of clinical epidemiology |
| PublicationTitleAlternate | J Clin Epidemiol |
| PublicationYear | 2021 |
| Publisher | Elsevier Inc Elsevier Limited Elsevier |
| Publisher_xml | – name: Elsevier Inc – name: Elsevier Limited – name: Elsevier |
| References | McKibbon, Lou, Haynes (bib21) 2009; 26 Shojania, Sampson, Ansari, Ji, Doucette (bib4) 2007; 147 Marshall, Storr, Kuiper, Thomas, Wallace (bib8) 2018; 9 Turner, Shamseer, Altman, Schulz, Moher (bib22) 2012; 29 Thomas, Noel-Storr, Marshall, Wallace, McDonald, Mavergames (bib10) 2017; 91 Brier (bib20) 1950 Noel-Storr, Dooley, Wisneiwski, Glanville, Thomas, Cox (bib12) 2020; 127 Bastian, Glasziou, Chalmers (bib3) 2010; 7 O’Mara-Eves, Thomas, McNaught, Miwa, Ananiadou (bib7) 2015; 4 Sain, Vapnik (bib17) 2006 (bib11) 2019 Platt (bib18) 1999 . Wallace, Noel-Storr, Marshall, Cohen, Smalheiser, Thomas (bib9) 2017; 24 Schulz, Altman, Moher (bib16) 2010; 8 Lefebvre, Glanville, Briscoe, Littlewood, Marshall, Metzendorf (bib2) 2019 Macleod, Michie, Roberts, Dirnagl, Chalmers, Ioannidis (bib5) 2014; 383 Nevin (bib14) 2018; 15 Steyerberg, Harrell, Borsboom, Eijkemans, Vergouwe, Habbema (bib19) 2001 (bib1) 2019 Noel-Storr A, Dooley G, Elliott J, Steele E, Shemilt I, Mavergames C, et al. An evaluation of Cochrane Crowd finds that crowdsourcing can help to address the challenge of information overload in evidence synthesis. Tsafnat, Dunn, Glasziou, Coiera (bib6) 2013; 346 Wilczynski, Douglas, Haynes (bib15) 2005; 5 Noel-Storr (10.1016/j.jclinepi.2020.11.003_bib12) 2020; 127 Tsafnat (10.1016/j.jclinepi.2020.11.003_bib6) 2013; 346 Bastian (10.1016/j.jclinepi.2020.11.003_bib3) 2010; 7 Macleod (10.1016/j.jclinepi.2020.11.003_bib5) 2014; 383 O’Mara-Eves (10.1016/j.jclinepi.2020.11.003_bib7) 2015; 4 Marshall (10.1016/j.jclinepi.2020.11.003_bib8) 2018; 9 Thomas (10.1016/j.jclinepi.2020.11.003_bib10) 2017; 91 Nevin (10.1016/j.jclinepi.2020.11.003_bib14) 2018; 15 Wilczynski (10.1016/j.jclinepi.2020.11.003_bib15) 2005; 5 Schulz (10.1016/j.jclinepi.2020.11.003_bib16) 2010; 8 Lefebvre (10.1016/j.jclinepi.2020.11.003_bib2) 2019 Steyerberg (10.1016/j.jclinepi.2020.11.003_bib19) 2001 Shojania (10.1016/j.jclinepi.2020.11.003_bib4) 2007; 147 10.1016/j.jclinepi.2020.11.003_bib13 Brier (10.1016/j.jclinepi.2020.11.003_bib20) 1950 Platt (10.1016/j.jclinepi.2020.11.003_bib18) 1999 Sain (10.1016/j.jclinepi.2020.11.003_bib17) 2006 Turner (10.1016/j.jclinepi.2020.11.003_bib22) 2012; 29 McKibbon (10.1016/j.jclinepi.2020.11.003_bib21) 2009; 26 Wallace (10.1016/j.jclinepi.2020.11.003_bib9) 2017; 24 |
| References_xml | – volume: 91 start-page: 31 year: 2017 end-page: 37 ident: bib10 article-title: Living systematic reviews: 2. Combining human and machine effort publication-title: J Clin Epidemiol – volume: 383 start-page: 101 year: 2014 end-page: 104 ident: bib5 article-title: Biomedical research: increasing value, reducing waste publication-title: Lancet – volume: 15 start-page: 4 year: 2018 end-page: 7 ident: bib14 article-title: Advancing the beneficial use of machine learning in health care and medicine: toward a community understanding publication-title: PLoS Med – start-page: 131 year: 2006 ident: bib17 article-title: The nature of statistical learning theory – volume: 9 start-page: 602 year: 2018 end-page: 614 ident: bib8 article-title: Machine learning for identifying randomized controlled trials: an evaluation and practitioner’s guide publication-title: Res Synth Methods – reference: . – volume: 147 start-page: 224 year: 2007 end-page: 233 ident: bib4 article-title: How quickly do systematic reviews go out of date? A survival analysis publication-title: Ann Intern Med – volume: 127 start-page: 142 year: 2020 end-page: 150 ident: bib12 article-title: Cochrane Centralised Search Service showed high sensitivity identifying randomized controlled trials: a retrospective analysis publication-title: J Clin Epidemiol – volume: 346 start-page: f139 year: 2013 ident: bib6 article-title: The automation of systematic reviews publication-title: BMJ – year: 2019 ident: bib11 article-title: About the CRS (Cochrane Register of Studies). Cochrane Community – year: 1950 ident: bib20 article-title: Verification of forecasts expressed in terms of probability publication-title: Mon Weather Rev – year: 1999 ident: bib18 article-title: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods publication-title: Adv Large Margin Classif – year: 2001 ident: bib19 article-title: Internal validation of predictive models: efficiency of some procedures for logistic regression analysis publication-title: J Clin Epidemiol – volume: 7 start-page: e1000326 year: 2010 ident: bib3 article-title: Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? publication-title: PLoS Med – reference: Noel-Storr A, Dooley G, Elliott J, Steele E, Shemilt I, Mavergames C, et al. An evaluation of Cochrane Crowd finds that crowdsourcing can help to address the challenge of information overload in evidence synthesis. – year: 2019 ident: bib1 article-title: Cochrane Library – volume: 8 start-page: 1 year: 2010 end-page: 9 ident: bib16 article-title: CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials publication-title: BMC Med – volume: 24 start-page: 1165 year: 2017 end-page: 1168 ident: bib9 article-title: Identifying reports of randomized controlled trials (RCTs) via a hybrid machine learning and crowdsourcing approach publication-title: J Am Med Inform Assoc – start-page: 67 year: 2019 end-page: 99 ident: bib2 article-title: Chapter 4: searching for and selecting studies publication-title: Cochrane Handbook for Systematic Reviews of Interventions – volume: 26 start-page: 187 year: 2009 end-page: 202 ident: bib21 article-title: Retrieving randomized controlled trials from MEDLINE: a comparison of 38 published search filters publication-title: Health Info Libr J – volume: 4 start-page: 5 year: 2015 ident: bib7 article-title: Using text mining for study identification in systematic reviews: a systematic review of current approaches publication-title: Syst Rev – volume: 5 start-page: 1 year: 2005 end-page: 15 ident: bib15 article-title: An overview of the design and methods for retrieving high-quality studies for clinical care publication-title: BMC Med Inform Decis Mak – volume: 29 start-page: 60 year: 2012 ident: bib22 article-title: Does use of the CONSORT statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review publication-title: Syst Rev – volume: 24 start-page: 1165 issue: 6 year: 2017 ident: 10.1016/j.jclinepi.2020.11.003_bib9 article-title: Identifying reports of randomized controlled trials (RCTs) via a hybrid machine learning and crowdsourcing approach publication-title: J Am Med Inform Assoc doi: 10.1093/jamia/ocx053 – year: 1950 ident: 10.1016/j.jclinepi.2020.11.003_bib20 article-title: Verification of forecasts expressed in terms of probability publication-title: Mon Weather Rev doi: 10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2 – volume: 91 start-page: 31 year: 2017 ident: 10.1016/j.jclinepi.2020.11.003_bib10 article-title: Living systematic reviews: 2. Combining human and machine effort publication-title: J Clin Epidemiol doi: 10.1016/j.jclinepi.2017.08.011 – year: 2001 ident: 10.1016/j.jclinepi.2020.11.003_bib19 article-title: Internal validation of predictive models: efficiency of some procedures for logistic regression analysis publication-title: J Clin Epidemiol doi: 10.1016/S0895-4356(01)00341-9 – volume: 147 start-page: 224 year: 2007 ident: 10.1016/j.jclinepi.2020.11.003_bib4 article-title: How quickly do systematic reviews go out of date? A survival analysis publication-title: Ann Intern Med doi: 10.7326/0003-4819-147-4-200708210-00179 – volume: 383 start-page: 101 year: 2014 ident: 10.1016/j.jclinepi.2020.11.003_bib5 article-title: Biomedical research: increasing value, reducing waste publication-title: Lancet doi: 10.1016/S0140-6736(13)62329-6 – volume: 29 start-page: 60 issue: 1 year: 2012 ident: 10.1016/j.jclinepi.2020.11.003_bib22 article-title: Does use of the CONSORT statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review publication-title: Syst Rev doi: 10.1186/2046-4053-1-60 – volume: 15 start-page: 4 issue: 11 year: 2018 ident: 10.1016/j.jclinepi.2020.11.003_bib14 article-title: Advancing the beneficial use of machine learning in health care and medicine: toward a community understanding publication-title: PLoS Med doi: 10.1371/journal.pmed.1002708 – volume: 26 start-page: 187 year: 2009 ident: 10.1016/j.jclinepi.2020.11.003_bib21 article-title: Retrieving randomized controlled trials from MEDLINE: a comparison of 38 published search filters publication-title: Health Info Libr J doi: 10.1111/j.1471-1842.2008.00827.x – volume: 4 start-page: 5 issue: 1 year: 2015 ident: 10.1016/j.jclinepi.2020.11.003_bib7 article-title: Using text mining for study identification in systematic reviews: a systematic review of current approaches publication-title: Syst Rev doi: 10.1186/2046-4053-4-5 – start-page: 67 year: 2019 ident: 10.1016/j.jclinepi.2020.11.003_bib2 article-title: Chapter 4: searching for and selecting studies – volume: 9 start-page: 602 year: 2018 ident: 10.1016/j.jclinepi.2020.11.003_bib8 article-title: Machine learning for identifying randomized controlled trials: an evaluation and practitioner’s guide publication-title: Res Synth Methods doi: 10.1002/jrsm.1287 – ident: 10.1016/j.jclinepi.2020.11.003_bib13 – year: 1999 ident: 10.1016/j.jclinepi.2020.11.003_bib18 article-title: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods publication-title: Adv Large Margin Classif – volume: 7 start-page: e1000326 issue: 9 year: 2010 ident: 10.1016/j.jclinepi.2020.11.003_bib3 article-title: Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? publication-title: PLoS Med doi: 10.1371/journal.pmed.1000326 – volume: 346 start-page: f139 issue: 1 year: 2013 ident: 10.1016/j.jclinepi.2020.11.003_bib6 article-title: The automation of systematic reviews publication-title: BMJ doi: 10.1136/bmj.f139 – volume: 8 start-page: 1 issue: 18 year: 2010 ident: 10.1016/j.jclinepi.2020.11.003_bib16 article-title: CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials publication-title: BMC Med – volume: 127 start-page: 142 year: 2020 ident: 10.1016/j.jclinepi.2020.11.003_bib12 article-title: Cochrane Centralised Search Service showed high sensitivity identifying randomized controlled trials: a retrospective analysis publication-title: J Clin Epidemiol doi: 10.1016/j.jclinepi.2020.08.008 – start-page: 131 year: 2006 ident: 10.1016/j.jclinepi.2020.11.003_bib17 – volume: 5 start-page: 1 issue: 20 year: 2005 ident: 10.1016/j.jclinepi.2020.11.003_bib15 article-title: An overview of the design and methods for retrieving high-quality studies for clinical care publication-title: BMC Med Inform Decis Mak |
| SSID | ssj0017075 |
| Score | 2.6730638 |
| Snippet | This study developed, calibrated, and evaluated a machine learning classifier designed to reduce study identification workload in Cochrane for producing... AbstractObjectivesThis study developed, calibrated, and evaluated of a machine learning classifier designed to reduce study identification workload in Cochrane... ObjectivesThis study developed, calibrated, and evaluated a machine learning classifier designed to reduce study identification workload in Cochrane for... • Systematic review processes need to become more efficient. • Machine learning is sufficiently mature for real-world use. • A machine learning classifier was... |
| SourceID | pubmedcentral proquest pubmed crossref elsevier |
| SourceType | Open Access Repository Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 140 |
| SubjectTerms | Algorithms Automation Bibliographic data bases Calibration Classification Classifiers Clinical trials Cochrane Library Confidence intervals Cost control Crowdsourcing Databases, Bibliographic - standards Databases, Bibliographic - statistics & numerical data Datasets Efficiency Epidemiology Evaluation Humans Information retrieval Information Storage and Retrieval - methods Information Storage and Retrieval - standards Information Storage and Retrieval - statistics & numerical data Internal Medicine Learning algorithms Machine Learning Methods/methodology Original Production methods Randomized controlled trials Randomized Controlled Trials as Topic - classification Randomized Controlled Trials as Topic - methods Randomized Controlled Trials as Topic - standards Randomized Controlled Trials as Topic - statistics & numerical data Recall Searching Study classifiers Systematic review Systematic reviews Systematic Reviews as Topic - methods Systematic Reviews as Topic - standards Validity Workflow Workload Workload - statistics & numerical data Workloads |
| Title | Machine learning reduced workload with minimal risk of missing studies: development and evaluation of a randomized controlled trial classifier for Cochrane Reviews |
| URI | https://www.clinicalkey.com/#!/content/1-s2.0-S0895435620311720 https://www.clinicalkey.es/playcontent/1-s2.0-S0895435620311720 https://dx.doi.org/10.1016/j.jclinepi.2020.11.003 https://www.ncbi.nlm.nih.gov/pubmed/33171275 https://www.proquest.com/docview/2529826511 https://www.proquest.com/docview/2459623441 https://pubmed.ncbi.nlm.nih.gov/PMC8168828 |
| Volume | 133 |
| WOSCitedRecordID | wos000653651200016&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: ScienceDirect database customDbUrl: eissn: 1878-5921 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0017075 issn: 0895-4356 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier – providerCode: PRVPQU databaseName: Health & Medical Collection customDbUrl: eissn: 1878-5921 dateEnd: 20251007 omitProxy: false ssIdentifier: ssj0017075 issn: 0895-4356 databaseCode: 7X7 dateStart: 20030101 isFulltext: true titleUrlDefault: https://search.proquest.com/healthcomplete providerName: ProQuest – providerCode: PRVPQU databaseName: Healthcare Administration Database customDbUrl: eissn: 1878-5921 dateEnd: 20251007 omitProxy: false ssIdentifier: ssj0017075 issn: 0895-4356 databaseCode: M0T dateStart: 20030101 isFulltext: true titleUrlDefault: https://search.proquest.com/healthmanagement providerName: ProQuest – providerCode: PRVPQU databaseName: Nursing & Allied Health Database customDbUrl: eissn: 1878-5921 dateEnd: 20251007 omitProxy: false ssIdentifier: ssj0017075 issn: 0895-4356 databaseCode: 7RV dateStart: 20030101 isFulltext: true titleUrlDefault: https://search.proquest.com/nahs providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 1878-5921 dateEnd: 20251007 omitProxy: false ssIdentifier: ssj0017075 issn: 0895-4356 databaseCode: BENPR dateStart: 20030101 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: Public Health Database customDbUrl: eissn: 1878-5921 dateEnd: 20251007 omitProxy: false ssIdentifier: ssj0017075 issn: 0895-4356 databaseCode: 8C1 dateStart: 20030101 isFulltext: true titleUrlDefault: https://search.proquest.com/publichealth providerName: ProQuest – providerCode: PRVPQU databaseName: Research Library customDbUrl: eissn: 1878-5921 dateEnd: 20251007 omitProxy: false ssIdentifier: ssj0017075 issn: 0895-4356 databaseCode: M2O dateStart: 20030101 isFulltext: true titleUrlDefault: https://search.proquest.com/pqrl providerName: ProQuest |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1Lb9QwELagRYhLedNAqYzENW3iPBxzQbBqxWWXqhS0Nyt-pJvVNlk2Ww78Hf4oM4mTbakECC5Woth5eTzz2fN5hpDXsTBJmGrrmzSECYoNEl-xNPeZEcLwOOdprNtkE3wyyaZTceIW3BpHq-x1YquoTa1xjfyQJUwAFAZ88Hb51cesUehddSk0bpPtEG03yDM__TJ4EXgXaDfIROIDLEiv7BCeH8xx66FdljBHZKg5DoI-cdZN43QTfP7KobxilI7v_-_nPCA7Do7Sd538PCS3bPWI3B07h_tj8mPcki0tddklzukKY71aQ5HQtahzOCjXM4oRSi7gRshUp3UB5w0uQtCmoym-oWZDTqJ5ZegmyjhWzymYTFNflN_hzo49v4DDNqcI1YjwywL-EQWITUe1nkF1Szu_RvOEfD4-Oht98F1eB1-ngVj7RgOsL5jSGCU9jrgqhAVUFgQ61rHiRsUqD5RJE27yvFCAuLgSxoIiVkkqrImekq2qruwuoZkVhRLQ1QW6kLMkM1FqrI1gzg9Y0jKPJH2HSu2CnmPujYXs2W1z2QuCREGAGRGGS_XI4dBu2YX9-GML3suL7De1ghqWYJn-raVtnDZpZCgbJgP5CQUZ5ZiBJgbgGXhEDC0dYOqA0F89da8XTDk8aCOVHnk1XAaRQS8SdGx9CXViTNgUQYd55Fk3BoZfFAEaxYwB8EnXRsdQAYOZX79SlbM2qDnmf4HZ__Pfv9YLco8hqahlnO6RrfXq0r4kd_S3ddms9tvRj-WUt2UGZTYK98n2-6PJySmcjYMzLNnHn_Szbyk |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Zb9NAEB6VgoAX7iNQYJHg0a2zPtaLhBAqVK3aREgUqW9b72HiKI1DnILg7_DOb2TGV1oqAULqA2-OvLt2duf4xvvtDMCzUNqoHxvn2biPAYrzI0_zOPW4ldKKMBVxaKpiE2I4TA4O5LsV-NGehSFaZWsTK0NtC0PfyDd4xCVCYcQHr2afPKoaRburbQmNWix23dcvGLKVL3fe4Po-53zr7f7mttdUFfBM7MuFZw2CyoxrQzm6w0DoTDrEBL5vQhNqYXWoU1_bOBI2TTON_l5oaR2aAR3F0tkAx70AF9GOC6KQiYMuwOuLOrGvn8jIQxgSnziRPF4f01FHN8sxJuVkqdb9tlDXWWd4Fuz-ytk84QS3rv9v03cDrjVwm72u9eMmrLjpLbg8aAgFt-H7oCKTOtZUz_jI5pTL1llGhLVJkeJFvhgxysByhAMRE58VGf4u6SMLK2sa5gtml-Qrlk4tW2ZRp-YpQ0hgi6P8G47cnA6Y4GVVM4UZimDyDNeEYQjBNgszwuaO1fs25R34cC5TdBdWp8XU3QeWOJlpiaKV0RZ5EiU2iK1zQSwJKzveg6gVIGWapO5UW2SiWvbeWLWCp0jwMOKjdLA92Oj6zeq0Jn_sIVr5VO2hXXQzCj3vv_V0ZWMtS9VXJVe-ek-KQ3rD0dMgsPZ7ILueDSCsgd5fPXWtVQTVPWipBT142t1GkaFdMlzY4hjbhFSQKsAF68G9Wue6KQoQbVNFBPxLp7Sxa0DJ2k_fmeajKmk71bdJePLg96_1BK5s7w_21N7OcPchXOVEoKrYtWuwupgfu0dwyXxe5OX8cWV5GByet67-BIBxyWQ |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3bbtNAEB2VgipeuBcCBRYJHt046-siIYRaIqqSqBJF6tvivZg4Su0QpyD4Hf6Cr2PGt7RUAoTUB94ceXft7M7ljPfsDMBTX5hgEGrrmHCAAYp1A0fxMHG4EcJEfhKFvq6KTUTjcXx0JA7W4Ed7FoZola1NrAy1KTR9I-_zgAuEwogP-mlDizjYHb6cf3KoghTttLblNGoR2bdfv2D4Vr7Y28W1fsb58PXhzhunqTDg6NAVS8doBJgpV5rydftepFJhER-4rva1ryKjfJW4yoRBZJIkVej7IyWMRZOgglBY4-G4l-By5Aec6GQj97DbwYjqJL9uLAIHIUl46nTydHtKxx7tPMP4lJPV2nbbol3nHeN54Psrf_OUQxxe_5-n8gZca2A4e1XrzU1Ys_kt2Bg1RIPb8H1UkUwta6pqfGQLynFrDSMi26xI8CJbThhlZjnGgYihz4oUf5f08YWVNT3zOTMrUhZLcsNW2dWpecIQKpjiOPuGIzenBmZ4WdVSYZoimyzF9WEYWrCdQk-wuWX1fk55B95fyBRtwnpe5PYesNiKVAkUs5S2zuMgNl5orPVCQRja8h4ErTBJ3SR7p5ojM9my-qayFUJJQoiRIKWJ7UG_6zev0538sUfUyqpsD_Oi-5Hokf-tpy0bK1rKgSy5dOU7UiLSIY4eCAG32wPR9WyAYg0A_-qpW61SyO5BK43owZPuNooM7Z7hwhYn2ManQlUeLlgP7tb6102RhyicKiXgXzqjmV0DSuJ-9k6eTapk7lT3Jubx_d-_1mPYQBWVb_fG-w_gKideVUW63YL15eLEPoQr-vMyKxePKiPE4MNFq-pPy2vSNA |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Machine+learning+reduced+workload+with+minimal+risk+of+missing+studies%3A+development+and+evaluation+of+a+randomized+controlled+trial+classifier+for+Cochrane+Reviews&rft.jtitle=Journal+of+clinical+epidemiology&rft.au=Thomas%2C+James&rft.au=McDonald%2C+Steve&rft.au=Noel-Storr%2C+Anna&rft.au=Shemilt%2C+Ian&rft.date=2021-05-01&rft.issn=1878-5921&rft.eissn=1878-5921&rft.volume=133&rft.spage=140&rft_id=info:doi/10.1016%2Fj.jclinepi.2020.11.003&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0895-4356&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0895-4356&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0895-4356&client=summon |