Area under the ROC Curve has the most consistent evaluation for binary classification
The proper use of model evaluation metrics is important for model evaluation and model selection in binary classification tasks. This study investigates how consistent different metrics are at evaluating models across data of different prevalence while the relationships between different variables a...
Uložené v:
| Vydané v: | PloS one Ročník 19; číslo 12; s. e0316019 |
|---|---|
| Hlavný autor: | |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
United States
Public Library of Science
23.12.2024
Public Library of Science (PLoS) |
| Predmet: | |
| ISSN: | 1932-6203, 1932-6203 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | The proper use of model evaluation metrics is important for model evaluation and model selection in binary classification tasks. This study investigates how consistent different metrics are at evaluating models across data of different prevalence while the relationships between different variables and the sample size are kept constant. Analyzing 156 data scenarios, 18 model evaluation metrics and five commonly used machine learning models as well as a naive random guess model, I find that evaluation metrics that are less influenced by prevalence offer more consistent evaluation of individual models and more consistent ranking of a set of models. In particular, Area Under the ROC Curve (AUC) which takes all decision thresholds into account when evaluating models has the smallest variance in evaluating individual models and smallest variance in ranking of a set of models. A close threshold analysis using all possible thresholds for all metrics further supports the hypothesis that considering all decision thresholds helps reduce the variance in model evaluation with respect to prevalence change in data. The results have significant implications for model evaluation and model selection in binary classification tasks. |
|---|---|
| AbstractList | The proper use of model evaluation metrics is important for model evaluation and model selection in binary classification tasks. This study investigates how consistent different metrics are at evaluating models across data of different prevalence while the relationships between different variables and the sample size are kept constant. Analyzing 156 data scenarios, 18 model evaluation metrics and five commonly used machine learning models as well as a naive random guess model, I find that evaluation metrics that are less influenced by prevalence offer more consistent evaluation of individual models and more consistent ranking of a set of models. In particular, Area Under the ROC Curve (AUC) which takes all decision thresholds into account when evaluating models has the smallest variance in evaluating individual models and smallest variance in ranking of a set of models. A close threshold analysis using all possible thresholds for all metrics further supports the hypothesis that considering all decision thresholds helps reduce the variance in model evaluation with respect to prevalence change in data. The results have significant implications for model evaluation and model selection in binary classification tasks. The proper use of model evaluation metrics is important for model evaluation and model selection in binary classification tasks. This study investigates how consistent different metrics are at evaluating models across data of different prevalence while the relationships between different variables and the sample size are kept constant. Analyzing 156 data scenarios, 18 model evaluation metrics and five commonly used machine learning models as well as a naive random guess model, I find that evaluation metrics that are less influenced by prevalence offer more consistent evaluation of individual models and more consistent ranking of a set of models. In particular, Area Under the ROC Curve (AUC) which takes all decision thresholds into account when evaluating models has the smallest variance in evaluating individual models and smallest variance in ranking of a set of models. A close threshold analysis using all possible thresholds for all metrics further supports the hypothesis that considering all decision thresholds helps reduce the variance in model evaluation with respect to prevalence change in data. The results have significant implications for model evaluation and model selection in binary classification tasks.The proper use of model evaluation metrics is important for model evaluation and model selection in binary classification tasks. This study investigates how consistent different metrics are at evaluating models across data of different prevalence while the relationships between different variables and the sample size are kept constant. Analyzing 156 data scenarios, 18 model evaluation metrics and five commonly used machine learning models as well as a naive random guess model, I find that evaluation metrics that are less influenced by prevalence offer more consistent evaluation of individual models and more consistent ranking of a set of models. In particular, Area Under the ROC Curve (AUC) which takes all decision thresholds into account when evaluating models has the smallest variance in evaluating individual models and smallest variance in ranking of a set of models. A close threshold analysis using all possible thresholds for all metrics further supports the hypothesis that considering all decision thresholds helps reduce the variance in model evaluation with respect to prevalence change in data. The results have significant implications for model evaluation and model selection in binary classification tasks. |
| Audience | Academic |
| Author | Li, Jing |
| AuthorAffiliation | Old Dominion University, UNITED STATES OF AMERICA Department of Political Science, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America |
| AuthorAffiliation_xml | – name: Old Dominion University, UNITED STATES OF AMERICA – name: Department of Political Science, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America |
| Author_xml | – sequence: 1 givenname: Jing orcidid: 0000-0001-9332-399X surname: Li fullname: Li, Jing |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/39715186$$D View this record in MEDLINE/PubMed |
| BookMark | eNqNk11r2zAUhs3oWNts_2BshsHYLpJJlqyP3YwQ9hEoBLp1t0KW5ERFsTJJDtu_n_LREpcyhi9sjp73lfT6nMvirPOdKYqXEEwgovDDre9DJ91kk8sTgCABkD8pLiBH1ZhUAJ2dfJ8XlzHeAlAjRsiz4hxxCmvIyEVxMw1Gln2nTSjTypTXi1k568PWlCsZ95W1j6lUvos2JtOl0myl62WyvitbH8rGdjL8KZWTMdrWqv3K8-JpK100L47vUXHz5fOP2bfx1eLrfDa9GivCURozhgGWtKGamEoZ3UDDcMNaVpsaaAJqTtqK16TFtKbUVJpiwlCFKW9qZrhGo-L1wXfjfBTHRKJAEDPKEaR1JuYHQnt5KzbBrvNphZdW7As-LIUMySpnBK60RoATQBuOW8KZVhWWjdKY01rXNHt9Ou7WN2ujVU4jSDcwHa50diWWfisgJIQAhLLDu6ND8L96E5NY26iMc7Izvj8cnCFeZXhUvHmAPn69I7WU-Qa2a33eWO1MxZRVEHDA-M5r8giVH23WNv9a09pcHwjeDwSZSeZ3Wso-RjH_fv3_7OLnkH17wq6MdGkVvet3PROH4KvTqO8zvmvcDOADoIKPMZj2HoFA7ObjLi6xmw9xnI8s-_hApmzat2xOxLp_i_8Cv8gUTw |
| CitedBy_id | crossref_primary_10_59395_ijadis_v6i2_1405 crossref_primary_10_1016_j_gr_2025_08_003 crossref_primary_10_1007_s10666_025_10061_x crossref_primary_10_1007_s11356_025_36621_y crossref_primary_10_1038_s41598_025_90089_x crossref_primary_10_1021_acs_jcim_5c01873 crossref_primary_10_3389_fonc_2025_1525414 crossref_primary_10_1016_j_msksp_2025_103321 crossref_primary_10_3390_sym17081261 crossref_primary_10_1038_s41598_025_10907_0 crossref_primary_10_3389_fmed_2025_1537139 crossref_primary_10_3390_buildings15172988 crossref_primary_10_3390_app15168955 crossref_primary_10_1016_j_amjmed_2025_05_047 crossref_primary_10_1155_jom_5357997 crossref_primary_10_1007_s42979_025_04307_4 |
| Cites_doi | 10.1038/s41598-022-09954-8 10.1186/s12880-015-0068-x 10.1136/emermed-2017-206735 10.1145/3310986.3311023 10.1093/bib/bbr008 10.1186/s12864-019-6413-7 10.1177/0962280212452199 10.1017/pan.2018.55 10.1177/0272989X8900900307 10.1016/S0031-3203(96)00142-2 10.1007/s11634-017-0295-9 10.1111/j.1466-8238.2007.00358.x 10.1093/clinchem/39.4.561 10.1007/s00330-014-3487-0 10.1007/s00357-019-09345-1 10.1177/0969141313517497 10.1016/j.eswa.2020.113391 10.1161/CIRCULATIONAHA.106.672402 10.1148/radiology.143.1.7063747 10.1038/nmeth.3945 10.1145/3383219.3383232 10.1016/j.patrec.2020.03.030 10.1109/ICIST.2012.6221710 10.1093/jnci/95.7.511 10.1186/s13040-023-00322-4 10.1016/j.jclinepi.2015.02.010 10.1016/B978-0-12-405888-0.00005-2 10.1109/ACCESS.2021.3068614 10.1017/S026988892100014X 10.3389/fpubh.2015.00057 10.1007/s10489-021-03041-7 10.1007/s10664-020-09861-4 10.1016/j.jtcvs.2021.07.041 10.1177/09622802211060515 10.1186/s13040-021-00244-z 10.1126/sciadv.aao5580 10.1186/s12911-019-1014-6 10.1111/2041-210X.13826 10.1186/1471-2105-9-410 10.1080/13658816.2013.862623 10.1093/bioinformatics/btq140 10.1109/ACCESS.2023.3278996 10.1159/000074404 10.5220/0003783303100313 10.1016/S0001-2998(78)80014-2 10.1016/j.patcog.2019.02.023 10.1145/347090.347126 10.12746/swrccc.v5i19.391 10.1007/978-3-030-62365-4_10 |
| ContentType | Journal Article |
| Copyright | Copyright: © 2024 Jing Li. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. COPYRIGHT 2024 Public Library of Science 2024 Jing Li. This is an open access article distributed under the terms of the Creative Commons Attribution License: http://creativecommons.org/licenses/by/4.0/ (the “License”), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. 2024 Jing Li 2024 Jing Li 2024 Jing Li. This is an open access article distributed under the terms of the Creative Commons Attribution License: http://creativecommons.org/licenses/by/4.0/ (the “License”), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| Copyright_xml | – notice: Copyright: © 2024 Jing Li. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. – notice: COPYRIGHT 2024 Public Library of Science – notice: 2024 Jing Li. This is an open access article distributed under the terms of the Creative Commons Attribution License: http://creativecommons.org/licenses/by/4.0/ (the “License”), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. – notice: 2024 Jing Li 2024 Jing Li – notice: 2024 Jing Li. This is an open access article distributed under the terms of the Creative Commons Attribution License: http://creativecommons.org/licenses/by/4.0/ (the “License”), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| DBID | AAYXX CITATION CGR CUY CVF ECM EIF NPM IOV ISR 3V. 7QG 7QL 7QO 7RV 7SN 7SS 7T5 7TG 7TM 7U9 7X2 7X7 7XB 88E 8AO 8C1 8FD 8FE 8FG 8FH 8FI 8FJ 8FK ABJCF ABUWG AEUYN AFKRA ARAPS ATCPS AZQEC BBNVY BENPR BGLVJ BHPHI C1K CCPQU D1I DWQXO FR3 FYUFA GHDGH GNUQQ H94 HCIFZ K9. KB. KB0 KL. L6V LK8 M0K M0S M1P M7N M7P M7S NAPCQ P5Z P62 P64 PATMY PDBOC PHGZM PHGZT PIMPY PJZUB PKEHL PPXIY PQEST PQGLB PQQKQ PQUKI PRINS PTHSS PYCSY RC3 7X8 5PM DOA |
| DOI | 10.1371/journal.pone.0316019 |
| DatabaseName | CrossRef Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed Gale In Context: Opposing Viewpoints Gale In Context: Science ProQuest Central (Corporate) Animal Behavior Abstracts Bacteriology Abstracts (Microbiology B) Biotechnology Research Abstracts Nursing & Allied Health Database Ecology Abstracts Entomology Abstracts (Full archive) Immunology Abstracts Meteorological & Geoastrophysical Abstracts Nucleic Acids Abstracts Virology and AIDS Abstracts Agricultural Science Collection Health & Medical Collection ProQuest Central (purchase pre-March 2016) Medical Database (Alumni Edition) ProQuest Pharma Collection Public Health Database Technology Research Database ProQuest SciTech Collection ProQuest Technology Collection ProQuest Natural Science Collection ProQuest Hospital Collection Hospital Premium Collection (Alumni Edition) ProQuest Central (Alumni) (purchase pre-March 2016) Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest One Sustainability ProQuest Central UK/Ireland ProQuest Advanced Technologies & Aerospace Collection Agricultural & Environmental Science Collection ProQuest Central Essentials Local Electronic Collection Information Biological Science Collection ProQuest Central Technology Collection Natural Science Collection Environmental Sciences and Pollution Management ProQuest One Community College ProQuest Materials Science Collection ProQuest Central Korea Engineering Research Database Health Research Premium Collection Health Research Premium Collection (Alumni) ProQuest Central Student AIDS and Cancer Research Abstracts SciTech Premium Collection ProQuest Health & Medical Complete (Alumni) ProQuest Materials Science Database (NC LIVE) Nursing & Allied Health Database (Alumni Edition) Meteorological & Geoastrophysical Abstracts - Academic ProQuest Engineering Collection Biological Sciences Agricultural Science Database ProQuest Health & Medical Collection PML(ProQuest Medical Library) Algology Mycology and Protozoology Abstracts (Microbiology C) Biological Science Database Engineering Database Nursing & Allied Health Premium Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection Biotechnology and BioEngineering Abstracts Environmental Science Database Materials Science Collection ProQuest Central Premium ProQuest One Academic (New) Publicly Available Content Database ProQuest Health & Medical Research Collection ProQuest One Academic Middle East (New) ProQuest One Health & Nursing ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection Environmental Science Collection Genetics Abstracts MEDLINE - Academic PubMed Central (Full Participant titles) Directory of Open Access Journals |
| DatabaseTitle | CrossRef MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) Agricultural Science Database Publicly Available Content Database ProQuest Central Student ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials Nucleic Acids Abstracts SciTech Premium Collection ProQuest Central China Environmental Sciences and Pollution Management ProQuest One Applied & Life Sciences ProQuest One Sustainability Health Research Premium Collection Meteorological & Geoastrophysical Abstracts Natural Science Collection Health & Medical Research Collection Biological Science Collection ProQuest Central (New) ProQuest Medical Library (Alumni) Engineering Collection Advanced Technologies & Aerospace Collection Engineering Database Virology and AIDS Abstracts ProQuest Biological Science Collection ProQuest One Academic Eastern Edition Agricultural Science Collection ProQuest Hospital Collection ProQuest Technology Collection Health Research Premium Collection (Alumni) Biological Science Database Ecology Abstracts ProQuest Hospital Collection (Alumni) Biotechnology and BioEngineering Abstracts Environmental Science Collection Entomology Abstracts Nursing & Allied Health Premium ProQuest Health & Medical Complete ProQuest One Academic UKI Edition Environmental Science Database ProQuest Nursing & Allied Health Source (Alumni) Engineering Research Database ProQuest One Academic Meteorological & Geoastrophysical Abstracts - Academic ProQuest One Academic (New) Technology Collection Technology Research Database ProQuest One Academic Middle East (New) Materials Science Collection ProQuest Health & Medical Complete (Alumni) ProQuest Central (Alumni Edition) ProQuest One Community College ProQuest One Health & Nursing ProQuest Natural Science Collection ProQuest Pharma Collection ProQuest Central ProQuest Health & Medical Research Collection Genetics Abstracts ProQuest Engineering Collection Biotechnology Research Abstracts Health and Medicine Complete (Alumni Edition) ProQuest Central Korea Bacteriology Abstracts (Microbiology B) Algology Mycology and Protozoology Abstracts (Microbiology C) Agricultural & Environmental Science Collection AIDS and Cancer Research Abstracts Materials Science Database ProQuest Materials Science Collection ProQuest Public Health ProQuest Nursing & Allied Health Source ProQuest SciTech Collection Advanced Technologies & Aerospace Database ProQuest Medical Library Animal Behavior Abstracts Materials Science & Engineering Collection Immunology Abstracts ProQuest Central (Alumni) MEDLINE - Academic |
| DatabaseTitleList | Agricultural Science Database MEDLINE MEDLINE - Academic CrossRef |
| Database_xml | – sequence: 1 dbid: DOA name: Directory of Open Access Journals - DOAJ (NTUSG) url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 3 dbid: PIMPY name: Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Sciences (General) |
| DocumentTitleAlternate | Consistent model evaluation for binary classification |
| EISSN | 1932-6203 |
| ExternalDocumentID | 3148793175 oai_doaj_org_article_42dd309607b94f698dc24abcd4975d57 PMC11666033 A821090890 39715186 10_1371_journal_pone_0316019 |
| Genre | Journal Article |
| GeographicLocations | United States |
| GeographicLocations_xml | – name: United States |
| GroupedDBID | --- 123 29O 2WC 53G 5VS 7RV 7X2 7X7 7XC 88E 8AO 8C1 8CJ 8FE 8FG 8FH 8FI 8FJ A8Z AAFWJ AAUCC AAWOE AAYXX ABDBF ABIVO ABJCF ABUWG ACCTH ACGFO ACIHN ACIWK ACPRK ACUHS ADBBV AEAQA AENEX AEUYN AFFHD AFKRA AFPKN AFRAH AHMBA ALMA_UNASSIGNED_HOLDINGS AOIJS APEBS ARAPS ATCPS BAIFH BAWUL BBNVY BBTPI BCNDV BENPR BGLVJ BHPHI BKEYQ BPHCQ BVXVI BWKFM CCPQU CITATION CS3 D1I D1J D1K DIK DU5 E3Z EAP EAS EBD EMOBN ESX EX3 F5P FPL FYUFA GROUPED_DOAJ GX1 HCIFZ HH5 HMCUK HYE IAO IEA IGS IHR IHW INH INR IOV IPY ISE ISR ITC K6- KB. KQ8 L6V LK5 LK8 M0K M1P M48 M7P M7R M7S M~E NAPCQ O5R O5S OK1 OVT P2P P62 PATMY PDBOC PHGZM PHGZT PIMPY PJZUB PPXIY PQGLB PQQKQ PROAC PSQYO PTHSS PV9 PYCSY RNS RPM RZL SV3 TR2 UKHRP WOQ WOW ~02 ~KM ADRAZ ALIPV BBORY CGR CUY CVF ECM EIF IPNFZ NPM RIG 3V. 7QG 7QL 7QO 7SN 7SS 7T5 7TG 7TM 7U9 7XB 8FD 8FK AZQEC C1K DWQXO ESTFP FR3 GNUQQ H94 K9. KL. M7N P64 PKEHL PQEST PQUKI PRINS RC3 7X8 PUEGO 5PM |
| ID | FETCH-LOGICAL-c693t-88404a7b7d6e2cedb1e84b8f85e50d60596f2956f47577e2d746832479b58e9d3 |
| IEDL.DBID | DOA |
| ISICitedReferencesCount | 15 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001400361000006&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1932-6203 |
| IngestDate | Wed Aug 13 01:18:59 EDT 2025 Fri Oct 03 12:41:21 EDT 2025 Tue Nov 04 02:03:41 EST 2025 Mon Sep 08 15:54:38 EDT 2025 Tue Oct 07 09:18:34 EDT 2025 Sat Nov 29 13:49:23 EST 2025 Sat Nov 29 10:34:16 EST 2025 Wed Nov 26 10:45:28 EST 2025 Wed Nov 26 10:45:12 EST 2025 Thu May 22 21:23:22 EDT 2025 Mon Mar 31 02:18:58 EDT 2025 Tue Nov 18 21:53:50 EST 2025 Sat Nov 29 03:55:39 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 12 |
| Language | English |
| License | Copyright: © 2024 Jing Li. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Creative Commons Attribution License |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c693t-88404a7b7d6e2cedb1e84b8f85e50d60596f2956f47577e2d746832479b58e9d3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 content type line 23 Competing Interests: The authors have declared that no competing interests exist. |
| ORCID | 0000-0001-9332-399X |
| OpenAccessLink | https://doaj.org/article/42dd309607b94f698dc24abcd4975d57 |
| PMID | 39715186 |
| PQID | 3148793175 |
| PQPubID | 1436336 |
| PageCount | e0316019 |
| ParticipantIDs | plos_journals_3148793175 doaj_primary_oai_doaj_org_article_42dd309607b94f698dc24abcd4975d57 pubmedcentral_primary_oai_pubmedcentral_nih_gov_11666033 proquest_miscellaneous_3148839260 proquest_journals_3148793175 gale_infotracmisc_A821090890 gale_infotracacademiconefile_A821090890 gale_incontextgauss_ISR_A821090890 gale_incontextgauss_IOV_A821090890 gale_healthsolutions_A821090890 pubmed_primary_39715186 crossref_primary_10_1371_journal_pone_0316019 crossref_citationtrail_10_1371_journal_pone_0316019 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-12-23 |
| PublicationDateYYYYMMDD | 2024-12-23 |
| PublicationDate_xml | – month: 12 year: 2024 text: 2024-12-23 day: 23 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States – name: San Francisco – name: San Francisco, CA USA |
| PublicationTitle | PloS one |
| PublicationTitleAlternate | PLoS One |
| PublicationYear | 2024 |
| Publisher | Public Library of Science Public Library of Science (PLoS) |
| Publisher_xml | – name: Public Library of Science – name: Public Library of Science (PLoS) |
| References | S Morasca (pone.0316019.ref020) 2020; 25 L Lavazza (pone.0316019.ref033) 2022; 31 AM Carrington (pone.0316019.ref041) 2020; 20 MH Zweig (pone.0316019.ref015) 1993; 39 A Jiménez-Valverde (pone.0316019.ref036) 2022; 13 S Parodi (pone.0316019.ref021) 2003; 101 D Chicco (pone.0316019.ref007) 2020; 21 Q Zhu (pone.0316019.ref006) 2020; 136 J Kruschke (pone.0316019.ref048) 2015 JA Hanley (pone.0316019.ref013) 1982; 143 S Yang (pone.0316019.ref023) 2017; 5 AR Redondo (pone.0316019.ref045) 2020 pone.0316019.ref043 pone.0316019.ref001 SG Baker (pone.0316019.ref018) 2003; 95 pone.0316019.ref002 D Chicco (pone.0316019.ref008) 2021; 14 pone.0316019.ref046 DK McClish (pone.0316019.ref032) 1989; 9 JM Vivo (pone.0316019.ref040) 2018; 12 J Lever (pone.0316019.ref003) 2016; 13 A Luque (pone.0316019.ref004) 2019; 91 SA Hicks (pone.0316019.ref010) 2022; 12 S Parodi (pone.0316019.ref037) 2008; 9 SJ Swamidass (pone.0316019.ref039) 2010; 26 J Yao (pone.0316019.ref047) 2020 S Parodi (pone.0316019.ref038) 2016; 25 CE Metz (pone.0316019.ref014) 1978; 8 IM De Diego (pone.0316019.ref009) 2022; 52 J Muschelli (pone.0316019.ref027) 2020; 37 S Halligan (pone.0316019.ref028) 2015; 25 N Wald (pone.0316019.ref029) 2014; 21 RG Pontius (pone.0316019.ref042) 2014; 28 L Lavazza (pone.0316019.ref011) 2023; 11 AP Bradley (pone.0316019.ref017) 1997; 30 ZH Hoo (pone.0316019.ref016) 2017; 34 ME Pérez-Pons (pone.0316019.ref019) 2022; 37 AS Jadhav (pone.0316019.ref005) 2020; 152 NR Cook (pone.0316019.ref025) 2007; 115 D Chicco (pone.0316019.ref034) 2021; 9 B Ozenne (pone.0316019.ref044) 2015; 68 K Bansak (pone.0316019.ref050) 2019; 27 AA Taha (pone.0316019.ref022) 2015; 15 pone.0316019.ref026 J Dressel (pone.0316019.ref049) 2018; 4 D Chicco (pone.0316019.ref012) 2023; 16 Y Yuan (pone.0316019.ref035) 2015; 3 F Movahedi (pone.0316019.ref030) 2023; 165 JM Lobo (pone.0316019.ref031) 2008; 17 D Berrar (pone.0316019.ref024) 2012; 13 |
| References_xml | – volume: 12 start-page: 5979 issue: 1 year: 2022 ident: pone.0316019.ref010 article-title: On evaluation metrics for medical applications of artificial intelligence publication-title: Scientific Reports doi: 10.1038/s41598-022-09954-8 – volume: 15 start-page: 29 issue: 1 year: 2015 ident: pone.0316019.ref022 article-title: Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool publication-title: BMC Medical Imaging doi: 10.1186/s12880-015-0068-x – volume: 34 start-page: 357 issue: 6 year: 2017 ident: pone.0316019.ref016 article-title: What is an ROC curve? publication-title: Emergency Medicine Journal doi: 10.1136/emermed-2017-206735 – ident: pone.0316019.ref046 doi: 10.1145/3310986.3311023 – volume: 13 start-page: 83 issue: 1 year: 2012 ident: pone.0316019.ref024 article-title: Caveats and pitfalls of ROC analysis in clinical microarray research (and how to avoid them) publication-title: Briefings in Bioinformatics doi: 10.1093/bib/bbr008 – volume: 21 start-page: 6 issue: 1 year: 2020 ident: pone.0316019.ref007 article-title: The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation publication-title: BMC Genomics doi: 10.1186/s12864-019-6413-7 – volume: 25 start-page: 294 issue: 1 year: 2016 ident: pone.0316019.ref038 article-title: Restricted ROC curves are useful tools to evaluate the performance of tumour markers publication-title: Statistical Methods in Medical Research doi: 10.1177/0962280212452199 – volume: 27 start-page: 370 issue: 3 year: 2019 ident: pone.0316019.ref050 article-title: Can nonexperts really emulate statistical learning methods? A comment on “The accuracy, fairness, and limits of predicting recidivism” publication-title: Polit Anal doi: 10.1017/pan.2018.55 – volume: 9 start-page: 190 issue: 3 year: 1989 ident: pone.0316019.ref032 article-title: Analyzing a Portion of the ROC Curve publication-title: Medical Decision Making doi: 10.1177/0272989X8900900307 – volume: 30 start-page: 1145 issue: 7 year: 1997 ident: pone.0316019.ref017 article-title: The use of the area under the ROC curve in the evaluation of machine learning algorithms publication-title: Pattern Recognition doi: 10.1016/S0031-3203(96)00142-2 – volume: 12 start-page: 683 issue: 3 year: 2018 ident: pone.0316019.ref040 article-title: Rethinking an ROC partial area index for evaluating the classification performance at a high specificity range publication-title: Advances in Data Analysis and Classification doi: 10.1007/s11634-017-0295-9 – volume: 17 start-page: 145 issue: 2 year: 2008 ident: pone.0316019.ref031 article-title: AUC: a misleading measure of the performance of predictive distribution models publication-title: Global Ecology and Biogeography doi: 10.1111/j.1466-8238.2007.00358.x – volume: 39 start-page: 561 issue: 4 year: 1993 ident: pone.0316019.ref015 article-title: Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine publication-title: Clinical chemistry doi: 10.1093/clinchem/39.4.561 – volume: 25 start-page: 932 issue: 4 year: 2015 ident: pone.0316019.ref028 article-title: Disadvantages of using the area under the receiver operating characteristic curve to assess imaging tests: A discussion and proposal for an alternative approach publication-title: European Radiology doi: 10.1007/s00330-014-3487-0 – volume: 37 start-page: 696 issue: 3 year: 2020 ident: pone.0316019.ref027 article-title: ROC and AUC with a Binary Predictor: a Potentially Misleading Metric publication-title: Journal of Classification doi: 10.1007/s00357-019-09345-1 – volume: 21 start-page: 51 issue: 1 year: 2014 ident: pone.0316019.ref029 article-title: Is the area under an ROC curve a valid measure of the performance of a screening or diagnostic test? publication-title: Journal of Medical Screening doi: 10.1177/0969141313517497 – volume: 152 start-page: 113391 year: 2020 ident: pone.0316019.ref005 article-title: A novel weighted TPR-TNR measure to assess performance of the classifiers publication-title: Expert Systems with Applications doi: 10.1016/j.eswa.2020.113391 – volume: 115 start-page: 928 issue: 7 year: 2007 ident: pone.0316019.ref025 article-title: Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction publication-title: Circulation doi: 10.1161/CIRCULATIONAHA.106.672402 – volume: 143 start-page: 29 issue: 1 year: 1982 ident: pone.0316019.ref013 article-title: The meaning and use of the area under a receiver operating characteristic (ROC) curve publication-title: Radiology doi: 10.1148/radiology.143.1.7063747 – volume: 13 start-page: 603 issue: 8 year: 2016 ident: pone.0316019.ref003 article-title: Classification evaluation publication-title: Nature Methods doi: 10.1038/nmeth.3945 – ident: pone.0316019.ref001 – start-page: 120 volume-title: Proceedings of the Evaluation and Assessment in Software Engineering year: 2020 ident: pone.0316019.ref047 doi: 10.1145/3383219.3383232 – volume: 136 start-page: 71 year: 2020 ident: pone.0316019.ref006 article-title: On the performance of Matthews correlation coefficient (MCC) for imbalanced dataset publication-title: Pattern Recognition Letters doi: 10.1016/j.patrec.2020.03.030 – ident: pone.0316019.ref026 doi: 10.1109/ICIST.2012.6221710 – volume: 95 start-page: 511 issue: 7 year: 2003 ident: pone.0316019.ref018 article-title: The Central Role of Receiver Operating Characteristic (ROC) Curves in Evaluating Tests for the Early Detection of Cancer publication-title: JNCI Journal of the National Cancer Institute doi: 10.1093/jnci/95.7.511 – volume: 16 issue: 1 year: 2023 ident: pone.0316019.ref012 article-title: The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification publication-title: BioData Mining doi: 10.1186/s13040-023-00322-4 – volume: 68 start-page: 855 issue: 8 year: 2015 ident: pone.0316019.ref044 article-title: The precision–recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases publication-title: Journal of Clinical Epidemiology doi: 10.1016/j.jclinepi.2015.02.010 – volume-title: Bayes’ rule: Doing Bayesian Data Analysis year: 2015 ident: pone.0316019.ref048 doi: 10.1016/B978-0-12-405888-0.00005-2 – volume: 9 start-page: 47112 year: 2021 ident: pone.0316019.ref034 article-title: The Benefits of the Matthews Correlation Coefficient (MCC) Over the Diagnostic Odds Ratio (DOR) in Binary Classification Assessment publication-title: IEEE Access doi: 10.1109/ACCESS.2021.3068614 – volume: 37 start-page: e1 year: 2022 ident: pone.0316019.ref019 article-title: Evaluation metrics and dimensional reduction for binary classification algorithms: a case study on bankruptcy prediction publication-title: The Knowledge Engineering Review doi: 10.1017/S026988892100014X – volume: 3 year: 2015 ident: pone.0316019.ref035 article-title: Threshold-Free Measures for Assessing the Performance of Medical Screening Tests publication-title: Frontiers in Public Health doi: 10.3389/fpubh.2015.00057 – volume: 52 start-page: 12049 issue: 10 year: 2022 ident: pone.0316019.ref009 article-title: General Performance Score for classification problems publication-title: Applied Intelligence doi: 10.1007/s10489-021-03041-7 – volume: 25 start-page: 3977 issue: 5 year: 2020 ident: pone.0316019.ref020 article-title: On the assessment of software defect prediction models via ROC curves publication-title: Empirical Software Engineering doi: 10.1007/s10664-020-09861-4 – volume: 165 start-page: 1433 issue: 4 year: 2023 ident: pone.0316019.ref030 article-title: Limitations of receiver operating characteristic curve on imbalanced data: Assist device mortality risk scores publication-title: The Journal of Thoracic and Cardiovascular Surgery doi: 10.1016/j.jtcvs.2021.07.041 – volume: 31 start-page: 419 issue: 3 year: 2022 ident: pone.0316019.ref033 article-title: Considerations on the region of interest in the ROC space publication-title: Statistical Methods in Medical Research doi: 10.1177/09622802211060515 – volume: 14 start-page: 13 issue: 1 year: 2021 ident: pone.0316019.ref008 article-title: The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation publication-title: BioData Mining doi: 10.1186/s13040-021-00244-z – volume: 4 start-page: eaao5580 issue: 1 year: 2018 ident: pone.0316019.ref049 article-title: The accuracy, fairness, and limits of predicting recidivism publication-title: Sci Adv doi: 10.1126/sciadv.aao5580 – volume: 20 start-page: 4 issue: 1 year: 2020 ident: pone.0316019.ref041 article-title: A new concordant partial AUC and partial c statistic for imbalanced data in the evaluation of machine learning algorithms publication-title: BMC Medical Informatics and Decision Making doi: 10.1186/s12911-019-1014-6 – volume: 13 start-page: 1224 issue: 6 year: 2022 ident: pone.0316019.ref036 article-title: The uniform AUC: Dealing with the representativeness effect in presence–absence models publication-title: Methods in Ecology and Evolution doi: 10.1111/2041-210X.13826 – volume: 9 start-page: 410 issue: 1 year: 2008 ident: pone.0316019.ref037 article-title: Not proper ROC curves as new tool for the analysis of differentially expressed genes in microarray experiments publication-title: BMC Bioinformatics doi: 10.1186/1471-2105-9-410 – volume: 28 start-page: 570 issue: 3 year: 2014 ident: pone.0316019.ref042 article-title: The total operating characteristic to measure diagnostic ability for multiple thresholds publication-title: International Journal of Geographical Information Science doi: 10.1080/13658816.2013.862623 – volume: 26 start-page: 1348 issue: 10 year: 2010 ident: pone.0316019.ref039 article-title: A CROC stronger than ROC: measuring, visualizing and optimizing early retrieval publication-title: Bioinformatics doi: 10.1093/bioinformatics/btq140 – volume: 11 start-page: 51515 year: 2023 ident: pone.0316019.ref011 article-title: Common Problems With the Usage of F-Measure and Accuracy Metrics in Medical Research publication-title: IEEE Access doi: 10.1109/ACCESS.2023.3278996 – volume: 101 start-page: 90 issue: 1 year: 2003 ident: pone.0316019.ref021 article-title: ROC curves are a suitable and flexible tool for the analysis of gene expression profiles publication-title: Cytogenetic and Genome Research doi: 10.1159/000074404 – ident: pone.0316019.ref002 doi: 10.5220/0003783303100313 – volume: 8 start-page: 283 issue: 4 year: 1978 ident: pone.0316019.ref014 article-title: Basic principles of ROC analysis publication-title: Seminars in Nuclear Medicine doi: 10.1016/S0001-2998(78)80014-2 – volume: 91 start-page: 216 year: 2019 ident: pone.0316019.ref004 article-title: The impact of class imbalance in classification performance metrics based on the binary confusion matrix publication-title: Pattern Recognition doi: 10.1016/j.patcog.2019.02.023 – ident: pone.0316019.ref043 doi: 10.1145/347090.347126 – volume: 5 start-page: 34 issue: 19 year: 2017 ident: pone.0316019.ref023 article-title: The receiver operating characteristic (ROC) curve publication-title: The Southwest Respiratory and Critical Care Chronicles doi: 10.12746/swrccc.v5i19.391 – start-page: 104 volume-title: Intelligent Data Engineering and Automated Learning—IDEAL 2020 year: 2020 ident: pone.0316019.ref045 doi: 10.1007/978-3-030-62365-4_10 |
| SSID | ssj0053866 |
| Score | 2.5362659 |
| Snippet | The proper use of model evaluation metrics is important for model evaluation and model selection in binary classification tasks. This study investigates how... |
| SourceID | plos doaj pubmedcentral proquest gale pubmed crossref |
| SourceType | Open Website Open Access Repository Aggregation Database Index Database Enrichment Source |
| StartPage | e0316019 |
| SubjectTerms | Accuracy Area Under Curve Biology and Life Sciences Classification Computer and Information Sciences Data analysis Data Interpretation, Statistical Datasets Evaluation Evaluation Studies as Topic Machine Learning Medicine and Health Sciences Models, Statistical Physical Sciences Ranking Research and Analysis Methods ROC Curve Sample Size Simulation Social Sciences Thresholds Variance Variance analysis |
| SummonAdditionalLinks | – databaseName: Nursing & Allied Health Database dbid: 7RV link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1Lb9QwELZg4cAFKI92oYBBSMAh7a6d-HFCy4oKJNSihVa9RfEjbaWSLJvd_n5mHG_aoAqQuGYmUTKeGX-OZz4T8hop2AubFgnjTiSpY-NEGa2TorTOc60KEaoJj77I_X11fKy_xh9uTSyrXOfEkKhdbfEf-S4H3A6-BLPd-_nPBE-Nwt3VeITGTXJrjNgY_FnOjtaZGGJZiNgux-V4N47Ozryu_A44swj8Olemo8Da3-Xmwfy8bq4Dnr_XT16ZkPbu_e-n3Cd3IxSlk9Z3NsgNXz0gGzHYG_o2MlK_e0gOJwAtKbabLSgARjo7mNLpanHh6WnRhCs_6mZJLRbbgtdUS3pJIk4BFVMTun6pRaiOtUlB8ogc7n38Pv2UxPMYEis0XyYKFoNpIY10wjPrnRl7lRpVqsxnIyfwIJ-SwXqrTGUmpWdOpgISRiq1yZTXjj8mgwpsv0XoCFCYc6UT2mncSTSYSrwVrMwMN6IYEr4eltxGsnI8M-M8DztwEhYtrZFyHMw8DuaQJN1d85as4y_6H3DEO12k2g4X6sVJHiM3T5lzHBd60ui0FFo5y9LCWJdqmblMDskL9Je87VvtEkY-UQyLXpUeDcmroIF0GxXW85wUq6bJPx8c_YPSt1lP6U1UKmswhy1iDwV8E9J49TS3e5qQNGxPvIXevbZKk1_6JNy59trrxS87MT4Ua_QqX69aHYTbAp6-2QZIZ1lAvQAtlRgS1Qudnun7kursNLCdj3Fje8T5kz-_11NyhwHexEojxrfJYLlY-Wfktr1YnjWL5yEv_ALPSmxy priority: 102 providerName: ProQuest – databaseName: Public Library of Science (PLoS) Journals Open Access dbid: FPL link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Lb9QwELZg4cAFKK9uKWAQEnBI2bUdP47LihVIVVsVWvUWxY_QSm2y2uz29zOTeFNStQKumbGVjMeTb-SZz4S8Rwr23Ik8YdzLRHg2TrQ1JskL5wM3OpdNNeHxrtrb0ycn5uAqUbx2gs_V-HO06c68KsMOuCBkEOYuuce4lJhszQ5215EX9q6UsT3utpG930_D0t_F4sH8vKpvAprX6yX_-AHNHv3vqz8mDyPUpJPWNzbInVA-IRtxM9f0Y2Sc_vSUHE0AOlJsJ1tQAIT0cH9Kp6vFZaCned08uajqJXVYTAteUS7pFUk4BdRLbdPVSx1Ccaw9aiTPyNHs68_ptyTet5A4afgy0ZDsiVxZ5WVgLng7DlpYXeg0pCMv8aKegkE-VQiVKhWYV0JCQBDK2FQH4_lzMijhUzcJHQHK8r7w0niDJ4UWQ0VwkhWp5VbmQ8LXy5C5SEaOd2KcZ80Jm4KkpDVShrbLou2GJOlGzVsyjr_of8EV7nSRSrt5AIuUxZ2ZCeY9x0ROWSMKabR3TOTWeWFU6lM1JG_QP7K2L7ULCNlEMyxq1WY0JO8aDaTTKLFe51e-quvs-_7xPyj9OOwpfYhKRQXmcHnskYBvQpqunuZ2TxOCguuJN9Gb11apMw5pL4RiAIswcu3hN4vfdmKcFGvwylCtWh2E0xJmf9FuiM6ygGoBOmo5JLq3VXqm70vKs9OGzXyMB9cjzrduf-WX5AEDLIlVRIxvk8FysQqvyH13uTyrF6-bGPAbTxFajQ priority: 102 providerName: Public Library of Science |
| Title | Area under the ROC Curve has the most consistent evaluation for binary classification |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/39715186 https://www.proquest.com/docview/3148793175 https://www.proquest.com/docview/3148839260 https://pubmed.ncbi.nlm.nih.gov/PMC11666033 https://doaj.org/article/42dd309607b94f698dc24abcd4975d57 http://dx.doi.org/10.1371/journal.pone.0316019 |
| Volume | 19 |
| WOSCitedRecordID | wos001400361000006&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: Directory of Open Access Journals - DOAJ (NTUSG) customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: DOA dateStart: 20060101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: M~E dateStart: 20060101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre – providerCode: PRVPQU databaseName: Advanced Technologies & Aerospace Database customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: P5Z dateStart: 20061201 isFulltext: true titleUrlDefault: https://search.proquest.com/hightechjournals providerName: ProQuest – providerCode: PRVPQU databaseName: Agricultural Science Database customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: M0K dateStart: 20061201 isFulltext: true titleUrlDefault: https://search.proquest.com/agriculturejournals providerName: ProQuest – providerCode: PRVPQU databaseName: Biological Science Database customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: M7P dateStart: 20061201 isFulltext: true titleUrlDefault: http://search.proquest.com/biologicalscijournals providerName: ProQuest – providerCode: PRVPQU databaseName: Engineering Database customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: M7S dateStart: 20061201 isFulltext: true titleUrlDefault: http://search.proquest.com providerName: ProQuest – providerCode: PRVPQU databaseName: Environmental Science Database customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: PATMY dateStart: 20061201 isFulltext: true titleUrlDefault: http://search.proquest.com/environmentalscience providerName: ProQuest – providerCode: PRVPQU databaseName: Health Medical collection customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: 7X7 dateStart: 20061201 isFulltext: true titleUrlDefault: https://search.proquest.com/healthcomplete providerName: ProQuest – providerCode: PRVPQU databaseName: Materials Science Database customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: KB. dateStart: 20061201 isFulltext: true titleUrlDefault: http://search.proquest.com/materialsscijournals providerName: ProQuest – providerCode: PRVPQU databaseName: Nursing & Allied Health Database customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: 7RV dateStart: 20061201 isFulltext: true titleUrlDefault: https://search.proquest.com/nahs providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: BENPR dateStart: 20061201 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: Public Health Database customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: 8C1 dateStart: 20061201 isFulltext: true titleUrlDefault: https://search.proquest.com/publichealth providerName: ProQuest – providerCode: PRVPQU databaseName: Publicly Available Content Database customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: PIMPY dateStart: 20061201 isFulltext: true titleUrlDefault: http://search.proquest.com/publiccontent providerName: ProQuest – providerCode: PRVATS databaseName: Public Library of Science (PLoS) Journals Open Access customDbUrl: eissn: 1932-6203 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0053866 issn: 1932-6203 databaseCode: FPL dateStart: 20060101 isFulltext: true titleUrlDefault: http://www.plos.org/publications/ providerName: Public Library of Science |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3db9MwELeg8MALYnytMIpBSMBDujRO_PHYVquYtnVRx6qxlyixHTZppFXT7u_nzklLgyaNB17uIT5H7d3Z-Z189zMhn5CCPdVh6gXMcC80Qc-TmVJemmtjmZIpd9WE02MxHsuLCxVvXfWFNWEVPXBluP0wMIYhzhaZCnOupNFBmGbahEpEJnJ95L5Q62Sq2oNhFXNeN8ox0duv_dKdzwrbhTDmjlln60Pk-Po3u3JrfjMr74Kcf1dObn2KRs_I0xpD0n7123fIA1s8Jzv1Ki3pl5pK-usLct4HTEixT2xBAenRyemQDleLW0uv0tI9-TUrl1RjlSy4u1jSP-zfFOAszVy7LtWIsbGoyI28JOejg-_Db159kYKnuWJLT0IWF6YiE4bbQFuT9awMM5nLyEa-4XgDTx5AopSHIhLCBkaEHFZ6KFQWSasMe0VaBZhul1Af4JMxueHKKDwCzHAPsJoHeZSxjKdtwtZWTXTNMo6XXdwk7uhMQLZRGSlBXyS1L9rE28yaVywb9-gP0GEbXeTIdg8gcpI6cpL7IqdN3qO7k6rhdLPSk74MsFpVKr9NPjoN5MkosBDnZ7oqy-TwdPoPSmeThtLnWimfgTl0Wjc_wH9C_q2G5l5DE1a7bgzvYnCurVImDPJZ2GMBBcLMdcDePfxhM4wvxeK6ws5WlQ7iZA5vf13F98ayAFcBE0reJrIR-Q3TN0eK6ytHU97DE2mfsTf_w1lvyZMA4CQWEgVsj7SWi5V9Rx7r2-V1ueiQh2IyRXkhnJQg5bDXIY8GB-N40nGbA8hRfAzyaNAFeeIfoRSxk2cg4-gSZsSHJ_GP3_k9aqY |
| linkProvider | Directory of Open Access Journals |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9NAEF6VgAQXoLwaKHRBIODgNlmv93FAKASqRg1pVdqq4mLs3XVbqdghTor4U_xGZvxIa1QBlx64esYrezwz-413HoQ8xxbskeGRx3wrPG5Z11Ox1l6UGOt8rSJRZBPuD-VopA4O9PYC-VnXwmBaZe0TC0dtM4P_yNd8wO2gS7DbvR1_83BqFJ6u1iM0SrXYdD--Q8iWvxm8h-_7grH1D7v9Da-aKuAZof2ppyCk4ZGMpRWOGWfjrlM8VokKXNCxAsfRJAyihoTLQErHrOQC1J5LHQfKaevDulfIVc5ZB61oO_hce37wHUJU5Xm-7K5V2rA6zlK3CsYjin4-57a_YkrAfC9ojU-y_CKg-3u-5rkNcP3W_ya62-RmBbVpr7SNRbLg0jtksXJmOX1Vddx-fZfs9QA6Uyynm1AAxHRnq0_7s8mpo0dRXlz5muVTajCZGKwindKzJukUUD-Ni6pmajAUwdyrgnKP7F3K690nrRS-9RKhHUCZ1iZWaKvxpDRGV-mMYEkQ-7GI2sSv1SA0VTN2nAlyEhYnjBKCslJIISpPWClPm3jzu8ZlM5K_8L9DDZvzYivx4kI2OQwrzxRyZq2PgayMNU-EVtYwHsXGci0DG8g2WUH9DMu63LlDDHuKYVKv0p02eVZwYDuRFPOVDqNZnoeDrf1_YPq002B6WTElGYjDRFWNCLwTtilrcC43OMEpmgZ5Ca2plkoentkA3FlbycXkp3MyLoo5iKnLZiUPhhMCVn9QGuRcsoDqATor0SaqYaoN0Tcp6fFR0c29iwf3Hd9_-OfnWiHXN3Y_DsPhYLT5iNxggK0xq4r5y6Q1nczcY3LNnE6P88mTwidR8uWyLfkXXsPHgg |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9NAEF6VgBAXoLwaKHRBIODgJtm193FAKKRERK3SqtCq4mLs3XVbqdghTor4a_w6ZvxIa1QBlx647oxX8Xi-2ZnsPAh5ji3YI-NHHuNWeL5lPU_FWntRYqzjWkWiyCbc35LjsTo40DtL5GddC4NplbVNLAy1zQz-R97h4LeDLsFp10mqtIidjeHbyTcPJ0jhTWs9TqNUkU334zuEb_mb0QZ86xeMDd9_GnzwqgkDnhGazzwF4Y0fyVha4ZhxNu455ccqUYELulbgaJqEQQSR-DKQ0jErfQEQ8KWOA-W05bDvFXIVVjWmE-4En-tTAOyIEFWpHpe9TqUZ65MsdesAJFH09jl3FBYTAxbnQmtykuUXOb2_526eOwyHt_5nMd4mNysXnPZLzCyTJZfeIcuVkcvpq6oT9-u7ZK8PLjXFMrspBUeZ7m4P6GA-PXX0KMqLla9ZPqMGk4wBLemMnjVPpxAN0LiodqYGQxTMySoo98jepbzefdJK4buvENoF79PaxAptNd6gxmhCnREsCWIei6hNeK0SoamatOOskJOwuHmUEKyVQgpRkcJKkdrEWzw1KZuU_IX_HWrbghdbjBcL2fQwrCxW6DNrOQa4MtZ-IrSyhvlRbKyvZWAD2SZrqKthWa-7MJRhXzFM9lW62ybPCg5sM5Kiph1G8zwPR9v7_8D0cbfB9LJiSjIQh4mq2hF4J2xf1uBcbXCCsTQN8goiq5ZKHp7hAZ6sEXMx-emCjJtibmLqsnnJg2GGgN0flOBcSBa8fXCplWgT1YBtQ_RNSnp8VHR57-GFfpfzh3_-XWvkOgA43BqNNx-RGwxcbky2YnyVtGbTuXtMrpnT2XE-fVKYJ0q-XDaQfwGwytBM |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Area+under+the+ROC+Curve+has+the+most+consistent+evaluation+for+binary+classification&rft.jtitle=PloS+one&rft.au=Jing+Li&rft.date=2024-12-23&rft.pub=Public+Library+of+Science+%28PLoS%29&rft.eissn=1932-6203&rft.volume=19&rft.issue=12&rft_id=info:doi/10.1371%2Fjournal.pone.0316019&rft.externalDBID=DOA&rft.externalDocID=oai_doaj_org_article_42dd309607b94f698dc24abcd4975d57 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1932-6203&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1932-6203&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1932-6203&client=summon |