ChatGPT outperforms crowd workers for text-annotation tasks
Many NLP applications require manual text annotations for a variety of tasks, notably to train classifiers or evaluate the performance of unsupervised models. Depending on the size and degree of complexity, the tasks may be conducted by crowd workers on platforms such as MTurk as well as trained ann...
Gespeichert in:
| Veröffentlicht in: | Proceedings of the National Academy of Sciences - PNAS Jg. 120; H. 30; S. e2305016120 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
United States
25.07.2023
|
| Schlagworte: | |
| ISSN: | 1091-6490, 1091-6490 |
| Online-Zugang: | Weitere Angaben |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Many NLP applications require manual text annotations for a variety of tasks, notably to train classifiers or evaluate the performance of unsupervised models. Depending on the size and degree of complexity, the tasks may be conducted by crowd workers on platforms such as MTurk as well as trained annotators, such as research assistants. Using four samples of tweets and news articles (
= 6,183), we show that ChatGPT outperforms crowd workers for several annotation tasks, including relevance, stance, topics, and frame detection. Across the four datasets, the zero-shot accuracy of ChatGPT exceeds that of crowd workers by about 25 percentage points on average, while ChatGPT's intercoder agreement exceeds that of both crowd workers and trained annotators for all tasks. Moreover, the per-annotation cost of ChatGPT is less than $0.003-about thirty times cheaper than MTurk. These results demonstrate the potential of large language models to drastically increase the efficiency of text classification. |
|---|---|
| AbstractList | Many NLP applications require manual text annotations for a variety of tasks, notably to train classifiers or evaluate the performance of unsupervised models. Depending on the size and degree of complexity, the tasks may be conducted by crowd workers on platforms such as MTurk as well as trained annotators, such as research assistants. Using four samples of tweets and news articles (n = 6,183), we show that ChatGPT outperforms crowd workers for several annotation tasks, including relevance, stance, topics, and frame detection. Across the four datasets, the zero-shot accuracy of ChatGPT exceeds that of crowd workers by about 25 percentage points on average, while ChatGPT's intercoder agreement exceeds that of both crowd workers and trained annotators for all tasks. Moreover, the per-annotation cost of ChatGPT is less than $0.003-about thirty times cheaper than MTurk. These results demonstrate the potential of large language models to drastically increase the efficiency of text classification.Many NLP applications require manual text annotations for a variety of tasks, notably to train classifiers or evaluate the performance of unsupervised models. Depending on the size and degree of complexity, the tasks may be conducted by crowd workers on platforms such as MTurk as well as trained annotators, such as research assistants. Using four samples of tweets and news articles (n = 6,183), we show that ChatGPT outperforms crowd workers for several annotation tasks, including relevance, stance, topics, and frame detection. Across the four datasets, the zero-shot accuracy of ChatGPT exceeds that of crowd workers by about 25 percentage points on average, while ChatGPT's intercoder agreement exceeds that of both crowd workers and trained annotators for all tasks. Moreover, the per-annotation cost of ChatGPT is less than $0.003-about thirty times cheaper than MTurk. These results demonstrate the potential of large language models to drastically increase the efficiency of text classification. Many NLP applications require manual text annotations for a variety of tasks, notably to train classifiers or evaluate the performance of unsupervised models. Depending on the size and degree of complexity, the tasks may be conducted by crowd workers on platforms such as MTurk as well as trained annotators, such as research assistants. Using four samples of tweets and news articles ( = 6,183), we show that ChatGPT outperforms crowd workers for several annotation tasks, including relevance, stance, topics, and frame detection. Across the four datasets, the zero-shot accuracy of ChatGPT exceeds that of crowd workers by about 25 percentage points on average, while ChatGPT's intercoder agreement exceeds that of both crowd workers and trained annotators for all tasks. Moreover, the per-annotation cost of ChatGPT is less than $0.003-about thirty times cheaper than MTurk. These results demonstrate the potential of large language models to drastically increase the efficiency of text classification. |
| Author | Gilardi, Fabrizio Alizadeh, Meysam Kubli, Maël |
| Author_xml | – sequence: 1 givenname: Fabrizio orcidid: 0000-0002-0635-3048 surname: Gilardi fullname: Gilardi, Fabrizio organization: Department of Political Science, University of Zurich, Zurich 8050, Switzerland – sequence: 2 givenname: Meysam orcidid: 0000-0001-6696-6471 surname: Alizadeh fullname: Alizadeh, Meysam organization: Department of Political Science, University of Zurich, Zurich 8050, Switzerland – sequence: 3 givenname: Maël orcidid: 0000-0002-5592-9648 surname: Kubli fullname: Kubli, Maël organization: Department of Political Science, University of Zurich, Zurich 8050, Switzerland |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/37463210$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNj81Lw0AUxBep2A89e5McvaS-fbvZJHiSUqtQ0EM9h03yFmub3bi7ofrfW7CCpxmGH8PMlI2ss8TYNYc5h1zc9VaHOQrIgCuOcMYmHEqeKlnC6J8fs2kIHwBQZgVcsLHIpRLIYcLuF-86rl43iRtiT94434Wk8e7QJgfnd-RDcsySSF8x1da6qOPW2STqsAuX7NzofaCrk87Y2-Nys3hK1y-r58XDOm0ylDHlQooW5XElJ8OxAFWQQiOUIYOyRaU1QC1zKgFkA3ltsCgaiVmWQc3bGmfs9re39-5zoBCrbhsa2u-1JTeECgtR5pJLUR7RmxM61B21Ve-3nfbf1d9h_AHYoFio |
| CitedBy_id | crossref_primary_10_1002_jee_70033 crossref_primary_10_1017_pan_2025_10017 crossref_primary_10_1038_s41598_025_92002_y crossref_primary_10_3390_foods14081355 crossref_primary_10_1177_20563051241254379 crossref_primary_10_1080_19312458_2024_2363776 crossref_primary_10_1177_08944393251366243 crossref_primary_10_1371_journal_pdig_0000631 crossref_primary_10_1017_rsm_2025_10031 crossref_primary_10_1177_00491241251338246 crossref_primary_10_1007_s40593_024_00414_0 crossref_primary_10_1111_pops_70004 crossref_primary_10_1016_j_jpi_2023_100338 crossref_primary_10_1371_journal_pone_0314136 crossref_primary_10_1162_tacl_a_00685 crossref_primary_10_1038_s42256_025_00986_z crossref_primary_10_1093_nsr_nwaf169 crossref_primary_10_2139_ssrn_5342180 crossref_primary_10_7717_peerj_cs_2358 crossref_primary_10_3390_electronics13081409 crossref_primary_10_1007_s11747_025_01097_2 crossref_primary_10_1007_s41109_024_00658_8 crossref_primary_10_2196_50150 crossref_primary_10_1002_jcpy_1380 crossref_primary_10_1109_ACCESS_2024_3402809 crossref_primary_10_3390_electronics13081525 crossref_primary_10_1111_tgis_70111 crossref_primary_10_1007_s12599_023_00834_7 crossref_primary_10_1080_17489725_2025_2501632 crossref_primary_10_1007_s10207_024_00860_w crossref_primary_10_1038_s41467_025_58344_x crossref_primary_10_1162_opmi_a_00144 crossref_primary_10_1080_1369118X_2025_2531130 crossref_primary_10_1007_s11263_025_02392_9 crossref_primary_10_1109_ACCESS_2025_3525493 crossref_primary_10_3758_s13428_024_02381_9 crossref_primary_10_1038_s41598_024_53124_x crossref_primary_10_1001_jamanetworkopen_2024_28276 crossref_primary_10_1007_s11192_025_05355_6 crossref_primary_10_3389_frai_2024_1365508 crossref_primary_10_1093_jamia_ocae197 crossref_primary_10_1080_00036846_2025_2532887 crossref_primary_10_1111_polp_12555 crossref_primary_10_1177_00491241251326819 crossref_primary_10_1007_s41060_024_00567_0 crossref_primary_10_1080_21645515_2024_2381925 crossref_primary_10_1108_EJIM_02_2024_0129 crossref_primary_10_1177_08944393251334977 crossref_primary_10_1002_ev_20557 crossref_primary_10_1073_pnas_2305016120 crossref_primary_10_1177_00491241251325243 crossref_primary_10_1007_s00146_025_02487_4 crossref_primary_10_1140_epjds_s13688_024_00456_3 crossref_primary_10_1057_s41599_025_05273_1 crossref_primary_10_1016_j_techfore_2025_124328 crossref_primary_10_20517_cf_2025_12 crossref_primary_10_1007_s42001_024_00272_9 crossref_primary_10_3390_app15042106 crossref_primary_10_1109_ACCESS_2024_3464374 crossref_primary_10_1109_MCI_2024_3504833 crossref_primary_10_3389_frsps_2024_1392128 crossref_primary_10_3390_app15148059 crossref_primary_10_2196_60678 crossref_primary_10_1177_08944393251349541 crossref_primary_10_1007_s40593_024_00431_z crossref_primary_10_1007_s42001_025_00388_6 crossref_primary_10_1007_s40820_024_01423_3 crossref_primary_10_1007_s10639_025_13633_2 crossref_primary_10_1057_s41599_025_04503_w crossref_primary_10_3390_info16050392 crossref_primary_10_1145_3676507 crossref_primary_10_1109_ACCESS_2023_3334260 crossref_primary_10_1080_10584609_2024_2364072 crossref_primary_10_3389_fpubh_2024_1352979 crossref_primary_10_1007_s10115_024_02321_1 crossref_primary_10_3389_fmed_2023_1259640 crossref_primary_10_1017_pan_2025_7 crossref_primary_10_1016_j_jss_2025_112533 crossref_primary_10_1177_16094069241231168 crossref_primary_10_1016_j_eswa_2025_126531 crossref_primary_10_1080_19312458_2025_2482538 crossref_primary_10_1162_coli_a_00561 crossref_primary_10_1109_ACCESS_2024_3354705 crossref_primary_10_1016_j_enpol_2025_114769 crossref_primary_10_1093_pnasnexus_pgae397 crossref_primary_10_1111_ecpo_12342 crossref_primary_10_1186_s12909_024_05239_y crossref_primary_10_1007_s44212_025_00074_y crossref_primary_10_1111_deci_12655 crossref_primary_10_1177_18344909251355673 crossref_primary_10_1038_s44387_025_00019_5 crossref_primary_10_1073_pnas_2314021121 crossref_primary_10_1145_3736402 crossref_primary_10_1007_s11432_023_3911_2 crossref_primary_10_1017_S0007123423000571 crossref_primary_10_1177_00491241241268453 crossref_primary_10_3758_s13428_024_02455_8 crossref_primary_10_1038_s44159_024_00392_z crossref_primary_10_1038_s42256_023_00783_6 crossref_primary_10_1007_s10462_025_11328_1 crossref_primary_10_1155_atr_4874071 crossref_primary_10_1007_s42001_024_00345_9 crossref_primary_10_2196_64447 crossref_primary_10_1017_pan_2024_23 crossref_primary_10_1093_pnasnexus_pgae165 crossref_primary_10_1080_10503307_2025_2539405 crossref_primary_10_1177_00491241251342008 crossref_primary_10_1016_j_aei_2025_103244 crossref_primary_10_1145_3710947 crossref_primary_10_3390_ijgi14090325 crossref_primary_10_3390_ai6040072 crossref_primary_10_1038_s41598_025_16650_w crossref_primary_10_1080_17502977_2024_2324567 crossref_primary_10_1080_10447318_2024_2392964 crossref_primary_10_2139_ssrn_5352879 crossref_primary_10_1177_2755323X251366728 crossref_primary_10_1098_rsos_250128 crossref_primary_10_1038_s41562_024_01938_0 crossref_primary_10_1038_s44387_025_00008_8 crossref_primary_10_1177_23780231241259651 crossref_primary_10_1017_pan_2024_31 crossref_primary_10_1097_JS9_0000000000001583 crossref_primary_10_1080_00380237_2025_2516814 crossref_primary_10_1016_j_envint_2025_109341 crossref_primary_10_1093_applin_amae071 crossref_primary_10_1177_13563890251330911 crossref_primary_10_1016_j_jad_2024_08_013 crossref_primary_10_1177_20563051251350975 crossref_primary_10_1080_15309576_2025_2539784 crossref_primary_10_1109_ACCESS_2024_3437197 crossref_primary_10_1145_3695993 crossref_primary_10_1080_19312458_2025_2549707 crossref_primary_10_1017_pan_2024_29 crossref_primary_10_3390_electronics14142800 crossref_primary_10_1016_j_ins_2024_121735 crossref_primary_10_1140_epjds_s13688_025_00548_8 crossref_primary_10_1026_0033_3042_a000699 crossref_primary_10_1002_advs_202508623 crossref_primary_10_1111_acem_14995 crossref_primary_10_1016_j_procs_2025_01_286 crossref_primary_10_1016_j_eswa_2024_125509 crossref_primary_10_1186_s40537_025_01236_0 crossref_primary_10_2196_51635 crossref_primary_10_3390_info16090812 crossref_primary_10_1177_00222429251314385 crossref_primary_10_1177_08944393241286471 crossref_primary_10_1111_lsq_12481 crossref_primary_10_1016_j_jmsy_2024_05_020 crossref_primary_10_1177_08944393251344865 crossref_primary_10_1177_00491241251339188 crossref_primary_10_1177_08944393251326175 crossref_primary_10_1080_19312458_2025_2500329 crossref_primary_10_1145_3649506 crossref_primary_10_1007_s10584_025_03959_8 crossref_primary_10_1080_13678868_2024_2336866 crossref_primary_10_1177_00491241251340608 crossref_primary_10_3390_make7030068 crossref_primary_10_1016_j_cmpb_2025_109070 crossref_primary_10_1016_j_rcim_2025_103076 crossref_primary_10_1002_csr_70148 crossref_primary_10_1098_rsos_241692 crossref_primary_10_3758_s13428_024_02337_z crossref_primary_10_1007_s11192_025_05385_0 crossref_primary_10_1177_14614448241279034 crossref_primary_10_1038_s41698_025_01103_4 crossref_primary_10_1073_pnas_2309350120 crossref_primary_10_1088_3033_4942_addd43 crossref_primary_10_1109_ACCESS_2025_3554586 crossref_primary_10_1002_jcpy_1453 crossref_primary_10_1371_journal_pone_0331566 crossref_primary_10_1109_JPROC_2025_3572744 crossref_primary_10_1017_S0047279424000382 crossref_primary_10_1111_tgis_13282 crossref_primary_10_1017_psrm_2025_10010 crossref_primary_10_1017_S1537592724001907 crossref_primary_10_1109_TVCG_2024_3456373 crossref_primary_10_1016_j_jbi_2024_104724 crossref_primary_10_1109_ACCESS_2024_3349425 crossref_primary_10_32604_cmes_2024_052256 crossref_primary_10_1177_10422587241254069 crossref_primary_10_1111_jopy_13006 crossref_primary_10_1016_j_poetic_2024_101966 crossref_primary_10_1017_pan_2024_5 crossref_primary_10_1088_2631_8695_ad299e crossref_primary_10_2196_65454 crossref_primary_10_1080_10810730_2024_2411320 crossref_primary_10_1016_j_esp_2024_11_003 crossref_primary_10_1038_s44284_025_00305_y crossref_primary_10_1080_19312458_2025_2551693 crossref_primary_10_1109_TSE_2024_3356819 crossref_primary_10_1057_s41599_025_04877_x crossref_primary_10_1016_j_knosys_2025_113386 crossref_primary_10_2139_ssrn_5357471 crossref_primary_10_1177_00104140241271297 crossref_primary_10_1007_s11558_024_09577_w crossref_primary_10_1017_psrm_2024_35 crossref_primary_10_1093_pnasnexus_pgaf069 crossref_primary_10_1161_CIRCEP_124_013023 crossref_primary_10_3389_frsps_2025_1460277 crossref_primary_10_1007_s10664_024_10474_4 crossref_primary_10_1002_widm_1570 crossref_primary_10_3390_bdcc8040041 crossref_primary_10_1109_ACCESS_2024_3403426 crossref_primary_10_2139_ssrn_5317057 crossref_primary_10_1002_jrsm_1749 crossref_primary_10_1126_science_adq1814 crossref_primary_10_1057_s41599_024_02922_9 crossref_primary_10_1016_j_elerap_2025_101497 crossref_primary_10_1002_advs_202405789 crossref_primary_10_1080_08824096_2024_2410263 crossref_primary_10_1080_0144929X_2025_2551570 crossref_primary_10_3389_fcomm_2025_1610404 crossref_primary_10_1162_dint_a_00255 crossref_primary_10_1007_s11356_025_36566_2 crossref_primary_10_1109_TLT_2025_3570775 crossref_primary_10_1111_joms_13122 crossref_primary_10_1038_s43588_023_00585_1 crossref_primary_10_1177_00491241251336794 crossref_primary_10_1016_j_accinf_2024_100715 crossref_primary_10_3390_math13193062 crossref_primary_10_1016_j_osnem_2025_100319 crossref_primary_10_1038_s41598_025_98385_2 crossref_primary_10_3389_frai_2025_1602984 crossref_primary_10_1007_s10479_025_06713_6 crossref_primary_10_1162_qss_a_00360 crossref_primary_10_1016_j_jss_2025_112580 crossref_primary_10_1371_journal_pone_0313932 crossref_primary_10_1007_s42001_025_00362_2 crossref_primary_10_1016_j_landusepol_2025_107716 crossref_primary_10_1017_dap_2024_44 crossref_primary_10_1016_j_iswa_2023_200308 crossref_primary_10_1111_jcal_70089 crossref_primary_10_1057_s41304_024_00488_3 crossref_primary_10_1109_ACCESS_2025_3532995 crossref_primary_10_1177_00491241251333372 crossref_primary_10_3390_su16031166 crossref_primary_10_3233_WEB_230363 crossref_primary_10_1177_20531680241311510 crossref_primary_10_1109_JIOT_2024_3396282 crossref_primary_10_1057_s41599_025_05656_4 crossref_primary_10_1016_j_patter_2025_101341 crossref_primary_10_1109_ACCESS_2025_3540388 crossref_primary_10_1371_journal_pone_0302380 crossref_primary_10_1136_bmjopen_2023_082344 crossref_primary_10_1073_pnas_2401227121 crossref_primary_10_1371_journal_pone_0306621 crossref_primary_10_1017_S1049096525101248 crossref_primary_10_1080_19312458_2024_2383453 crossref_primary_10_1177_20531680251351910 crossref_primary_10_1007_s10579_025_09831_6 crossref_primary_10_1186_s12909_025_06881_w crossref_primary_10_1145_3703454 crossref_primary_10_1038_s43588_025_00843_4 crossref_primary_10_1007_s42001_024_00338_8 crossref_primary_10_3390_app132011171 crossref_primary_10_1007_s11943_023_00332_y |
| ContentType | Journal Article |
| DBID | NPM 7X8 |
| DOI | 10.1073/pnas.2305016120 |
| DatabaseName | PubMed MEDLINE - Academic |
| DatabaseTitle | PubMed MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic PubMed |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Sciences (General) |
| EISSN | 1091-6490 |
| ExternalDocumentID | 37463210 |
| Genre | Journal Article |
| GrantInformation_xml | – fundername: EC | European Research Council (ERC) grantid: 883121 |
| GroupedDBID | --- -DZ -~X .55 0R~ 123 29P 2FS 2WC 4.4 53G 5RE 5VS 85S AACGO AAFWJ AANCE ABOCM ABPLY ABPPZ ABTLG ABZEH ACGOD ACIWK ACNCT ACPRK AENEX AFFNX AFOSN AFRAH ALMA_UNASSIGNED_HOLDINGS BKOMP CS3 D0L DIK DU5 E3Z EBS F5P FRP GX1 H13 HH5 HYE JLS JSG KQ8 L7B LU7 N9A NPM N~3 O9- OK1 PNE PQQKQ R.V RHI RNA RNS RPM RXW SJN TAE TN5 UKR W8F WH7 WOQ WOW X7M XSW Y6R YBH YKV YSK ZCA ~02 ~KM 7X8 JENOY |
| ID | FETCH-LOGICAL-c524t-1343d242301ef128068e62f36fef24d26aa00b47e9004c07bf288c425550b1db2 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 391 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001083464000007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1091-6490 |
| IngestDate | Wed Oct 01 13:59:50 EDT 2025 Thu Apr 03 06:58:24 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 30 |
| Keywords | human annotations large language models text classification ChatGPT text as data |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c524t-1343d242301ef128068e62f36fef24d26aa00b47e9004c07bf288c425550b1db2 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0001-6696-6471 0000-0002-5592-9648 0000-0002-0635-3048 |
| OpenAccessLink | https://www.pnas.org/doi/10.1073/pnas.2305016120 |
| PMID | 37463210 |
| PQID | 2839741439 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_2839741439 pubmed_primary_37463210 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-07-25 |
| PublicationDateYYYYMMDD | 2023-07-25 |
| PublicationDate_xml | – month: 07 year: 2023 text: 2023-07-25 day: 25 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | Proceedings of the National Academy of Sciences - PNAS |
| PublicationTitleAlternate | Proc Natl Acad Sci U S A |
| PublicationYear | 2023 |
| SSID | ssj0009580 |
| Score | 2.7630637 |
| Snippet | Many NLP applications require manual text annotations for a variety of tasks, notably to train classifiers or evaluate the performance of unsupervised models.... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | e2305016120 |
| Title | ChatGPT outperforms crowd workers for text-annotation tasks |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/37463210 https://www.proquest.com/docview/2839741439 |
| Volume | 120 |
| WOSCitedRecordID | wos001083464000007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8QwEA7qevCirs_1RQQPeijbJmnT4EFkcfXisocV9lbSPFCEtpqu_n0nbRe9CIKXHgqBMsk383VmMh9CF0YSEdlYA75DHTDCbCA55UFujdAhz2mi80Zsgk8m6Xwupl3CzXVtlUuf2DhqXSqfIx9CGATqC9Fd3FRvgVeN8tXVTkJjFfUoUBkPTD5PfwzdTdtpBCIKEibC5WgfTodVIZ1vgo495SHh7_yyiTPjrf9-4Tba7Bgmvm2PRB-tmGIH9TsMO3zZDZq-2kXXo2dZ309nuFzUVXuBwGFwzJ8a-3YtIIYY3mHfGxLIoijbqj2upXt1e-hpfDcbPQSdmEKgYsK85Dyj2pOnMDI28vXU1CTE0sQaS5gmiZRhmDNuBMBGwS5ZkqYKEA2_MHmkc7KP1oqyMIcIKx3DJjJNwTUySnNYmeo0FEoyoaNEDdD50kAZHFZfgZCFKRcu-zbRAB20Vs6qdqpGRjlL_IWioz-sPkYbXvbd51hJfIJ6FqBqTtG6-qhf3PtZcwrgOZk-fgFS8rwP |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=ChatGPT+outperforms+crowd+workers+for+text-annotation+tasks&rft.jtitle=Proceedings+of+the+National+Academy+of+Sciences+-+PNAS&rft.au=Gilardi%2C+Fabrizio&rft.au=Alizadeh%2C+Meysam&rft.au=Kubli%2C+Ma%C3%ABl&rft.date=2023-07-25&rft.eissn=1091-6490&rft.volume=120&rft.issue=30&rft.spage=e2305016120&rft_id=info:doi/10.1073%2Fpnas.2305016120&rft_id=info%3Apmid%2F37463210&rft_id=info%3Apmid%2F37463210&rft.externalDocID=37463210 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1091-6490&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1091-6490&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1091-6490&client=summon |