A technological construction of society: Comparing GPT‐4 and human respondents for occupational evaluation in the UK

Despite initial research about the biases and perceptions of large language models (LLMs), we lack evidence on how LLMs evaluate occupations, especially in comparison to human evaluators. In this paper, we present a systematic comparison of occupational evaluations by GPT‐4 with those from an in‐dep...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:British journal of industrial relations Ročník 63; číslo 1; s. 180 - 208
Hlavní autoři: Gmyrek, Paweł, Lutz, Christoph, Newlands, Gemma
Médium: Journal Article
Jazyk:angličtina
Vydáno: London Blackwell Publishing Ltd 01.03.2025
Témata:
Chatbots
Humans
Language modeling
Large language models
Occupational prestige
Occupations
Prestige
Program evaluation
Respondents
Sociological research
Stigma
United Kingdom > UK
ISSN:0007-1080, 1467-8543
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Despite initial research about the biases and perceptions of large language models (LLMs), we lack evidence on how LLMs evaluate occupations, especially in comparison to human evaluators. In this paper, we present a systematic comparison of occupational evaluations by GPT‐4 with those from an in‐depth, high‐quality and recent human respondents survey in the UK. Covering the full ISCO‐08 occupational landscape, with 580 occupations and two distinct metrics (prestige and social value), our findings indicate that GPT‐4 and human scores are highly correlated across all ISCO‐08 major groups. At the same time, GPT‐4 substantially under‐ or overestimates the occupational prestige and social value of many occupations, particularly for emerging digital and stigmatized or illicit occupations. Our analyses show both the potential and risk of using LLM‐generated data for sociological and occupational research. We also discuss the policy implications of our findings for the integration of LLM tools into the world of work.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0007-1080
1467-8543
DOI:10.1111/bjir.12840