Out of One, Many: Using Language Models to Simulate Human Samples

We propose and explore the possibility that language models can be studied as effective proxies for specific human subpopulations in social science research. Practical and research applications of artificial intelligence tools have sometimes been limited by problematic biases (such as racism or sexi...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Political analysis Ročník 31; číslo 3; s. 337 - 351
Hlavní autoři: Argyle, Lisa P., Busby, Ethan C., Fulda, Nancy, Gubler, Joshua R., Rytting, Christopher, Wingate, David
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York, USA Cambridge University Press 01.07.2023
Témata:
Algorithms
Artificial intelligence
Attitudes
Baby boomers
Bias
Conditioning
Cultural factors
Fidelity
Humans
Language
Language attitudes
Language modeling
Natural language
Probability distribution
Property
Public opinion
Racial bias
Racism
Research applications
Science
Sexism
Silicon
Social factors
Social research
Social sciences
Sociocultural factors
Sociodemographics
United States > US
public opinion
machine learning
computational social science
artificial intelligence
ISSN:1047-1987, 1476-4989
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We propose and explore the possibility that language models can be studied as effective proxies for specific human subpopulations in social science research. Practical and research applications of artificial intelligence tools have sometimes been limited by problematic biases (such as racism or sexism), which are often treated as uniform properties of the models. We show that the “algorithmic bias” within one such tool—the GPT-3 language model—is instead both fine-grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups. We term this property algorithmic fidelity and explore its extent in GPT-3. We create “silicon samples” by conditioning the model on thousands of sociodemographic backstories from real human participants in multiple large surveys conducted in the United States. We then compare the silicon and human samples to demonstrate that the information contained in GPT-3 goes far beyond surface similarity. It is nuanced, multifaceted, and reflects the complex interplay between ideas, attitudes, and sociocultural context that characterize human attitudes. We suggest that language models with sufficient algorithmic fidelity thus constitute a novel and powerful tool to advance understanding of humans and society across a variety of disciplines.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1047-1987
1476-4989
DOI:10.1017/pan.2023.2