Topic models do not model topics: epistemological remarks and steps towards best practices

The social sciences and digital humanities have recently adopted the machine learning technique of topic modeling to address research questions in their fields. This is problematic in a number of ways, some of which have not received much attention in the debate yet. This paper adds epistemological...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of data mining and digital humanities Ročník 2021
Hlavní autor: Shadrova, Anna
Médium: Journal Article
Jazyk:angličtina
Vydáno: INRIA 27.10.2021
Nicolas Turenne
Témata:
[info.info-dl]computer science [cs]/digital libraries [cs.dl]
[shs.langue]humanities and social sciences/linguistics
[shs]humanities and social sciences
Computer Science
digital humanities
Digital Libraries
Humanities and Social Sciences
information extraction for scientific inquiry
Linguistics
topic modeling
Topic modeling
digital humanities
information extraction for scientific inquiry
ISSN:2416-5999, 2416-5999
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The social sciences and digital humanities have recently adopted the machine learning technique of topic modeling to address research questions in their fields. This is problematic in a number of ways, some of which have not received much attention in the debate yet. This paper adds epistemological concerns centering around the interface between topic modeling and linguistic concepts and the argumentative embedding of evidence obtained through topic modeling. It concludes that topic modeling in its present state of methodological integration does not meet the requirements of an independent research method. It operates from relevantly unrealistic assumptions, is non-deterministic, cannot effectively be validated against a reasonable number of competing models, does not lock into a well-defined linguistic interface, and does not scholarly model topics in the sense of themes or content. These features are intrinsic and make the interpretation of its results prone to apophenia (the human tendency to perceive random sets of elements as meaningful patterns) and confirmation bias (the human tendency to perceptually prefer patterns that are in alignment with pre-existing biases). While partial validation of the statistical model is possible, a conceptual validation would require an extended triangulation with other methods and human ratings, and clarification of whether statistical distinctivity of lexical co-occurrence correlates with conceputal topics in any reliable way.
ISSN:2416-5999
2416-5999
DOI:10.46298/jdmdh.7595