Using Supervised Machine Learning in Automated Content Analysis: An Example Using Relational Uncertainty

The goal of this research is to make progress towards using supervised machine learning for automated content analysis dealing with complex interpretations of text. For Step 1, two humans coded a sub-sample of online forum posts for relational uncertainty. For Step 2, we evaluated reliability, in wh...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Communication methods and measures Ročník 13; číslo 4; s. 287 - 304
Hlavní autoři: Pilny, Andrew, McAninch, Kelly, Slone, Amanda, Moore, Kelsey
Médium: Journal Article
Jazyk:angličtina
Vydáno: Philadelphia Routledge 02.10.2019
Taylor & Francis Ltd
Témata:
Artificial intelligence
Automation
Content analysis
Machine learning
Personal communication
Reliability analysis
Uncertainty
ISSN:1931-2458, 1931-2466
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The goal of this research is to make progress towards using supervised machine learning for automated content analysis dealing with complex interpretations of text. For Step 1, two humans coded a sub-sample of online forum posts for relational uncertainty. For Step 2, we evaluated reliability, in which we trained three different classifiers to learn from those subjective human interpretations. Reliability was established when two different metrics of inter-coder reliability could not distinguish whether a human or a machine coded the text on a separate hold-out set. Finally, in Step 3 we assessed validity. To accomplish this, we administered a survey in which participants described their own relational uncertainty/certainty via text and completed a questionnaire. After classifying the text, the machine's classifications of the participants' text positively correlated with the subjects' own self-reported relational uncertainty and relational satisfaction. We discuss our results in line with areas of computational communication science, content analysis, and interpersonal communication.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1931-2458
1931-2466
DOI:10.1080/19312458.2019.1650166