Inter-Coder Agreement in One-to-Many Classification: Fuzzy Kappa

Content analysis involves classification of textual, visual, or audio data. The inter-coder agreement is estimated by making two or more coders to classify the same data units, with subsequent comparison of their results. The existing methods of agreement estimation, e.g., Cohen's kappa, requir...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:PloS one Jg. 11; H. 3; S. e0149787
Hauptverfasser: Kirilenko, Andrei P., Stepchenkova, Svetlana
Format: Journal Article
Sprache:Englisch
Veröffentlicht: United States Public Library of Science 02.03.2016
Public Library of Science (PLoS)
Schlagworte:
Audio data
Biology and Life Sciences
Building codes
Classification
Clinical Coding - methods
Coders
Coding
Communication
Computer and Information Sciences
Content analysis
Cultural differences
Culture
Databases, Factual
Earth Sciences
Fuzzy logic
Geography
Humans
Medicine and Health Sciences
Methods
People and places
Research and Analysis Methods
Social Sciences
Sports management
Tourism
United States > US
ISSN:1932-6203, 1932-6203
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Content analysis involves classification of textual, visual, or audio data. The inter-coder agreement is estimated by making two or more coders to classify the same data units, with subsequent comparison of their results. The existing methods of agreement estimation, e.g., Cohen's kappa, require that coders place each unit of content into one and only one category (one-to-one coding) from the pre-established set of categories. However, in certain data domains (e.g., maps, photographs, databases of texts and images), this requirement seems overly restrictive. The restriction could be lifted, provided that there is a measure to calculate the inter-coder agreement in the one-to-many protocol. Building on the existing approaches to one-to-many coding in geography and biomedicine, such measure, fuzzy kappa, which is an extension of Cohen's kappa, is proposed. It is argued that the measure is especially compatible with data from certain domains, when holistic reasoning of human coders is utilized in order to describe the data and access the meaning of communication.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
Competing Interests: The authors have declared that no competing interests exist.
Analyzed the data: APK. Wrote the paper: APK SS. Contributed the data: SS.
ISSN:1932-6203
1932-6203
DOI:10.1371/journal.pone.0149787