A Bayesian index of association: comparison with other measures and performance

The article discusses a Bayesian measure of association, B-index, and compares it with the other existing measures of agreement, association, and similarity, both chance-corrected and non-corrected: Scott’s π, Krippendorff’s α, Cohen’s κ, Bennett, Alpert & Goldstein’s S, Cosine similarity, and t...

Full description

Saved in:
Bibliographic Details
Published in:Quality & quantity Vol. 58; no. 1; pp. 277 - 305
Main Author: Oleinik, Anton
Format: Journal Article
Language:English
Published: Dordrecht Springer Netherlands 01.02.2024
Springer
Springer Nature B.V
Subjects:
Algorithms
Analysis
Associations
Bayesian analysis
Computational linguistics
Computer mediated communication
Computer science
Content analysis
Indexes
Information management
Language processing
Methodology of the Social Sciences
Natural language interfaces
Psychology
Social Sciences
Measures of inter-coder agreement
Classification
Measures of similarity
PageRank
Annotation
Consensus
Content analysis
ISSN:0033-5177, 1573-7845
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The article discusses a Bayesian measure of association, B-index, and compares it with the other existing measures of agreement, association, and similarity, both chance-corrected and non-corrected: Scott’s π, Krippendorff’s α, Cohen’s κ, Bennett, Alpert & Goldstein’s S, Cosine similarity, and the Jaccard similarity coefficient. PageRank adapted to particularities of annotation is also added to this list. Two versions of B-index are considered: with the informative and non-informative priors. An algorithm for calculating B-index written in pseudocode is provided. Particular attention is devoted to the uses of those measures in content analysis, communication studies, computational linguistics, psychology, computer science and network science. Real-world data gathered using an online platform for content analysis allowed comparing the behavior of all eight measures included in the scope of analysis. Three short texts (164 data points/sentences in total) were coded by 66 annotators. The behaviors of B-index with the non-informative prior and Bennett, Alpert & Goldstein’s S have some common patterns.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0033-5177
1573-7845
DOI:10.1007/s11135-023-01639-2