GenAI Reliability in Content Analysis: Assessing Agreement Between LLMs in Measuring Discursive Violence

This study investigates the reliability of three leading large language models (LLMs), ChatGPT 4.5, Claude 3.7 Sonnet, and Gemini 2.0 Flash, in measuring discursive violence against women in Eminem's lyrics. Through a three-phase experimental design, we assessed both inter-coder reliability bet...

Full description

Bibliographic Details

Published in:	International Conference on Control Systems and Computer Science (Online) pp. 604 - 611
Main Authors:	Rughinis, Cosima, Dascalu, Mihai, Rasnayake, Susantha
Format:	Conference Proceeding
Language:	English
Published:	IEEE 27.05.2025
Subjects:	AI-assisted research Artificial intelligence Chatbots Content management Correlation Discursive violence Engines Generative AI Generative content analysis Inter-coder reliability Large language models Measurement fidelity Particle measurements Reliability Reliability engineering
ISSN:	2379-0482
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!