AI-Assisted Qualitative Coding: Comparing Human to Machine Outputs

This study explores the application of generative AI in the qualitative coding of survey responses, comparing its performance to that of human coders. By using ChatGPT-4o, we aimed to automate the coding process traditionally performed manually, assessing the AI's ability to identify themes and...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IPCC proceedings (Print) s. 230 - 233
Hlavní autoři: Hodges, Amy, Ponce, Timothy, Seawright, Leslie
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 20.07.2025
Témata:
Accuracy
Artificial intelligence
Codes
Encoding
Generative AI
human vs. AI
Iterative methods
qualitative data coding
research methodology
survey responses
Surveys
Testing
Training
ISSN:2158-1002
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This study explores the application of generative AI in the qualitative coding of survey responses, comparing its performance to that of human coders. By using ChatGPT-4o, we aimed to automate the coding process traditionally performed manually, assessing the AI's ability to identify themes and patterns within textual data. Our findings reveal that while AI demonstrates remarkable efficiency and speed, it struggles with the nuanced understanding required for complex coding tasks. The AI frequently misinterpreted coding definitions and over-relied on certain codes, indicating a need for more balanced training data and iterative refinement. Despite these challenges, AI proved valuable in providing summative feedback and identifying overall trends, suggesting its potential as a complementary tool in qualitative research. The study underscores the importance of developing codebooks collaboratively with AI and highlights the necessity of human oversight to ensure accuracy and depth.
ISSN:2158-1002
DOI:10.1109/ProComm64814.2025.00051