DravidianCodeMix: sentiment analysis and offensive language identification dataset for Dravidian languages in code-mixed text

This paper describes the development of a multilingual, manually annotated dataset for three under-resourced Dravidian languages generated from social media comments. The dataset was annotated for sentiment analysis and offensive language identification for a total of more than 60,000 YouTube commen...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Language resources and evaluation Jg. 56; H. 3; S. 765 - 806
Hauptverfasser: Chakravarthi, Bharathi Raja, Priyadharshini, Ruba, Muralidaran, Vigneshwaran, Jose, Navya, Suryawanshi, Shardul, Sherly, Elizabeth, McCrae, John P.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Dordrecht Springer Netherlands 01.09.2022
Springer Nature B.V
Schlagworte:
Archaeology
Bilingualism
Code switching
Comments
Computational Linguistics
Computer mediated communication
Computer Science
Data mining
Datasets
Deep learning
Dravidian languages
English language
Experiments
Identification
Internet
Kannada language
Language
Language and Literature
Language identification
Languages
Linguistics
Machine learning
Malayalam
Mass media
Multilingualism
Natural language
Original Paper
Presidential elections
Product reviews
Reputation management
Scripts
Sentiment analysis
Social media
Social networks
Social Sciences
Tamil language
User generated content
Verbal communication
Dravidian languages
Code-mixed
Sentiment analysis
Malayalam
Corpora
Kannada
Offensive language identification
Tamil
ISSN:1574-020X, 1574-0218
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper describes the development of a multilingual, manually annotated dataset for three under-resourced Dravidian languages generated from social media comments. The dataset was annotated for sentiment analysis and offensive language identification for a total of more than 60,000 YouTube comments. The dataset consists of around 44,000 comments in Tamil-English, around 7000 comments in Kannada-English, and around 20,000 comments in Malayalam-English. The data was manually annotated by volunteer annotators and has a high inter-annotator agreement in Krippendorff’s alpha. The dataset contains all types of code-mixing phenomena since it comprises user-generated content from a multilingual country. We also present baseline experiments to establish benchmarks on the dataset using machine learning and deep learning methods. The dataset is available on Github and Zenodo.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1574-020X
1574-0218
DOI:10.1007/s10579-022-09583-7