Emotion Classification In Software Engineering Texts: A Comparative Analysis of Pre-trained Transformers Language Models
Emotion recognition in software engineering texts is critical for understanding developer expressions and improving collaboration. This paper presents a comparative analysis of state-of-the-art Pretrained Language Models (PTMs) for fine-grained emotion classification on two benchmark datasets from G...
Gespeichert in:
| Veröffentlicht in: | 2024 IEEE/ACM International Workshop on Natural Language-Based Software Engineering (NLBSE) S. 73 - 80 |
|---|---|
| 1. Verfasser: | |
| Format: | Tagungsbericht |
| Sprache: | Englisch |
| Veröffentlicht: |
ACM
20.04.2024
|
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Emotion recognition in software engineering texts is critical for understanding developer expressions and improving collaboration. This paper presents a comparative analysis of state-of-the-art Pretrained Language Models (PTMs) for fine-grained emotion classification on two benchmark datasets from GitHub and Stack Overflow We evaluate six transformer models - BERT, RoBERTa, ALBERT, DeBERTa, CodeBERT and GraphCodeBERT against the current best-performing tool SEntiMoji. Our analysis reveals consistent improvements ranging from 1.17 \% to 16.79 \% in terms of macroaveraged and micro-averaged F1 scores, with general domain models outperforming specialized ones. To further enhance PTMs, we incorporate polarity features in attention layer during training demonstrating additional average gains of 1.0 \% to 10.23 \% over baseline PTMs approaches. Our work provides strong evidence for the advancements afforded by PTMs in recognizing nuanced emotions like Anger, Love, Fear, Joy, Sadness, and Surprise in software engineering contexts. Through comprehensive benchmarking and error analysis, we also outline scope for improvements to address contextual gaps. |
|---|---|
| AbstractList | Emotion recognition in software engineering texts is critical for understanding developer expressions and improving collaboration. This paper presents a comparative analysis of state-of-the-art Pretrained Language Models (PTMs) for fine-grained emotion classification on two benchmark datasets from GitHub and Stack Overflow We evaluate six transformer models - BERT, RoBERTa, ALBERT, DeBERTa, CodeBERT and GraphCodeBERT against the current best-performing tool SEntiMoji. Our analysis reveals consistent improvements ranging from 1.17 \% to 16.79 \% in terms of macroaveraged and micro-averaged F1 scores, with general domain models outperforming specialized ones. To further enhance PTMs, we incorporate polarity features in attention layer during training demonstrating additional average gains of 1.0 \% to 10.23 \% over baseline PTMs approaches. Our work provides strong evidence for the advancements afforded by PTMs in recognizing nuanced emotions like Anger, Love, Fear, Joy, Sadness, and Surprise in software engineering contexts. Through comprehensive benchmarking and error analysis, we also outline scope for improvements to address contextual gaps. |
| Author | Imran, Mia Mohammad |
| Author_xml | – sequence: 1 givenname: Mia Mohammad surname: Imran fullname: Imran, Mia Mohammad email: imranm3@vcu.edu organization: Virginia Commonwealth University,Richmond,Virginia,USA |
| BookMark | eNotjFFLwzAUhSMoqHPPvviQP9CZNEmT-DbK1MFEwfk8btebEmiTkVTd_r1Fffo4nO-ca3IeYkBCbjlbcC7Vvaik0EYvJhom5BmZW22NZEwzpavyksxz9g1TlbTcVvaKHFdDHH0MtO5hqpzfw29cB_oe3fgNCekqdD4gJh86usXjmB_oktZxOECa5C-kywD9KftMo6NvCYsxwTRo6TZByC6mAVOmGwjdJ3RIX2KLfb4hFw76jPN_zsjH42pbPxeb16d1vdwUUBo7FlI0jClVgrXSOetMq1xZ7XWrGlQSmalQOeDYGKGUtg4QbGk0nwy7ByHFjNz9_XpE3B2SHyCddpxVUnOtxA-J5V7H |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK ESBDL RIE RIL |
| DOI | 10.1145/3643787.3648034 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Xplore Open Access Journals IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9798400705762 |
| EndPage | 80 |
| ExternalDocumentID | 10647175 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IL ACM ALMA_UNASSIGNED_HOLDINGS APO CBEJK ESBDL LHSKQ RIE RIL |
| ID | FETCH-LOGICAL-a289t-43b00552a994ff9f8d5f26c7d5be54e086e5fa1eb835579faea92871c7d9ca343 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001313494100015&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:03:10 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a289t-43b00552a994ff9f8d5f26c7d5be54e086e5fa1eb835579faea92871c7d9ca343 |
| OpenAccessLink | https://ieeexplore.ieee.org/document/10647175 |
| PageCount | 8 |
| ParticipantIDs | ieee_primary_10647175 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-April-20 |
| PublicationDateYYYYMMDD | 2024-04-20 |
| PublicationDate_xml | – month: 04 year: 2024 text: 2024-April-20 day: 20 |
| PublicationDecade | 2020 |
| PublicationTitle | 2024 IEEE/ACM International Workshop on Natural Language-Based Software Engineering (NLBSE) |
| PublicationTitleAbbrev | NLBSE |
| PublicationYear | 2024 |
| Publisher | ACM |
| Publisher_xml | – name: ACM |
| SSID | ssib056491969 |
| Score | 1.9108168 |
| Snippet | Emotion recognition in software engineering texts is critical for understanding developer expressions and improving collaboration. This paper presents a... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 73 |
| SubjectTerms | ALBERT Analytical models Benchmark testing BERT CodeBERT DeBERTa Emotion Classification Emotion recognition GraphCodeBERT Large Language Models RoBERTa Software Task analysis Training Transformers |
| Title | Emotion Classification In Software Engineering Texts: A Comparative Analysis of Pre-trained Transformers Language Models |
| URI | https://ieeexplore.ieee.org/document/10647175 |
| WOSCitedRecordID | wos001313494100015&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA62ePCkYsU3OXhNbfPYbLxJaVEopWCV3ko2mYAgW-lD_flO0l3biwdPG5aFwGQz-WYm832E3HonYm5NscwqDFCUKJgtvGBShwzhaxEgkVW_DvVolE-nZlw1q6deGABIl8-gHYeplu_nbh1TZbjDM_SlWjVIQ-ts06xV_zwqkyZSvVT0PV2p7kQsSuW6jc-8E5WRd_RT0vExOPznxEektW3Eo-PfI-aY7EF5Qr77G-kdmgQt41WfZF36VNJndKpfdgF0h2aQTtD_Lu_pA-1tmb5pTUZC5wFnAJakIsDTSQ1kERbSYZXMpFEx7X3ZIi-D_qT3yCoBBWYxjloxKeImU9waI0MwIfcq8MxprwpQEjCaARVsFwqEYUqbYMGaGEHhF8ZZIcUpaZbzEs4IVdbxgned5aBlR3prnJMWYxPhTY6Q7Jy0otlmHxuOjFltsYs_3l-SA47wINZleOeKNFeLNVyTffe5elsubtLK_gCB0qdF |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA46BT2pOPG3OXjt7JqkbbzJ2NiwjoFVdhtp8gKCdLIf6p_vS9a6XTx4aiiFwEvz8r338r6PkFujmcutiSBWAgMUwYpAFYYFPLExwtfCgierfs2S4TAdj-Woalb3vTAA4C-fQcsNfS3fTPXSpcpwh8foSxOxTXYE51G4ateqfx8Rc-nIXioCnzYXd8yVpdKkhc80dNrIGwoq_gDpHfxz6kPSXLfi0dHvIXNEtqA8Jt_dlfgO9ZKW7rKPty8dlPQZ3eqXmgHdIBqkOXrg-T19oJ011zet6Ujo1OIMEHixCDA0r6EsAkOaVelM6jTT3udN8tLr5p1-UEkoBAojqUXAmdtmIlJScmulTY2wUawTIwoQHDCeAWFVGwoEYiKRVoGSLobCL6RWjLMT0iinJZwSKpSOiqitVQQJD7lRUmuuMDphRqYIys5I05lt8rFiyZjUFjv_4_0N2evnT9kkGwwfL8h-hGDBVWmi8JI0FrMlXJFd_bl4m8-u_Sr_AGBfqow |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2024+IEEE%2FACM+International+Workshop+on+Natural+Language-Based+Software+Engineering+%28NLBSE%29&rft.atitle=Emotion+Classification+In+Software+Engineering+Texts%3A+A+Comparative+Analysis+of+Pre-trained+Transformers+Language+Models&rft.au=Imran%2C+Mia+Mohammad&rft.date=2024-04-20&rft.pub=ACM&rft.spage=73&rft.epage=80&rft_id=info:doi/10.1145%2F3643787.3648034&rft.externalDocID=10647175 |