How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversations

Large Language Models (LLMs), like ChatGPT, have gained widespread popularity and usage in various software engineering tasks, including refactoring, testing, code review, and program comprehension. Despite recent studies delving into refactoring documentation in commit messages, issues, and code re...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings (IEEE/ACM International Conference on Mining Software Repositories. Online) pp. 202 - 206
Main Authors: AlOmar, Eman Abdullah, Venkatakrishnan, Anushkrishna, Mkaouer, Mohamed Wiem, Newman, Christian D., Ouni, Ali
Format: Conference Proceeding
Language:English
Published: ACM 15.04.2024
Subjects:
Chatbots
ChatGPT
Codes
Documentation
mining software repositories
Oral communication
Refactoring documentation
Reviews
Software quality
Text mining
ISSN:2574-3864
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Large Language Models (LLMs), like ChatGPT, have gained widespread popularity and usage in various software engineering tasks, including refactoring, testing, code review, and program comprehension. Despite recent studies delving into refactoring documentation in commit messages, issues, and code review, little is known about how developers articulate their refactoring needs when interacting with ChatGPT. In this paper, our goal is to explore conversations between developers and ChatGPT related to refactoring to better understand how developers identify areas for improvement in code and how ChatGPT addresses developers' needs. Our approach relies on text mining refactoring-related conversations from 17,913 ChatGPT prompts and responses, and investigating developers' explicit refactoring intention. Our results reveal that (1) developer-ChatGPT conversations commonly involve generic and specific terms/phrases; (2) developers often make generic refactoring requests, while ChatGPT typically includes the refactoring intention; and (3) various learning settings when prompting ChatGPT in the context of refactoring. We envision that our findings contribute to a broader understanding of the collaboration between developers and AI models.CCS CONCEPTS* Software Engineering → Software Quality; Refactoring.
AbstractList Large Language Models (LLMs), like ChatGPT, have gained widespread popularity and usage in various software engineering tasks, including refactoring, testing, code review, and program comprehension. Despite recent studies delving into refactoring documentation in commit messages, issues, and code review, little is known about how developers articulate their refactoring needs when interacting with ChatGPT. In this paper, our goal is to explore conversations between developers and ChatGPT related to refactoring to better understand how developers identify areas for improvement in code and how ChatGPT addresses developers' needs. Our approach relies on text mining refactoring-related conversations from 17,913 ChatGPT prompts and responses, and investigating developers' explicit refactoring intention. Our results reveal that (1) developer-ChatGPT conversations commonly involve generic and specific terms/phrases; (2) developers often make generic refactoring requests, while ChatGPT typically includes the refactoring intention; and (3) various learning settings when prompting ChatGPT in the context of refactoring. We envision that our findings contribute to a broader understanding of the collaboration between developers and AI models.CCS CONCEPTS* Software Engineering → Software Quality; Refactoring.
Author AlOmar, Eman Abdullah
Newman, Christian D.
Ouni, Ali
Venkatakrishnan, Anushkrishna
Mkaouer, Mohamed Wiem
Author_xml – sequence: 1
  givenname: Eman Abdullah
  surname: AlOmar
  fullname: AlOmar, Eman Abdullah
  email: ealomar@stevens.edu
  organization: Stevens Institute of Technology,Hoboken,New Jersey,USA
– sequence: 2
  givenname: Anushkrishna
  surname: Venkatakrishnan
  fullname: Venkatakrishnan, Anushkrishna
  email: av3278@rit.edu
  organization: Rochester Institute of Technology,Rochester,New York,USA
– sequence: 3
  givenname: Mohamed Wiem
  surname: Mkaouer
  fullname: Mkaouer, Mohamed Wiem
  email: mmkaouer@umich.edu
  organization: University of Michigan-Flint,Flint,Michigan,USA
– sequence: 4
  givenname: Christian D.
  surname: Newman
  fullname: Newman, Christian D.
  email: cnewman@se.rit.edu
  organization: Rochester Institute of Technology,Rochester,New York,USA
– sequence: 5
  givenname: Ali
  surname: Ouni
  fullname: Ouni, Ali
  email: ali.ouni@etsmtl.ca
  organization: University of Quebec,ETS Montreal,Montreal,Quebec,Canada
BookMark eNo9jV1LwzAYhaMoOOeuvfEif6DzzVebXMmocxMGis5bR5q-cYWZjDZO---tKF49Bw7POefkJMSAhFwymDIm1bXIpTCGTQcq0OyITExhtAQoQOlCH5MRV4XMhM7lGZl0XVOByjk3SpoReV3GT5oifUJvXYotTdumo2Ws8YbOAp1_7XextUPR0-f0Ufc0BnqLB9zFPbZZubVp8bj-t5vwNrjhgG1nUxNDd0FOvd11OPnjmLzczdflMls9LO7L2SqzXOqU1cKCRgDmKs557bUROZPeMcd_MndMM29AgrG5rn3lazscIhaVQqeVE2Ny9bvbIOJm3zbvtu03DJRSGnLxDegkVyA
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1145/3643991.3645081
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9798400705878
EISSN 2574-3864
EndPage 206
ExternalDocumentID 10555806
Genre orig-research
GroupedDBID 6IE
6IL
6IN
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
OCL
RIE
RIL
ID FETCH-LOGICAL-a248t-d3a08e001cb222df893614fc1c2f8932c181f90409a68dfbfdafacee7b5ec85c3
IEDL.DBID RIE
ISICitedReferencesCount 9
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001267321100032&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Thu May 08 06:04:09 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a248t-d3a08e001cb222df893614fc1c2f8932c181f90409a68dfbfdafacee7b5ec85c3
PageCount 5
ParticipantIDs ieee_primary_10555806
PublicationCentury 2000
PublicationDate 2024-April-15
PublicationDateYYYYMMDD 2024-04-15
PublicationDate_xml – month: 04
  year: 2024
  text: 2024-April-15
  day: 15
PublicationDecade 2020
PublicationTitle Proceedings (IEEE/ACM International Conference on Mining Software Repositories. Online)
PublicationTitleAbbrev MSR
PublicationYear 2024
Publisher ACM
Publisher_xml – name: ACM
SSID ssib056229549
ssj0003211714
Score 2.0054228
Snippet Large Language Models (LLMs), like ChatGPT, have gained widespread popularity and usage in various software engineering tasks, including refactoring, testing,...
SourceID ieee
SourceType Publisher
StartPage 202
SubjectTerms Chatbots
ChatGPT
Codes
Documentation
mining software repositories
Oral communication
Refactoring documentation
Reviews
Software quality
Text mining
Title How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversations
URI https://ieeexplore.ieee.org/document/10555806
WOSCitedRecordID wos001267321100032&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELWgYmACRBHf8sDqNo6dJp4QqigdUFWhInWiiu2LYElQSUH8e-6ctGVhYLMs2Yl8id_Z790dYzc2Q5gcGC-UT0Eg3mphpcuELXTi8XyAHnaw9GM6mWTzuZm2weohFgYAgvgMetQMXL6v3IquyvoyZKeiBNu7aTpogrXWHw_i-Jayom1Y4dEmlbpN5yN10leEvkb2iHmLKJH1r3oqAU5GB_98kUPW3Qbm8ekGco7YDpTH7GVcffG64k_QFM_hVGOYDysPt_yu5I3KLpDpnFSD37wqeasVgqUYvub1w3S2GY3z4tiS1BrNZV6XPY_uZ8OxaMsmiDzWWS28yqMMEH6cRfD3BXokiMGFky6mduwQ1AuDP6_JB5kvbOFzfABAahNwWeLUCeuUVQmnjEdOaatwwzTWaK_J4LnEDpvSdJE8Y11anMV7kxljsV6X8z_6L9h-jE4BsTEyuWSdermCK7bnPuu3j-V1sOcPJbyg8Q
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA4yBT2pOPG3OXjN1rTp2p5EhnPiHEMm7ORoklf00krtFP9730u7zYsHbyGQtOS1-V7yfe89xq50jDDZS6wIbAQC8VYJLU0sdKZCi-cD9LCdpUfReBzPZsmkCVZ3sTAA4MRn0KGm4_JtYRZ0VdaVLjsVJdjepNJZTbjW8vNBJF-TVrQRB3i4iaRqEvpIFXYDwt9Edoh78yiV9a-KKg5QBrv_fJU91l6H5vHJCnT22QbkB-xlWHzxquBPUJfP4VRlmPcLC9f8Jue1zs7R6Zx0g9-8yHmjFoJS9F_T6m4yXY3GeXFsTnqN-jqvzZ4Ht9P-UDSFE0Tqq7gSNki9GBCAjEb4txn6JIjCmZHGp7ZvENazBH_fJO3FNtOZTfEBAJEOwcShCQ5ZKy9yOGLcM4HSAW6ZiU6UVWTyVGKHjmg6Tx6zNi3O_L3OjTFfrsvJH_2XbHs4fRzNR_fjh1O246OLQNyMDM9YqyoXcM62zGf19lFeONv-ALc5pDo
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28IEEE%2FACM+International+Conference+on+Mining+Software+Repositories.+Online%29&rft.atitle=How+to+Refactor+this+Code%3F+An+Exploratory+Study+on+Developer-ChatGPT+Refactoring+Conversations&rft.au=AlOmar%2C+Eman+Abdullah&rft.au=Venkatakrishnan%2C+Anushkrishna&rft.au=Mkaouer%2C+Mohamed+Wiem&rft.au=Newman%2C+Christian+D.&rft.date=2024-04-15&rft.pub=ACM&rft.eissn=2574-3864&rft.spage=202&rft.epage=206&rft_id=info:doi/10.1145%2F3643991.3645081&rft.externalDocID=10555806