Trust Dynamics in AI-Assisted Development: Definitions, Factors, and Implications
Software developers increasingly rely on AI code generation utilities. To ensure that "good" code is accepted into the code base and "bad" code is rejected, developers must know when to trust an AI suggestion. Understanding how developers build this intuition is crucial to enhanc...
Gespeichert in:
| Veröffentlicht in: | Proceedings / International Conference on Software Engineering S. 1678 - 1690 |
|---|---|
| Hauptverfasser: | , , , , , , |
| Format: | Tagungsbericht |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
26.04.2025
|
| Schlagworte: | |
| ISSN: | 1558-1225 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Software developers increasingly rely on AI code generation utilities. To ensure that "good" code is accepted into the code base and "bad" code is rejected, developers must know when to trust an AI suggestion. Understanding how developers build this intuition is crucial to enhancing developer-AI collaborative programming. In this paper, we seek to understand how developers (1) define and (2) evaluate the trustworthiness of a code suggestion and (3) how trust evolves when using AI code assistants. To answer these questions, we conducted a mixed method study consisting of an in-depth exploratory survey with (n=29) developers followed by an observation study (n=10). We found that comprehensibility and perceived correctness were the most frequently used factors to evaluate code suggestion trustworthiness. However, the gap in developers' definition and evaluation of trust points to a lack of support for evaluating trustworthy code in real-time. We also found that developers often alter their trust decisions, keeping only 52% of original suggestions. Based on these findings, we extracted four guidelines to enhance developer-AI interactions. We validated the guidelines through a survey with (n=7) domain experts and survey members (n=8). We discuss the validated guidelines, how to apply them, and tools to help adopt them. |
|---|---|
| AbstractList | Software developers increasingly rely on AI code generation utilities. To ensure that "good" code is accepted into the code base and "bad" code is rejected, developers must know when to trust an AI suggestion. Understanding how developers build this intuition is crucial to enhancing developer-AI collaborative programming. In this paper, we seek to understand how developers (1) define and (2) evaluate the trustworthiness of a code suggestion and (3) how trust evolves when using AI code assistants. To answer these questions, we conducted a mixed method study consisting of an in-depth exploratory survey with (n=29) developers followed by an observation study (n=10). We found that comprehensibility and perceived correctness were the most frequently used factors to evaluate code suggestion trustworthiness. However, the gap in developers' definition and evaluation of trust points to a lack of support for evaluating trustworthy code in real-time. We also found that developers often alter their trust decisions, keeping only 52% of original suggestions. Based on these findings, we extracted four guidelines to enhance developer-AI interactions. We validated the guidelines through a survey with (n=7) domain experts and survey members (n=8). We discuss the validated guidelines, how to apply them, and tools to help adopt them. |
| Author | Ziyadi, Morteza Lindemann, Lars Medvidovic, Nenad Sabouri, Sadra Zhou, Xinyi Eibl, Philipp Chattopadhyay, Souti |
| Author_xml | – sequence: 1 givenname: Sadra surname: Sabouri fullname: Sabouri, Sadra email: sabourih@usc.edu organization: University of Southern California,Department of Computer Science,Los Angeles,California – sequence: 2 givenname: Philipp surname: Eibl fullname: Eibl, Philipp email: eibl@usc.edu organization: University of Southern California,Department of Computer Science,Los Angeles,California – sequence: 3 givenname: Xinyi surname: Zhou fullname: Zhou, Xinyi email: xzhou141@usc.edu organization: University of Southern California,Department of Computer Science,Los Angeles,California – sequence: 4 givenname: Morteza surname: Ziyadi fullname: Ziyadi, Morteza email: mziyadi@amazon.com organization: Amazon AGI – sequence: 5 givenname: Nenad surname: Medvidovic fullname: Medvidovic, Nenad email: neno@usc.edu organization: University of Southern California,Department of Computer Science,Los Angeles,California – sequence: 6 givenname: Lars surname: Lindemann fullname: Lindemann, Lars email: llindema@usc.edu organization: University of Southern California,Department of Computer Science,Los Angeles,California – sequence: 7 givenname: Souti surname: Chattopadhyay fullname: Chattopadhyay, Souti email: schattop@usc.edu organization: University of Southern California,Department of Computer Science,Los Angeles,California |
| BookMark | eNotkF1LwzAYhaMouM39g130B9iZN9mbNN6VfWhhIOK8Hm_TFCJrWpoo7N871KtzHg48F2fKbkIfHGML4EsAbh6r9fsWUa70UnCBS87BmCs2N9oUUgJyVAau2QQQixyEwDs2jfGTc65WxkzY22H8iinbnAN13sbMh6ys8jJGH5Nrso37dqd-6FxITxdoffDJ9yE-ZDuyqR8vhUKTVd1w8pZ-p3t229Ipuvl_ztjHbntYv-T71-dqXe5zEoqnnGyhlCrAtQIBpGp4oRG1UbK1WpO1Td1SjTXUoDTUCokTETRoZC2hJTljiz-vd84dh9F3NJ6Pl0-EMaKQP2s8UgM |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/ICSE55347.2025.00199 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798331505691 |
| EISSN | 1558-1225 |
| EndPage | 1690 |
| ExternalDocumentID | 11029928 |
| Genre | orig-research |
| GroupedDBID | -~X .4S .DC 29O 5VS 6IE 6IF 6IH 6IK 6IL 6IM 6IN 8US AAJGR AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS ARCSS AVWKF BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO EDO FEDTE I-F IEGSK IJVOP IPLJI M43 OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-a260t-ac866681ef251136d087557963fc77accdbfab5b1b1671b65a0aaa1d593b31fa3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001538318100131&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 01:40:27 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a260t-ac866681ef251136d087557963fc77accdbfab5b1b1671b65a0aaa1d593b31fa3 |
| PageCount | 13 |
| ParticipantIDs | ieee_primary_11029928 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-April-26 |
| PublicationDateYYYYMMDD | 2025-04-26 |
| PublicationDate_xml | – month: 04 year: 2025 text: 2025-April-26 day: 26 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings / International Conference on Software Engineering |
| PublicationTitleAbbrev | ICSE |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0006499 |
| Score | 2.3038652 |
| Snippet | Software developers increasingly rely on AI code generation utilities. To ensure that "good" code is accepted into the code base and "bad" code is rejected,... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1678 |
| SubjectTerms | AI-code assistants Artificial intelligence Codes Collaboration Guidelines Programming Real-time systems Software Software development Software development management Software engineering Surveys Trust |
| Title | Trust Dynamics in AI-Assisted Development: Definitions, Factors, and Implications |
| URI | https://ieeexplore.ieee.org/document/11029928 |
| WOSCitedRecordID | wos001538318100131&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07b8IwED4V1KETfVD1LQ8dScFxbCfdKgoqUoWoSis25MdFYgkVhP7-2k6gLB262V4i3cl3-c733Qdwb2NqGSKPDLJelHiOrmY69dUmI6RimQ0qEZ-vcjxOZ7NsUpPVAxcGEUPzGT74ZXjLt0uz8aWyrktVLnrGaQMaUoqKrLULu8L9u9fcONrLuqP--4BzlkiHAWNfNwnjXfcUVEICGbb--eljaP9S8chkl2RO4ACLU2httRhIfTXP4G3qyRPkuRKYX5NFQZ5GkTO-d6Mle71Bj26TL4qqVatDhpXgToeowpLRXn95Gz6Gg2n_JarlEiLlQEkZKZM6LJJSzD1sYML6YfWeaspyI6Uyxupcaa6ppkJSLbjqKaWo5RnTjOaKnUOzWBZ4AcRoxBRTSrUViUSemSSjXBoXHRVzkOQS2t5E869qIsZ8a52rP86v4ch7wb_CxOIGmuVqg7dwaL7LxXp1F_z4A-lsnqw |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED5BQYKpPIp444GxoXUcxwkbKq0aUaoiCupW-XGRsqSoD34_dpKWLgxstpdId_JdvvN99wHcG58ahsg9jaztBY6jq5iKXLVJh0Ky2BQqEZ8DMRxGk0k8qsjqBRcGEYvmM3xwy-It38z0ypXKWjZV2ejpR7uw56SzKrrWJvCG9u-9YsfRdtxKOu9dzlkgLAr0XeWkGPC6paFSpJBe_Z8fP4LGLxmPjDZp5hh2MD-B-lqNgVSX8xTexo4-QZ5LifkFyXLylHjW_M6Rhmx1Bz3aTZrlZbNWk_RKyZ0mkbkhyVaHeQM-et1xp-9VggmetLBk6UkdWTQSUUwdcGChcePqHdmUpVoIqbVRqVRcUUVDQVXIZVtKSQ2PmWI0lewMavksx3MgWiFGGFGqTBgI5LEOYsqFtvFRMgtKLqDhTDT9KmdiTNfWufzj_A4O-uPXwXSQDF-u4NB5xL3J-OE11JbzFd7Avv5eZov5beHTH10qofU |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%2F+International+Conference+on+Software+Engineering&rft.atitle=Trust+Dynamics+in+AI-Assisted+Development%3A+Definitions%2C+Factors%2C+and+Implications&rft.au=Sabouri%2C+Sadra&rft.au=Eibl%2C+Philipp&rft.au=Zhou%2C+Xinyi&rft.au=Ziyadi%2C+Morteza&rft.date=2025-04-26&rft.pub=IEEE&rft.eissn=1558-1225&rft.spage=1678&rft.epage=1690&rft_id=info:doi/10.1109%2FICSE55347.2025.00199&rft.externalDocID=11029928 |