Assessing the Impact of Integrating ChatGPT as an Advice Generator in Educational Software
This paper reports on the study of the integration of ChatGPT as an advice generator in custom educational software developed for Java programming. The software, in cooperation with ChatGPT API, pursues providing real-time, context-specific advice to students for better learning. This work adopted a...
Uložené v:
| Vydané v: | 2024 9th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM) s. 127 - 133 |
|---|---|
| Hlavní autori: | , , , , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
20.09.2024
|
| Predmet: | |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | This paper reports on the study of the integration of ChatGPT as an advice generator in custom educational software developed for Java programming. The software, in cooperation with ChatGPT API, pursues providing real-time, context-specific advice to students for better learning. This work adopted a two-fold evaluation approach to evaluating this integration. First, this study examines the effectiveness of this integration with the help of the Interrupted Time Series Analysis methodology to measure possible improvement in the performance metrics of the students in terms of error rates and task completion times. Second, this work presents a custom-designed questionnaire used to get student perceptions regarding the clarity, usefulness, and impact of ChatGPT's advice, and the level of student satisfaction with the user interface. The key takeaways from this research study are the substantial improvements in performance metrics that were noted quantitatively, with students achieving lower error rates and faster completion times after the intervention of ChatGPT. Qualitatively, learners express their satisfaction with the clarity of advice, which gives them an understanding that works on their learning and confidence in Java programming. These findings point toward the promise of integrating such advanced AI solutions in educational software toward a significant improvement in learning outcomes and the necessity of human-aided continuous user feedback for system refinement. |
|---|---|
| AbstractList | This paper reports on the study of the integration of ChatGPT as an advice generator in custom educational software developed for Java programming. The software, in cooperation with ChatGPT API, pursues providing real-time, context-specific advice to students for better learning. This work adopted a two-fold evaluation approach to evaluating this integration. First, this study examines the effectiveness of this integration with the help of the Interrupted Time Series Analysis methodology to measure possible improvement in the performance metrics of the students in terms of error rates and task completion times. Second, this work presents a custom-designed questionnaire used to get student perceptions regarding the clarity, usefulness, and impact of ChatGPT's advice, and the level of student satisfaction with the user interface. The key takeaways from this research study are the substantial improvements in performance metrics that were noted quantitatively, with students achieving lower error rates and faster completion times after the intervention of ChatGPT. Qualitatively, learners express their satisfaction with the clarity of advice, which gives them an understanding that works on their learning and confidence in Java programming. These findings point toward the promise of integrating such advanced AI solutions in educational software toward a significant improvement in learning outcomes and the necessity of human-aided continuous user feedback for system refinement. |
| Author | Krouska, Akrivi Papakostas, Christos Troussas, Christos Sgouropoulou, Cleo Mylonas, Phivos |
| Author_xml | – sequence: 1 givenname: Christos surname: Troussas fullname: Troussas, Christos email: ctrouss@uniwa.gr organization: University of West Attica,Department of Informatics and Computer Engineering,Egaleo,Greece – sequence: 2 givenname: Akrivi surname: Krouska fullname: Krouska, Akrivi email: akrouska@uniwa.gr organization: University of West Attica,Department of Informatics and Computer Engineering,Egaleo,Greece – sequence: 3 givenname: Christos surname: Papakostas fullname: Papakostas, Christos email: cpapakostas@uniwa.gr organization: University of West Attica,Department of Informatics and Computer Engineering,Egaleo,Greece – sequence: 4 givenname: Phivos surname: Mylonas fullname: Mylonas, Phivos email: mylonasf@uniwa.gr organization: University of West Attica,Department of Informatics and Computer Engineering,Egaleo,Greece – sequence: 5 givenname: Cleo surname: Sgouropoulou fullname: Sgouropoulou, Cleo email: csgouro@uniwa.gr organization: University of West Attica,Department of Informatics and Computer Engineering,Egaleo,Greece |
| BookMark | eNotzM1Og0AUhuEx0YXW3oGLWbgF5w-YWRJESlJ_Eli5aQ7DmZakHRoYNd69GF19i-fLe0Mu_eiRkHvOYs6ZeWjK8jGPirJ4aZ5TqTIdCyZUzBiT_IKsTWa0lDxhSmlzTd7zecZ5HvyehgPS-nQGG-joaO0D7icIv1IcIFRvLYWZgqd5_zlYpBV6XHyc6OBp2X_Y5Tt6ONJmdOELJrwlVw6OM67_d0Xap7ItNtH2taqLfBsNhofIGQWaAQjTZcilTvs-tZ2wwukk4T3TUsjOSuQGDThmeaZcCn0iWWY7lwq5Ind_2QERd-dpOMH0veMsk2pR-QNxAFJs |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/SEEDA-CECNSM63478.2024.00031 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE/IET Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9798331504489 |
| EndPage | 133 |
| ExternalDocumentID | 10734623 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IL CBEJK RIE RIL |
| ID | FETCH-LOGICAL-i91t-f94a80aa29b7e1386dd6cb2c2f8551d08323bc3e19e9af0c174f6ad5307cbf623 |
| IEDL.DBID | RIE |
| IngestDate | Wed Nov 06 05:53:28 EST 2024 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i91t-f94a80aa29b7e1386dd6cb2c2f8551d08323bc3e19e9af0c174f6ad5307cbf623 |
| PageCount | 7 |
| ParticipantIDs | ieee_primary_10734623 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-Sept.-20 |
| PublicationDateYYYYMMDD | 2024-09-20 |
| PublicationDate_xml | – month: 09 year: 2024 text: 2024-Sept.-20 day: 20 |
| PublicationDecade | 2020 |
| PublicationTitle | 2024 9th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM) |
| PublicationTitleAbbrev | SEEDA-CECNSM |
| PublicationYear | 2024 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| Score | 1.8839839 |
| Snippet | This paper reports on the study of the integration of ChatGPT as an advice generator in custom educational software developed for Java programming. The... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 127 |
| SubjectTerms | Advice generator AI-Powered Tutoring Systems Artificial intelligence Artificial Intelligence in Education Chatbots ChatGPT in Education Custom Questionnaire Assessment Educational Software Error analysis Generators Interrupted Time Series Analysis Java Java Programming Learning Programming Education Programming profession Software Technology-Enhanced Learning Time measurement Time series analysis User Experience Evaluation User interfaces |
| Title | Assessing the Impact of Integrating ChatGPT as an Advice Generator in Educational Software |
| URI | https://ieeexplore.ieee.org/document/10734623 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA9uiHhSceI3OexabZu2aY6jdjrQMdiQ4WWkyQsOpJWt03_fl7ROLx68hYQQ8sL7SPJ-v0dIX2hWSA7M04r5XhQz1Dlmq7nzWCbS0qk4hNzzIx-P0_lcTFqwusPCAIBLPoMb23R_-bpSG_tUhhqOk9Ffd0iHc96AtfZIv-XNvJ3m-d3Ay_JsPH1KWMRt5lZoybF9W0LuV_EU5zuGB_9c9ZD0flB4dLL1L0dkB8pj8tL80mIHxdCNjhzIkVaGjlreBzuSvcr6fjKjck1lSQfaWgPaEEzjFZsuS7rN65BvdIqW-FOuoEdmw3yWPXhtfQRvKYLaMyKSqS9lKAoOAUsTrRNVhCo0KYZBGmOrkBWKQSBASOMrvHuYROoYtVoVBjd0QrplVcIpoYpHwhgMVoJIo9xiYRToBEKeyoDpOD0jPSuXxXvDgLH4Fsn5H_0XZN-K3uZVhP4l6darDVyRXfVRL9era3duXzV-mio |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5aRT2pWPFtDr2u7ibZR45l3dpiuxS6SPFSsnlgQbbSbvXvO9ldqxcP3kJCCJlhMpNkvm8Q6nBFcxFq6ihJXYf5FGyO2mruoS8CYelUKoTc8zBM02g65eMGrF5hYbTWVfKZvrPN6i9fLeTaPpWBhcNk8NfbaMdnjHg1XGsPdRrmzPtJkjx0nTiJ08kooCy0uVvE0mO7tojcr_IplffoHf5z3SPU_sHh4fHGwxyjLV2coJf6nxY6MARveFDBHPHC4EHD_GBH4ldRPo4zLFZYFLir7HmAa4ppuGTjeYE3mR3iDU_gLP4US91GWS_J4r7TVEhw5twrHcOZiFwhCM9D7dEoUCqQOZHERBAIKYiuCM0l1R7XXBhXwu3DBEL5YNcyN7ChU9QqFoU-Q1iGjBsD4YrHFMjN50ZqFWgSRsKjyo_OUdvKZfZec2DMvkVy8Uf_LdrvZ6PhbDhIny7RgVWDzbIg7hVqlcu1vka78qOcr5Y3lQ6_AGiynXE |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2024+9th+South-East+Europe+Design+Automation%2C+Computer+Engineering%2C+Computer+Networks+and+Social+Media+Conference+%28SEEDA-CECNSM%29&rft.atitle=Assessing+the+Impact+of+Integrating+ChatGPT+as+an+Advice+Generator+in+Educational+Software&rft.au=Troussas%2C+Christos&rft.au=Krouska%2C+Akrivi&rft.au=Papakostas%2C+Christos&rft.au=Mylonas%2C+Phivos&rft.date=2024-09-20&rft.pub=IEEE&rft.spage=127&rft.epage=133&rft_id=info:doi/10.1109%2FSEEDA-CECNSM63478.2024.00031&rft.externalDocID=10734623 |