Is Chatgpt a menace for creative writing ability? An experiment - Niloy, Ahnaf Chowdhury; Akter, Salma; Sultana, Nayeema; Sultana, Jakia; Rahman, Sayed Imran Ur (Journal of computer assisted learning, 2024)
Background The increasing prevalence of Artificial Intelligence (AI) language models, exemplified by ChatGPT, has sparked inquiries into their influence on creative writing skills in educational contexts. This study aims to quantitatively investigate whether ChatGPT's use negatively affects university students' creative writing abilities, focusing on originality, content presentation, accuracy, and elaboration in essays. The research adopts an experimental approach to shed light on this concern. Objective This study aims to quantitatively investigate whether the utilization of ChatGPT, an AI chatbot, adversely affects specific dimensions of creative writing skills among university students, with an emphasis on originality, content presentation, accuracy, and elaboration. Method The experimental study involves 600 students from 10 universities, divided into a control and an experimental group (EGp). The EGp incorporates ChatGPT in their creative writing process as an intervention. The study evaluates originality, content presentation, accuracy, and elaboration, utilizing the Wilcoxon Signed‐Rank Test for analysis. Results and Conclusion The findings reveal a detrimental association between ChatGPT use and university students' creative writing abilities. Analysing both machine‐based and human‐based assessments substantiates earlier qualitative observations regarding ChatGPT's adverse impact on creative writing. This study highlights the necessity of approaching AI integration, particularly in creative writing disciplines, with caution. While AI tools have merits, their integration should be thoughtful, considering the potential drawbacks. These insights inform future research and educational practices, guiding the effective incorporation of AI while nurturing students' writing skills. Lay Description What is already known about this topic ChatGPT poses an ethical dilemma regarding its use in the field of academia Qualitative claims and opinions have been raised in prior studies regarding its use in the creative writing process Prior studies have both supported and opposed its use but with very limited quantitative approaches while most of the opinions remain qualitative Some prior studies opine in support of ChatGPT's ability as an author Several factors measuring creativity has been identified by previous studies but a constructive approach in the light of advanced Artificial Intelligence (AI) based chatbots like ChatGPT is missing in such literature What this paper adds An experimental approach to provide a valid quantitative proof of the qualitative claims over ChatGPT's detrimental effect towards creativity in writing, which was absent in prior studies A multifactor‐based formula to measure creativity in a quantitative form A quantitative view of the factors that are affected in either a positive way or a negative way in a user by ChatGPT, providing a holistic picture to determine its extent of use A statistical and theoretical understanding over an unexplored topic like creative writing in the light of ChatGPT A quantitative proof why ChatGPT should not be considered as an author Implications for practice and/or policy Educators may implement changes in assigning tasks to students compared to their earlier practices, based on the identified factors that are being affected negatively, to ensure ChatGPT does not hinder a student's creativity at a greater extent The extent of using ChatGPT should be limited to self‐learning as positive effect was experienced through the experiment Policymakers may use the findings of the study to impose strict policies in academia for ensuring academic integrity (Example: must use of plagiarism detecting software for checking scripts, assigning tasks to students which require more analytical abilities, providing tasks which are not properly readable by LLM's like ChatGPT such as image‐based questions, case studies etc.)