Performance of ChatGPT on Specialty Certificate Examination in Dermatology multiple-choice questions

Abstract ChatGPT is a large language model trained on increasingly large datasets by OpenAI to perform language-based tasks. It is capable of answering multiple-choice questions, such as those posed by the Specialty Certificate Examination (SCE) in Dermatology. We asked two iterations of ChatGPT: Ch...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Clinical and experimental dermatology Ročník 49; číslo 7; s. 722 - 727
Hlavní autoři: Passby, Lauren, Jenko, Nathan, Wernham, Aaron
Médium: Journal Article
Jazyk:angličtina
Vydáno: UK Oxford University Press 25.06.2024
Témata:
Artificial Intelligence
Certification - standards
Chatbots
Clinical Competence - standards
Dermatology
Dermatology - education
Educational Measurement - methods
Educational Measurement - standards
Humans
Multiple choice
ISSN:0307-6938, 1365-2230, 1365-2230
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Abstract ChatGPT is a large language model trained on increasingly large datasets by OpenAI to perform language-based tasks. It is capable of answering multiple-choice questions, such as those posed by the Specialty Certificate Examination (SCE) in Dermatology. We asked two iterations of ChatGPT: ChatGPT-3.5 and ChatGPT-4 84 multiple-choice sample questions from the sample SCE in Dermatology question bank. ChatGPT-3.5 achieved an overall score of 63%, and ChatGPT-4 scored 90% (a significant improvement in performance; P < 0.001). The typical pass mark for the SCE in Dermatology is 70–72%. ChatGPT-4 is therefore capable of answering clinical questions and achieving a passing grade in these sample questions. There are many possible educational and clinical implications for increasingly advanced artificial intelligence (AI) and its use in medicine, including in the diagnosis of dermatological conditions. Such advances should be embraced provided that patient safety is a core tenet, and the limitations of AI in the nuances of complex clinical cases are recognized. ChatGPT is a large language model trained on increasingly large datasets by OpenAI to perform language-based tasks. ChatGPT-4 was asked 84 sample Specialty Certificate Examination (SCE) in Dermatology questions and it answered 90% correctly. There are many possible educational and clinical implications for increasingly advanced artificial intelligence (AI) and its use in medicine, including in the diagnosis of dermatological conditions. Such advances should be embraced provided that patient safety is a core tenet, and the limitations of AI in the nuances of complex clinical cases are recognized.
Bibliografie:SourceType-Scholarly Journals-1
content type line 14
ObjectType-Report-1
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0307-6938
1365-2230
1365-2230
DOI:10.1093/ced/llad197