View in EDS

Is the information provided by large language models valid in educating patients about adolescent idiopathic scoliosis? An evaluation of content, clarity, and empathy

Saved in:
Bibliographic Details
Title: Is the information provided by large language models valid in educating patients about adolescent idiopathic scoliosis? An evaluation of content, clarity, and empathy
Authors: Lang, Siegmund, Vitale, Jacopo Antonino, Galbusera, Fabio, Fekete, Tamás F., Charles, Yann Philippe, Haddad, Sleiman, boissiere, louis, Núñez-Pereira, Susana
Contributors: Institut Català de la Salut, [Lang S] Department of Trauma Surgery, University Hospital Regensburg, Regensburg, Germany. Department of Spine Surgery, Schulthess Klinik, Zurich, Switzerland. [Vitale J, Galbusera F] Spine Center, Schulthess Klinik, Zurich, Switzerland. [Fekete T] Department of Spine Surgery, Schulthess Klinik, Zurich, Switzerland. [Boissiere L] Spine Unit Orthopaedic Department, Hôpital Pellegrin Bordeaux, Bordeaux, France. [Charles YP] Dept. of Spine Surgery, Hôpitaux Universitaires de Strasbourg, Université de Strasbourg, Strasbourg, France. [Núñez Pereira S, Haddad S] Unitat de Cirurgia de la Columna Vertebral, Vall d’Hebron Hospital Universitari, Barcelona, Spain, Vall d'Hebron Barcelona Hospital Campus
Source: Scientia
Publisher Information: Springer, 2025.
Publication Year: 2025
Subject Terms: Educació dels pacients, Metge i pacient, PSIQUIATRÍA Y PSICOLOGÍA::conducta y mecanismos de la conducta::psicología social::relaciones interpersonales::relaciones profesional-paciente::relaciones médico-paciente, Intel·ligència artificial, Escoliosi, DENOMINACIONES DE GRUPOS::personas::Grupos de Edad::adolescente, DISEASES::Musculoskeletal Diseases::Bone Diseases::Spinal Diseases::Spinal Curvatures::Scoliosis, FENÓMENOS Y PROCESOS::conceptos matemáticos::algoritmos::inteligencia artificial, Adolescents, PSYCHIATRY AND PSYCHOLOGY::Behavior and Behavior Mechanisms::Psychology, Social::Interpersonal Relations::Professional-Patient Relations::Physician-Patient Relations, NAMED GROUPS::Persons::Age Groups::Adolescent, PHENOMENA AND PROCESSES::Mathematical Concepts::Algorithms::Artificial Intelligence, ENFERMEDADES::enfermedades musculoesqueléticas::enfermedades óseas::enfermedades de la columna vertebral::desviaciones de la columna vertebral::escoliosis, HEALTH CARE::Health Care Facilities, Manpower, and Services::Health Services::Preventive Health Services::Health Education::Patient Education as Topic, ATENCIÓN DE SALUD::instalaciones, servicios y personal de asistencia sanitaria::servicios de salud::servicios preventivos de salud::educación en salud::educación de pacientes como asunto
Description: Purpose Large language models (LLM) have the potential to bridge knowledge gaps in patient education and enrich patient-surgeon interactions. This study evaluated three chatbots for delivering empathetic and precise adolescent idiopathic scoliosis (AIS) related information and management advice. Specifically, we assessed the accuracy, clarity, and relevance of the information provided, aiming to determine the effectiveness of LLMs in addressing common patient queries and enhancing their understanding of AIS. Methods We sourced 20 webpages for the top frequently asked questions (FAQs) about AIS and formulated 10 critical questions based on them. Three advanced LLMs—ChatGPT 3.5, ChatGPT 4.0, and Google Bard—were selected to answer these questions, with responses limited to 200 words. The LLMs’ responses were evaluated by a blinded group of experienced deformity surgeons (members of the European Spine Study Group) from seven European spine centers. A pre-established 4-level rating system from excellent to unsatisfactory was used with a further rating for clarity, comprehensiveness, and empathy on the 5-point Likert scale. If not rated 'excellent', the raters were asked to report the reasons for their decision for each question. Lastly, raters were asked for their opinion towards AI in healthcare in general in six questions. Results The responses among all LLMs were ‘excellent’ in 26% of responses, with ChatGPT-4.0 leading (39%), followed by Bard (17%). ChatGPT-4.0 was rated superior to Bard and ChatGPT 3.5 (p = 0.003). Discrepancies among raters were significant (p 3.0 on 5.0) and did not demonstrate any differences among LLMs. However, GPT-3.5 struggled with language suitability and empathy, while Bard’s responses were overly detailed and less empathetic. Overall, raters found that 9% of answers were off-topic and 22% contained clear mistakes. Conclusion Our study offers crucial insights into the strengths and weaknesses of current LLMs in AIS patient and parent education, highlighting the promise of advancements like ChatGPT-4.o and Gemini alongside the need for continuous improvement in empathy, contextual understanding, and language appropriateness.
Escoliosis idiopática del adolescente; Modelos de lenguaje extenso; Educación para el paciente
Escoliosi idiopàtica de l'adolescent; Models de llenguatge extens; Educació pel pacient
Adolescent idiopathic scoliosis; Large language models; Patient education
Open Access funding enabled and organized by Projekt DEAL.
Document Type: Article
File Description: application/pdf
Language: English
DOI: 10.1007/s43390-024-00955-3
Access URL: http://hdl.handle.net/11351/12820
Rights: CC BY
Accession Number: edsair.od......3991..8594d33840a8d11d10f4f2571d221729
Database: OpenAIRE
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://explore.openaire.eu/search/publication?articleId=od______3991%3A%3A8594d33840a8d11d10f4f2571d221729
    Name: EDS - OpenAIRE (s4221598)
    Category: fullText
    Text: View record at OpenAIRE
  – Url: https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=EBSCO&SrcAuth=EBSCO&DestApp=WOS&ServiceName=TransferToWoS&DestLinkType=GeneralSearchSummary&Func=Links&author=Lang%20S
    Name: ISI
    Category: fullText
    Text: Nájsť tento článok vo Web of Science
    Icon: https://imagesrvr.epnet.com/ls/20docs.gif
    MouseOverText: Nájsť tento článok vo Web of Science
Header DbId: edsair
DbLabel: OpenAIRE
An: edsair.od......3991..8594d33840a8d11d10f4f2571d221729
RelevancyScore: 965
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 964.7607421875
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Is the information provided by large language models valid in educating patients about adolescent idiopathic scoliosis? An evaluation of content, clarity, and empathy
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Lang%2C+Siegmund%22">Lang, Siegmund</searchLink><br /><searchLink fieldCode="AR" term="%22Vitale%2C+Jacopo+Antonino%22">Vitale, Jacopo Antonino</searchLink><br /><searchLink fieldCode="AR" term="%22Galbusera%2C+Fabio%22">Galbusera, Fabio</searchLink><br /><searchLink fieldCode="AR" term="%22Fekete%2C+Tamás+F%2E%22">Fekete, Tamás F.</searchLink><br /><searchLink fieldCode="AR" term="%22Charles%2C+Yann+Philippe%22">Charles, Yann Philippe</searchLink><br /><searchLink fieldCode="AR" term="%22Haddad%2C+Sleiman%22">Haddad, Sleiman</searchLink><br /><searchLink fieldCode="AR" term="%22boissiere%2C+louis%22">boissiere, louis</searchLink><br /><searchLink fieldCode="AR" term="%22Núñez-Pereira%2C+Susana%22">Núñez-Pereira, Susana</searchLink>
– Name: Author
  Label: Contributors
  Group: Au
  Data: Institut Català de la Salut<br />[Lang S] Department of Trauma Surgery, University Hospital Regensburg, Regensburg, Germany. Department of Spine Surgery, Schulthess Klinik, Zurich, Switzerland. [Vitale J, Galbusera F] Spine Center, Schulthess Klinik, Zurich, Switzerland. [Fekete T] Department of Spine Surgery, Schulthess Klinik, Zurich, Switzerland. [Boissiere L] Spine Unit Orthopaedic Department, Hôpital Pellegrin Bordeaux, Bordeaux, France. [Charles YP] Dept. of Spine Surgery, Hôpitaux Universitaires de Strasbourg, Université de Strasbourg, Strasbourg, France. [Núñez Pereira S, Haddad S] Unitat de Cirurgia de la Columna Vertebral, Vall d’Hebron Hospital Universitari, Barcelona, Spain<br />Vall d'Hebron Barcelona Hospital Campus
– Name: TitleSource
  Label: Source
  Group: Src
  Data: Scientia
– Name: Publisher
  Label: Publisher Information
  Group: PubInfo
  Data: Springer, 2025.
– Name: DatePubCY
  Label: Publication Year
  Group: Date
  Data: 2025
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Educació+dels+pacients%22">Educació dels pacients</searchLink><br /><searchLink fieldCode="DE" term="%22Metge+i+pacient%22">Metge i pacient</searchLink><br /><searchLink fieldCode="DE" term="%22PSIQUIATRÍA+Y+PSICOLOGÍA%3A%3Aconducta+y+mecanismos+de+la+conducta%3A%3Apsicología+social%3A%3Arelaciones+interpersonales%3A%3Arelaciones+profesional-paciente%3A%3Arelaciones+médico-paciente%22">PSIQUIATRÍA Y PSICOLOGÍA::conducta y mecanismos de la conducta::psicología social::relaciones interpersonales::relaciones profesional-paciente::relaciones médico-paciente</searchLink><br /><searchLink fieldCode="DE" term="%22Intel·ligència+artificial%22">Intel·ligència artificial</searchLink><br /><searchLink fieldCode="DE" term="%22Escoliosi%22">Escoliosi</searchLink><br /><searchLink fieldCode="DE" term="%22DENOMINACIONES+DE+GRUPOS%3A%3Apersonas%3A%3AGrupos+de+Edad%3A%3Aadolescente%22">DENOMINACIONES DE GRUPOS::personas::Grupos de Edad::adolescente</searchLink><br /><searchLink fieldCode="DE" term="%22DISEASES%3A%3AMusculoskeletal+Diseases%3A%3ABone+Diseases%3A%3ASpinal+Diseases%3A%3ASpinal+Curvatures%3A%3AScoliosis%22">DISEASES::Musculoskeletal Diseases::Bone Diseases::Spinal Diseases::Spinal Curvatures::Scoliosis</searchLink><br /><searchLink fieldCode="DE" term="%22FENÓMENOS+Y+PROCESOS%3A%3Aconceptos+matemáticos%3A%3Aalgoritmos%3A%3Ainteligencia+artificial%22">FENÓMENOS Y PROCESOS::conceptos matemáticos::algoritmos::inteligencia artificial</searchLink><br /><searchLink fieldCode="DE" term="%22Adolescents%22">Adolescents</searchLink><br /><searchLink fieldCode="DE" term="%22PSYCHIATRY+AND+PSYCHOLOGY%3A%3ABehavior+and+Behavior+Mechanisms%3A%3APsychology%2C+Social%3A%3AInterpersonal+Relations%3A%3AProfessional-Patient+Relations%3A%3APhysician-Patient+Relations%22">PSYCHIATRY AND PSYCHOLOGY::Behavior and Behavior Mechanisms::Psychology, Social::Interpersonal Relations::Professional-Patient Relations::Physician-Patient Relations</searchLink><br /><searchLink fieldCode="DE" term="%22NAMED+GROUPS%3A%3APersons%3A%3AAge+Groups%3A%3AAdolescent%22">NAMED GROUPS::Persons::Age Groups::Adolescent</searchLink><br /><searchLink fieldCode="DE" term="%22PHENOMENA+AND+PROCESSES%3A%3AMathematical+Concepts%3A%3AAlgorithms%3A%3AArtificial+Intelligence%22">PHENOMENA AND PROCESSES::Mathematical Concepts::Algorithms::Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22ENFERMEDADES%3A%3Aenfermedades+musculoesqueléticas%3A%3Aenfermedades+óseas%3A%3Aenfermedades+de+la+columna+vertebral%3A%3Adesviaciones+de+la+columna+vertebral%3A%3Aescoliosis%22">ENFERMEDADES::enfermedades musculoesqueléticas::enfermedades óseas::enfermedades de la columna vertebral::desviaciones de la columna vertebral::escoliosis</searchLink><br /><searchLink fieldCode="DE" term="%22HEALTH+CARE%3A%3AHealth+Care+Facilities%2C+Manpower%2C+and+Services%3A%3AHealth+Services%3A%3APreventive+Health+Services%3A%3AHealth+Education%3A%3APatient+Education+as+Topic%22">HEALTH CARE::Health Care Facilities, Manpower, and Services::Health Services::Preventive Health Services::Health Education::Patient Education as Topic</searchLink><br /><searchLink fieldCode="DE" term="%22ATENCIÓN+DE+SALUD%3A%3Ainstalaciones%2C+servicios+y+personal+de+asistencia+sanitaria%3A%3Aservicios+de+salud%3A%3Aservicios+preventivos+de+salud%3A%3Aeducación+en+salud%3A%3Aeducación+de+pacientes+como+asunto%22">ATENCIÓN DE SALUD::instalaciones, servicios y personal de asistencia sanitaria::servicios de salud::servicios preventivos de salud::educación en salud::educación de pacientes como asunto</searchLink>
– Name: Abstract
  Label: Description
  Group: Ab
  Data: Purpose Large language models (LLM) have the potential to bridge knowledge gaps in patient education and enrich patient-surgeon interactions. This study evaluated three chatbots for delivering empathetic and precise adolescent idiopathic scoliosis (AIS) related information and management advice. Specifically, we assessed the accuracy, clarity, and relevance of the information provided, aiming to determine the effectiveness of LLMs in addressing common patient queries and enhancing their understanding of AIS. Methods We sourced 20 webpages for the top frequently asked questions (FAQs) about AIS and formulated 10 critical questions based on them. Three advanced LLMs—ChatGPT 3.5, ChatGPT 4.0, and Google Bard—were selected to answer these questions, with responses limited to 200 words. The LLMs’ responses were evaluated by a blinded group of experienced deformity surgeons (members of the European Spine Study Group) from seven European spine centers. A pre-established 4-level rating system from excellent to unsatisfactory was used with a further rating for clarity, comprehensiveness, and empathy on the 5-point Likert scale. If not rated 'excellent', the raters were asked to report the reasons for their decision for each question. Lastly, raters were asked for their opinion towards AI in healthcare in general in six questions. Results The responses among all LLMs were ‘excellent’ in 26% of responses, with ChatGPT-4.0 leading (39%), followed by Bard (17%). ChatGPT-4.0 was rated superior to Bard and ChatGPT 3.5 (p = 0.003). Discrepancies among raters were significant (p 3.0 on 5.0) and did not demonstrate any differences among LLMs. However, GPT-3.5 struggled with language suitability and empathy, while Bard’s responses were overly detailed and less empathetic. Overall, raters found that 9% of answers were off-topic and 22% contained clear mistakes. Conclusion Our study offers crucial insights into the strengths and weaknesses of current LLMs in AIS patient and parent education, highlighting the promise of advancements like ChatGPT-4.o and Gemini alongside the need for continuous improvement in empathy, contextual understanding, and language appropriateness.<br />Escoliosis idiopática del adolescente; Modelos de lenguaje extenso; Educación para el paciente<br />Escoliosi idiopàtica de l'adolescent; Models de llenguatge extens; Educació pel pacient<br />Adolescent idiopathic scoliosis; Large language models; Patient education<br />Open Access funding enabled and organized by Projekt DEAL.
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Article
– Name: Format
  Label: File Description
  Group: SrcInfo
  Data: application/pdf
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s43390-024-00955-3
– Name: URL
  Label: Access URL
  Group: URL
  Data: <link linkTarget="URL" linkTerm="http://hdl.handle.net/11351/12820" linkWindow="_blank">http://hdl.handle.net/11351/12820</link>
– Name: Copyright
  Label: Rights
  Group: Cpyrght
  Data: CC BY
– Name: AN
  Label: Accession Number
  Group: ID
  Data: edsair.od......3991..8594d33840a8d11d10f4f2571d221729
PLink https://erproxy.cvtisr.sk/sfx/access?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsair&AN=edsair.od......3991..8594d33840a8d11d10f4f2571d221729
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s43390-024-00955-3
    Languages:
      – Text: English
    Subjects:
      – SubjectFull: Educació dels pacients
        Type: general
      – SubjectFull: Metge i pacient
        Type: general
      – SubjectFull: PSIQUIATRÍA Y PSICOLOGÍA::conducta y mecanismos de la conducta::psicología social::relaciones interpersonales::relaciones profesional-paciente::relaciones médico-paciente
        Type: general
      – SubjectFull: Intel·ligència artificial
        Type: general
      – SubjectFull: Escoliosi
        Type: general
      – SubjectFull: DENOMINACIONES DE GRUPOS::personas::Grupos de Edad::adolescente
        Type: general
      – SubjectFull: DISEASES::Musculoskeletal Diseases::Bone Diseases::Spinal Diseases::Spinal Curvatures::Scoliosis
        Type: general
      – SubjectFull: FENÓMENOS Y PROCESOS::conceptos matemáticos::algoritmos::inteligencia artificial
        Type: general
      – SubjectFull: Adolescents
        Type: general
      – SubjectFull: PSYCHIATRY AND PSYCHOLOGY::Behavior and Behavior Mechanisms::Psychology, Social::Interpersonal Relations::Professional-Patient Relations::Physician-Patient Relations
        Type: general
      – SubjectFull: NAMED GROUPS::Persons::Age Groups::Adolescent
        Type: general
      – SubjectFull: PHENOMENA AND PROCESSES::Mathematical Concepts::Algorithms::Artificial Intelligence
        Type: general
      – SubjectFull: ENFERMEDADES::enfermedades musculoesqueléticas::enfermedades óseas::enfermedades de la columna vertebral::desviaciones de la columna vertebral::escoliosis
        Type: general
      – SubjectFull: HEALTH CARE::Health Care Facilities, Manpower, and Services::Health Services::Preventive Health Services::Health Education::Patient Education as Topic
        Type: general
      – SubjectFull: ATENCIÓN DE SALUD::instalaciones, servicios y personal de asistencia sanitaria::servicios de salud::servicios preventivos de salud::educación en salud::educación de pacientes como asunto
        Type: general
    Titles:
      – TitleFull: Is the information provided by large language models valid in educating patients about adolescent idiopathic scoliosis? An evaluation of content, clarity, and empathy
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Lang, Siegmund
      – PersonEntity:
          Name:
            NameFull: Vitale, Jacopo Antonino
      – PersonEntity:
          Name:
            NameFull: Galbusera, Fabio
      – PersonEntity:
          Name:
            NameFull: Fekete, Tamás F.
      – PersonEntity:
          Name:
            NameFull: Charles, Yann Philippe
      – PersonEntity:
          Name:
            NameFull: Haddad, Sleiman
      – PersonEntity:
          Name:
            NameFull: boissiere, louis
      – PersonEntity:
          Name:
            NameFull: Núñez-Pereira, Susana
      – PersonEntity:
          Name:
            NameFull: Institut Català de la Salut
      – PersonEntity:
          Name:
            NameFull: [Lang S] Department of Trauma Surgery, University Hospital Regensburg, Regensburg, Germany. Department of Spine Surgery, Schulthess Klinik, Zurich, Switzerland. [Vitale J, Galbusera F] Spine Center, Schulthess Klinik, Zurich, Switzerland. [Fekete T] Department of Spine Surgery, Schulthess Klinik, Zurich, Switzerland. [Boissiere L] Spine Unit Orthopaedic Department, Hôpital Pellegrin Bordeaux, Bordeaux, France. [Charles YP] Dept. of Spine Surgery, Hôpitaux Universitaires de Strasbourg, Université de Strasbourg, Strasbourg, France. [Núñez Pereira S, Haddad S] Unitat de Cirurgia de la Columna Vertebral, Vall d’Hebron Hospital Universitari, Barcelona, Spain
      – PersonEntity:
          Name:
            NameFull: Vall d'Hebron Barcelona Hospital Campus
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 21
              M: 03
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-locals
              Value: edsair
ResultId 1