Quality of Information About Kidney Stones from Artificial Intelligence Chatbots
Kidney stones are common and morbid conditions in the general population with a rising incidence globally. Prior studies show substantial limitations of online sources of information regarding prevention and treatment. The objective of this study was to examine the quality of information about kidne...
Saved in:
| Published in: | Journal of endourology Vol. 38; no. 10; p. 1056 |
|---|---|
| Main Authors: | , , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
United States
01.10.2024
|
| ISSN: | 1557-900X, 1557-900X |
| Online Access: | Get more information |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Kidney stones are common and morbid conditions in the general population with a rising incidence globally. Prior studies show substantial limitations of online sources of information regarding prevention and treatment. The objective of this study was to examine the quality of information about kidney stones from artificial intelligence (AI) chatbots.
The most common online searches about kidney stones from Google Trends and headers from the National Institute of Diabetes and Digestive and Kidney Diseases website were used as inputs to 4 AI chatbots (ChatGPT version 3.5, Perplexity, Chat Sonic, and Bing AI). Validated instruments were used to assess the quality (DISCERN instrument from 1 low to 5 high), understandability and actionability (PEMAT, from 0 to 100%) of the chatbot outputs. In addition, we examined the reading level of the information and whether there was misinformation compared to guidelines (5 point Likert scale).
AI chatbots generally provide high-quality consumer health information (median DISCERN 4 out of 5) and did not include misinformation (median 1 out of 5). The median understandability was moderate (median 69.6%) and actionability was moderate to poor (median 40%). Responses were presented at an advanced reading level (11th grade; median Flesch-Kincaid score 11.3).
AI chatbots provide generally accurate information on kidney stones and lack misinformation; however, it is not easily actionable and is presented above the recommended reading level for consumer health information. |
|---|---|
| AbstractList | Kidney stones are common and morbid conditions in the general population with a rising incidence globally. Prior studies show substantial limitations of online sources of information regarding prevention and treatment. The objective of this study was to examine the quality of information about kidney stones from artificial intelligence (AI) chatbots.
The most common online searches about kidney stones from Google Trends and headers from the National Institute of Diabetes and Digestive and Kidney Diseases website were used as inputs to 4 AI chatbots (ChatGPT version 3.5, Perplexity, Chat Sonic, and Bing AI). Validated instruments were used to assess the quality (DISCERN instrument from 1 low to 5 high), understandability and actionability (PEMAT, from 0 to 100%) of the chatbot outputs. In addition, we examined the reading level of the information and whether there was misinformation compared to guidelines (5 point Likert scale).
AI chatbots generally provide high-quality consumer health information (median DISCERN 4 out of 5) and did not include misinformation (median 1 out of 5). The median understandability was moderate (median 69.6%) and actionability was moderate to poor (median 40%). Responses were presented at an advanced reading level (11th grade; median Flesch-Kincaid score 11.3).
AI chatbots provide generally accurate information on kidney stones and lack misinformation; however, it is not easily actionable and is presented above the recommended reading level for consumer health information. Introduction: Kidney stones are common and morbid conditions in the general population with a rising incidence globally. Previous studies show substantial limitations of online sources of information regarding prevention and treatment. The objective of this study was to examine the quality of information on kidney stones from artificial intelligence (AI) chatbots. Methods: The most common online searches about kidney stones from Google Trends and headers from the National Institute of Diabetes and Digestive and Kidney Diseases website were used as inputs to four AI chatbots (ChatGPT version 3.5, Perplexity, Chat Sonic, and Bing AI). Validated instruments were used to assess the quality (DISCERN instrument from 1 low to 5 high), understandability, and actionability (PEMAT, from 0% to 100%) of the chatbot outputs. In addition, we examined the reading level of the information and whether there was misinformation compared with guidelines (5 point Likert scale). Results: AI chatbots generally provided high-quality consumer health information (median DISCERN 4 out of 5) and did not include misinformation (median 1 out of 5). The median understandability was moderate (median 69.6%), and actionability was moderate to poor (median 40%). Responses were presented at an advanced reading level (11th grade; median Flesch-Kincaid score 11.3). Conclusions: AI chatbots provide generally accurate information on kidney stones and lack misinformation; however, it is not easily actionable and is presented above the recommended reading level for consumer health information.Introduction: Kidney stones are common and morbid conditions in the general population with a rising incidence globally. Previous studies show substantial limitations of online sources of information regarding prevention and treatment. The objective of this study was to examine the quality of information on kidney stones from artificial intelligence (AI) chatbots. Methods: The most common online searches about kidney stones from Google Trends and headers from the National Institute of Diabetes and Digestive and Kidney Diseases website were used as inputs to four AI chatbots (ChatGPT version 3.5, Perplexity, Chat Sonic, and Bing AI). Validated instruments were used to assess the quality (DISCERN instrument from 1 low to 5 high), understandability, and actionability (PEMAT, from 0% to 100%) of the chatbot outputs. In addition, we examined the reading level of the information and whether there was misinformation compared with guidelines (5 point Likert scale). Results: AI chatbots generally provided high-quality consumer health information (median DISCERN 4 out of 5) and did not include misinformation (median 1 out of 5). The median understandability was moderate (median 69.6%), and actionability was moderate to poor (median 40%). Responses were presented at an advanced reading level (11th grade; median Flesch-Kincaid score 11.3). Conclusions: AI chatbots provide generally accurate information on kidney stones and lack misinformation; however, it is not easily actionable and is presented above the recommended reading level for consumer health information. |
| Author | Kabarriti, Abdo E Borin, James Musheyev, David Loeb, Stacy Pan, Alexander |
| Author_xml | – sequence: 1 givenname: David surname: Musheyev fullname: Musheyev, David email: david.musheyev@downstate.edu organization: SUNY Downstate Health Sciences University, Urology, 450 Clarkson Avenue, Brooklyn, New York, United States, 11203-2098; david.musheyev@downstate.edu – sequence: 2 givenname: Alexander surname: Pan fullname: Pan, Alexander email: alexander.pan@downstate.edu organization: SUNY Downstate Health Sciences University, Urology, Brooklyn, New York, United States; alexander.pan@downstate.edu – sequence: 3 givenname: Abdo E surname: Kabarriti fullname: Kabarriti, Abdo E email: abdo.kabarriti@downstate.edu organization: SUNY Downstate Health Sciences University, Urology, Brooklyn, New York, United States; abdo.kabarriti@downstate.edu – sequence: 4 givenname: Stacy surname: Loeb fullname: Loeb, Stacy email: stacyloeb@gmail.com organization: New York University Grossman School of Medicine, 227 E 30th Street #612, NY, New York, United States, 10016; stacyloeb@gmail.com – sequence: 5 givenname: James surname: Borin fullname: Borin, James email: james.borin@nyulangone.org organization: New York, United States; james.borin@nyulangone.org |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/39001821$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNkM1LxDAQxYOsuB969Co5eumaj6ZNj8vi6uKCigreSppMNdIma5Me9r834Aqe5jH85s28maOJ8w4QuqRkSYmsbsCZJSOML0ku8xM0o0KUWUXI--SfnqJ5CF-EUF5QfoamPDWpZHSGnp5H1dl4wL7FW9f6oVfReodXjR8jfrDGwQG_xLQz4HbwPV4N0bZWW9UlPkLX2Q9wGvD6U8XGx3COTlvVBbg41gV629y-ru-z3ePddr3aZZqXLGaFaJqSt1JVkFNNpQFptChIXhgGvBXa5LwsiJaN0II1SgpJUkyTCwOQZtgCXf_67gf_PUKIdW-DTvcoB34MNSdlVaX8BUno1REdmx5MvR9sr4ZD_fcF9gNf7WCw |
| CitedBy_id | crossref_primary_10_1016_j_ddj_2025_100008 crossref_primary_10_1097_MOU_0000000000001267 crossref_primary_10_1007_s00240_025_01761_w crossref_primary_10_1038_s41585_024_00928_1 crossref_primary_10_1016_j_urology_2025_01_054 crossref_primary_10_3390_jcm14144942 crossref_primary_10_1016_j_jbmt_2025_05_043 |
| ContentType | Journal Article |
| DBID | NPM 7X8 |
| DOI | 10.1089/end.2023.0484 |
| DatabaseName | PubMed MEDLINE - Academic |
| DatabaseTitle | PubMed MEDLINE - Academic |
| DatabaseTitleList | PubMed MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Medicine |
| EISSN | 1557-900X |
| ExternalDocumentID | 39001821 |
| Genre | Journal Article |
| GroupedDBID | --- 0R~ 29K 4.4 53G 5GY 5RE ABBKN ACGFS ADBBV AENEX ALMA_UNASSIGNED_HOLDINGS BNQNF CS3 D-I DU5 EBS F5P IAO IHR IM4 MV1 NPM NQHIM O9- P2P RML UE5 7X8 EMOBN SCNPE |
| ID | FETCH-LOGICAL-c372t-65bb73f8a9e41c18de8dc56046d2e3f5cd43760c8b5c52ba8580202d45dee9e42 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 9 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001278901000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1557-900X |
| IngestDate | Thu Sep 04 19:28:33 EDT 2025 Thu Apr 03 07:08:13 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 10 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c372t-65bb73f8a9e41c18de8dc56046d2e3f5cd43760c8b5c52ba8580202d45dee9e42 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| PMID | 39001821 |
| PQID | 3079955760 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_3079955760 pubmed_primary_39001821 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-10-01 |
| PublicationDateYYYYMMDD | 2024-10-01 |
| PublicationDate_xml | – month: 10 year: 2024 text: 2024-10-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | Journal of endourology |
| PublicationTitleAlternate | J Endourol |
| PublicationYear | 2024 |
| SSID | ssj0013613 |
| Score | 2.4585056 |
| Snippet | Kidney stones are common and morbid conditions in the general population with a rising incidence globally. Prior studies show substantial limitations of online... Introduction: Kidney stones are common and morbid conditions in the general population with a rising incidence globally. Previous studies show substantial... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | 1056 |
| Title | Quality of Information About Kidney Stones from Artificial Intelligence Chatbots |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/39001821 https://www.proquest.com/docview/3079955760 |
| Volume | 38 |
| WOSCitedRecordID | wos001278901000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1NS8MwGA7qRLz4_TG_iOA1Lm2SJj2JDIeiGzuo9DbaJMVd2mmrsH_vmzbDXQTBS09NeXnzNHmSN3kehK6sTKmWlBJgG5YAKAQBUu8WroGG9UQUqcYl4vVJjkYqSeKx33Cr_LHKxZjYDNSm1G6PvAdYjGMB7JjezN6Jc41y1VVvobGKOgyojEO1TJaqCFFjjwxTpiQxpYnX2KQq7tnCyYSG7BoQzH9nl80sM9j-b3w7aMvzS3zbAmIXrdhiD20MfQV9H41b0Yw5LnPsryK5rsGu_FPjx6kp7Bw3Ct0VdldPmi-1KhP4YUm-E_ff0jor6-oAvQzunvv3xLsqEM1kWJNIZJlkuUpjywMdKGOV0cB7eGRCy3KhDXcHZbTKhBZhliqhgFKGhgtjLbQJD9FaAVEcI2xYCi8yRU0GxAD62EY2lkJzSHxmed5Fl4tcTQC1rhSRFrb8rCY_2eqiozbhk1krrzFhsXMKDIOTP7Q-RZsQG29P152hTg7_rD1H6_qrnlYfFw0c4DkaD78BrmO-TQ |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Quality+of+Information+About+Kidney+Stones+from+Artificial+Intelligence+Chatbots&rft.jtitle=Journal+of+endourology&rft.au=Musheyev%2C+David&rft.au=Pan%2C+Alexander&rft.au=Kabarriti%2C+Abdo+E&rft.au=Loeb%2C+Stacy&rft.date=2024-10-01&rft.issn=1557-900X&rft.eissn=1557-900X&rft.volume=38&rft.issue=10&rft.spage=1056&rft_id=info:doi/10.1089%2Fend.2023.0484&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1557-900X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1557-900X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1557-900X&client=summon |