Evaluation of four chatbots in autoimmune liver disease: a comparative analysis
Introduction and Objectives - Autoimmune liver diseases (AILDs) are rare and require precise evaluation, which is often challenging for medical providers. Chatbots are innovative solutions to assist healthcare professionals in clinical management. In our study, ten liver specialists systematically e...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
January/February 2025
|
| In: |
Annals of hepatology
Year: 2025, Volume: 30, Issue: 1, Pages: 1-7 |
| DOI: | 10.1016/j.aohep.2024.101537 |
| Online Access: | Verlag, kostenfrei, Volltext: https://doi.org/10.1016/j.aohep.2024.101537 Verlag, kostenfrei, Volltext: https://www.sciencedirect.com/science/article/pii/S1665268124003314 
|
| Author Notes: | Jimmy Daza, Lucas Soares Bezerra, Laura Santamaría, Roberto Rueda-Esteban, Heike Bantel, Marcos Girala, Matthias Ebert, Florian Van Bömmel, Andreas Geier, Andres Gomez Aldana, Kevin Yau, Mario Alvares-da-Silva, Markus Peck-Radosavljevic, Ezequiel Ridruejo, Arndt Weinmann, Andreas Teufel |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1917883641 | ||
| 003 | DE-627 | ||
| 005 | 20250716233353.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 250224s2025 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1016/j.aohep.2024.101537 |2 doi | |
| 035 | |a (DE-627)1917883641 | ||
| 035 | |a (DE-599)KXP1917883641 | ||
| 035 | |a (OCoLC)1528019569 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 33 |2 sdnb | ||
| 100 | 1 | |a Daza Barragán, Jimmy Andres |d 1990- |e VerfasserIn |0 (DE-588)1237697921 |0 (DE-627)176462193X |4 aut | |
| 245 | 1 | 0 | |a Evaluation of four chatbots in autoimmune liver disease |b a comparative analysis |c Jimmy Daza, Lucas Soares Bezerra, Laura Santamaría, Roberto Rueda-Esteban, Heike Bantel, Marcos Girala, Matthias Ebert, Florian Van Bömmel, Andreas Geier, Andres Gomez Aldana, Kevin Yau, Mario Alvares-da-Silva, Markus Peck-Radosavljevic, Ezequiel Ridruejo, Arndt Weinmann, Andreas Teufel |
| 264 | 1 | |c January/February 2025 | |
| 300 | |b Illustrationen | ||
| 300 | |a 7 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Online verfügbar: 13. August 2024, Artikelversion: 5. September 2024 | ||
| 500 | |a Gesehen am 24.02.2025 | ||
| 520 | |a Introduction and Objectives - Autoimmune liver diseases (AILDs) are rare and require precise evaluation, which is often challenging for medical providers. Chatbots are innovative solutions to assist healthcare professionals in clinical management. In our study, ten liver specialists systematically evaluated four chatbots to determine their utility as clinical decision support tools in the field of AILDs. - Materials and Methods - We constructed a 56-question questionnaire focusing on AILD evaluation, diagnosis, and management of Autoimmune Hepatitis (AIH), Primary Biliary Cholangitis (PBC), and Primary Sclerosing Cholangitis (PSC). Four chatbots -ChatGPT 3.5, Claude, Microsoft Copilot, and Google Bard- were presented with the questions in their free tiers in December 2023. Responses underwent critical evaluation by ten liver specialists using a standardized 1 to 10 Likert scale. The analysis included mean scores, the number of highest-rated replies, and the identification of common shortcomings in chatbots performance. - Results - Among the assessed chatbots, specialists rated Claude highest with a mean score of 7.37 (SD = 1.91), followed by ChatGPT (7.17, SD = 1.89), Microsoft Copilot (6.63, SD = 2.10), and Google Bard (6.52, SD = 2.27). Claude also excelled with 27 best-rated replies, outperforming ChatGPT (20), while Microsoft Copilot and Google Bard lagged with only 6 and 9, respectively. Common deficiencies included listing details over specific advice, limited dosing options, inaccuracies for pregnant patients, insufficient recent data, over-reliance on CT and MRI imaging, and inadequate discussion regarding off-label use and fibrates in PBC treatment. Notably, internet access for Microsoft Copilot and Google Bard did not enhance precision compared to pre-trained models. - Conclusions - Chatbots hold promise in AILD support, but our study underscores key areas for improvement. Refinement is needed in providing specific advice, accuracy, and focused up-to-date information. Addressing these shortcomings is essential for enhancing the utility of chatbots in AILD management, guiding future development, and ensuring their effectiveness as clinical decision-support tools. | ||
| 650 | 4 | |a Artificial intelligence | |
| 650 | 4 | |a Autoimmune liver disease | |
| 650 | 4 | |a Chatbots | |
| 650 | 4 | |a Clinical decision support tools | |
| 700 | 1 | |a Bezerra, Lucas Soares |e VerfasserIn |0 (DE-588)1357417829 |0 (DE-627)1917884184 |4 aut | |
| 700 | 1 | |a Santamaría, Laura |e VerfasserIn |4 aut | |
| 700 | 1 | |a Rueda-Esteban, Roberto |e VerfasserIn |4 aut | |
| 700 | 1 | |a Bantel, Heike |e VerfasserIn |4 aut | |
| 700 | 1 | |a Girala, Marcos |e VerfasserIn |4 aut | |
| 700 | 1 | |a Ebert, Matthias |d 1968- |e VerfasserIn |0 (DE-588)1030133522 |0 (DE-627)734827083 |0 (DE-576)377938432 |4 aut | |
| 700 | 1 | |a Van Bömmel, Florian |e VerfasserIn |4 aut | |
| 700 | 1 | |a Geier, Andreas |e VerfasserIn |4 aut | |
| 700 | 1 | |a Aldana, Andres Gomez |e VerfasserIn |4 aut | |
| 700 | 1 | |a Yau, Kevin |e VerfasserIn |4 aut | |
| 700 | 1 | |a Alvares-da-Silva, Mario |e VerfasserIn |4 aut | |
| 700 | 1 | |a Peck-Radosavljevic, Markus |e VerfasserIn |4 aut | |
| 700 | 1 | |a Ridruejo, Ezequiel |e VerfasserIn |4 aut | |
| 700 | 1 | |a Weinmann, Arndt |e VerfasserIn |4 aut | |
| 700 | 1 | |a Teufel, Andreas |d 1971- |e VerfasserIn |0 (DE-588)121120031 |0 (DE-627)081100035 |0 (DE-576)292546750 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Annals of hepatology |d México : Assoc., 2002 |g 30(2025), 1 vom: Jan./Feb., Artikel-ID 101537, Seite 1-7 |h Online-Ressource |w (DE-627)374600961 |w (DE-600)2128499-4 |w (DE-576)343623919 |7 nnas |a Evaluation of four chatbots in autoimmune liver disease a comparative analysis |
| 773 | 1 | 8 | |g volume:30 |g year:2025 |g number:1 |g month:01/02 |g elocationid:101537 |g pages:1-7 |g extent:7 |a Evaluation of four chatbots in autoimmune liver disease a comparative analysis |
| 856 | 4 | 0 | |u https://doi.org/10.1016/j.aohep.2024.101537 |x Verlag |x Resolving-System |z kostenfrei |3 Volltext |
| 856 | 4 | 0 | |u https://www.sciencedirect.com/science/article/pii/S1665268124003314 |x Verlag |z kostenfrei |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20250224 | ||
| 993 | |a Article | ||
| 994 | |a 2025 | ||
| 998 | |g 121120031 |a Teufel, Andreas |m 121120031:Teufel, Andreas |d 60000 |d 61100 |e 60000PT121120031 |e 61100PT121120031 |k 0/60000/ |k 1/60000/61100/ |p 16 |y j | ||
| 998 | |g 1030133522 |a Ebert, Matthias |m 1030133522:Ebert, Matthias |d 60000 |d 61100 |e 60000PE1030133522 |e 61100PE1030133522 |k 0/60000/ |k 1/60000/61100/ |p 7 | ||
| 998 | |g 1357417829 |a Soares Bezerra, Lucas |m 1357417829:Soares Bezerra, Lucas |p 2 | ||
| 998 | |g 1237697921 |a Daza Barragán, Jimmy Andres |m 1237697921:Daza Barragán, Jimmy Andres |d 60000 |d 61100 |e 60000PD1237697921 |e 61100PD1237697921 |k 0/60000/ |k 1/60000/61100/ |p 1 |x j | ||
| 999 | |a KXP-PPN1917883641 |e 4670038257 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"relHost":[{"note":["Gesehen am 24.11.2021"],"type":{"media":"Online-Ressource","bibl":"periodical"},"pubHistory":["1.2002 -"],"language":["eng"],"physDesc":[{"extent":"Online-Ressource"}],"title":[{"title_sort":"Annals of hepatology","subtitle":"the official journal of the Mexican Association of Hepatology","title":"Annals of hepatology"}],"recId":"374600961","disp":"Evaluation of four chatbots in autoimmune liver disease a comparative analysisAnnals of hepatology","origin":[{"dateIssuedDisp":"2002-","dateIssuedKey":"2002","publisher":"Assoc.","publisherPlace":"México"}],"id":{"eki":["374600961"],"zdb":["2128499-4"]},"part":{"extent":"7","text":"30(2025), 1 vom: Jan./Feb., Artikel-ID 101537, Seite 1-7","pages":"1-7","volume":"30","issue":"1","year":"2025"}}],"note":["Online verfügbar: 13. August 2024, Artikelversion: 5. September 2024","Gesehen am 24.02.2025"],"recId":"1917883641","name":{"displayForm":["Jimmy Daza, Lucas Soares Bezerra, Laura Santamaría, Roberto Rueda-Esteban, Heike Bantel, Marcos Girala, Matthias Ebert, Florian Van Bömmel, Andreas Geier, Andres Gomez Aldana, Kevin Yau, Mario Alvares-da-Silva, Markus Peck-Radosavljevic, Ezequiel Ridruejo, Arndt Weinmann, Andreas Teufel"]},"origin":[{"dateIssuedDisp":"January/February 2025","dateIssuedKey":"2025"}],"id":{"doi":["10.1016/j.aohep.2024.101537"],"eki":["1917883641"]},"type":{"media":"Online-Ressource","bibl":"article-journal"},"physDesc":[{"noteIll":"Illustrationen","extent":"7 S."}],"language":["eng"],"title":[{"title_sort":"Evaluation of four chatbots in autoimmune liver disease","title":"Evaluation of four chatbots in autoimmune liver disease","subtitle":"a comparative analysis"}],"person":[{"display":"Daza Barragán, Jimmy Andres","role":"aut","given":"Jimmy Andres","family":"Daza Barragán"},{"role":"aut","given":"Lucas Soares","family":"Bezerra","display":"Bezerra, Lucas Soares"},{"family":"Santamaría","given":"Laura","role":"aut","display":"Santamaría, Laura"},{"display":"Rueda-Esteban, Roberto","given":"Roberto","role":"aut","family":"Rueda-Esteban"},{"display":"Bantel, Heike","role":"aut","given":"Heike","family":"Bantel"},{"family":"Girala","role":"aut","given":"Marcos","display":"Girala, Marcos"},{"given":"Matthias","role":"aut","family":"Ebert","display":"Ebert, Matthias"},{"given":"Florian","role":"aut","family":"Van Bömmel","display":"Van Bömmel, Florian"},{"family":"Geier","given":"Andreas","role":"aut","display":"Geier, Andreas"},{"display":"Aldana, Andres Gomez","family":"Aldana","given":"Andres Gomez","role":"aut"},{"display":"Yau, Kevin","family":"Yau","given":"Kevin","role":"aut"},{"display":"Alvares-da-Silva, Mario","family":"Alvares-da-Silva","role":"aut","given":"Mario"},{"role":"aut","given":"Markus","family":"Peck-Radosavljevic","display":"Peck-Radosavljevic, Markus"},{"given":"Ezequiel","role":"aut","family":"Ridruejo","display":"Ridruejo, Ezequiel"},{"display":"Weinmann, Arndt","given":"Arndt","role":"aut","family":"Weinmann"},{"display":"Teufel, Andreas","given":"Andreas","role":"aut","family":"Teufel"}]} | ||
| SRT | |a DAZABARRAGEVALUATION2025 | ||