Enhancing clinicians’ trust in large language models via transparent source attribution: A randomized controlled evaluation in uro-oncology
Introduction - Large language models (LLMs) are utilized to answer queries in urology and oncology, yet the performance is limited due to outdated data and missing source transparency, which undermines clinical reliability and therefore adoption. - Material and methods - We developed UroBot, a urolo...
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
17 January 2026
|
| In: |
European journal of cancer
Year: 2026, Jahrgang: 233, Pages: 1-7 |
| ISSN: | 1879-0852 |
| DOI: | 10.1016/j.ejca.2025.116168 |
| Online-Zugang: | Verlag, kostenfrei, Volltext: https://doi.org/10.1016/j.ejca.2025.116168 Verlag, kostenfrei, Volltext: https://www.sciencedirect.com/science/article/pii/S0959804925010548 
|
| Verfasserangaben: | Nicolas Carl, Martin Joachim Hetz, Christoph Wies, Sarah Haggenmüller, Jana Theres Winterstein, Maurin Helen Mangold, Lasse Maywald, Thomas Stefan Worst, Niklas Westhoff, Maurice Stephan Michel, Frederik Wessels, Titus Josef Brinker |
MARC
| LEADER | 00000naa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1961118122 | ||
| 003 | DE-627 | ||
| 005 | 20260216105237.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 260216s2026 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1016/j.ejca.2025.116168 |2 doi | |
| 035 | |a (DE-627)1961118122 | ||
| 035 | |a (DE-599)KXP1961118122 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 33 |2 sdnb | ||
| 100 | 1 | |a Carl, Nicolas |d 1995- |e VerfasserIn |0 (DE-588)1274470323 |0 (DE-627)1824326920 |4 aut | |
| 245 | 1 | 0 | |a Enhancing clinicians’ trust in large language models via transparent source attribution |b A randomized controlled evaluation in uro-oncology |c Nicolas Carl, Martin Joachim Hetz, Christoph Wies, Sarah Haggenmüller, Jana Theres Winterstein, Maurin Helen Mangold, Lasse Maywald, Thomas Stefan Worst, Niklas Westhoff, Maurice Stephan Michel, Frederik Wessels, Titus Josef Brinker |
| 264 | 1 | |c 17 January 2026 | |
| 300 | |a 7 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Online verfügbar: 11. Dezember 2025, Artikelversion: 15. Dezember 2025 | ||
| 500 | |a Gesehen am 16.02.2026 | ||
| 520 | |a Introduction - Large language models (LLMs) are utilized to answer queries in urology and oncology, yet the performance is limited due to outdated data and missing source transparency, which undermines clinical reliability and therefore adoption. - Material and methods - We developed UroBot, a urology-specific chatbot integrating retrieval-augmented generation (RAG) to provide in-line references and source text previews for each response. In a randomized controlled reader study, UroBot and ChatGPT were compared across ten uro-oncological case rounds. Thirty urologists assessed recommendation correctness, source verifiability and trust with preference ratings collected after each round. - Results - UroBot performed significantly better than ChatGPT in recommendation correctness (73% vs. 50%; p<0.001), source attribution (74% vs. 30%; p<0.001) and verifiability of sources (84% vs. 35%; p<0.001). Furthermore, clinicians consistently preferred UroBot for accuracy, source verifiability and trust. Qualitative analysis showed that ChatGPT often produced vague or incorrect citations, with 28% being non-existent or outdated and 83% lacking specific sections, whereas UroBot achieved complete alignment on guideline sub-section and page level. These gains in citation precision were mirrored by higher clinician ratings for verifiability and trust. Limitations include the small sample size of ten cases due to feasibility, which may not cover the full uro-oncological spectrum. - Conclusion - Our findings show that combining LLMs with RAG with in-line references and source text previews markedly enhances perceived source attribution and verifiability compared to state-of-the-art conventional LLMs. Importantly, this approach is readily transferable across medical subspecialties, enabling reliable and up-to-date clinical decision support. | ||
| 650 | 4 | |a Chatbot | |
| 650 | 4 | |a Explainability | |
| 650 | 4 | |a In-line references | |
| 650 | 4 | |a Retrieval augmented generation | |
| 650 | 4 | |a Source text preview | |
| 650 | 4 | |a Verifiability | |
| 700 | 1 | |a Hetz, Martin Joachim |d 1993- |e VerfasserIn |0 (DE-588)1380936772 |0 (DE-627)1940604591 |4 aut | |
| 700 | 1 | |a Wies, Christoph |e VerfasserIn |0 (DE-588)1307730442 |0 (DE-627)1868667650 |4 aut | |
| 700 | 1 | |a Haggenmüller, Sarah |d 1995- |e VerfasserIn |0 (DE-588)1231946709 |0 (DE-627)1755618042 |4 aut | |
| 700 | 1 | |a Winterstein, Jana Therés |e VerfasserIn |0 (DE-588)1371584729 |0 (DE-627)1930549865 |4 aut | |
| 700 | 1 | |a Mangold, Maurin Helen |d 1996- |e VerfasserIn |0 (DE-588)1282415905 |0 (DE-627)1838145826 |4 aut | |
| 700 | 1 | |a Maywald, Lasse |d 1996- |e VerfasserIn |0 (DE-588)1283585405 |0 (DE-627)1839353996 |4 aut | |
| 700 | 1 | |a Worst, Thomas |d 1985- |e VerfasserIn |0 (DE-588)1028335679 |0 (DE-627)73058500X |0 (DE-576)375827501 |4 aut | |
| 700 | 1 | |a Westhoff, Niklas Christian |d 1988- |e VerfasserIn |0 (DE-588)107322306X |0 (DE-627)828672628 |0 (DE-576)434741752 |4 aut | |
| 700 | 1 | |a Michel, Maurice Stephan |d 1970- |e VerfasserIn |0 (DE-588)121842568 |0 (DE-627)705674134 |0 (DE-576)29291363X |4 aut | |
| 700 | 1 | |a Wessels, Frederik |d 1990- |e VerfasserIn |0 (DE-588)1183323255 |0 (DE-627)1663137226 |4 aut | |
| 700 | 1 | |a Brinker, Titus Josef |d 1990- |e VerfasserIn |0 (DE-588)1156309395 |0 (DE-627)1018860487 |0 (DE-576)502097434 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t European journal of cancer |d Amsterdam [u.a.] : Elsevier, 1992 |g 233(2026), Artikel-ID 116168, Seite 1-7 |w (DE-627)266883400 |w (DE-600)1468190-0 |w (DE-576)090954173 |x 1879-0852 |7 nnas |a Enhancing clinicians’ trust in large language models via transparent source attribution A randomized controlled evaluation in uro-oncology |
| 773 | 1 | 8 | |g volume:233 |g year:2026 |g elocationid:116168 |g pages:1-7 |g extent:7 |a Enhancing clinicians’ trust in large language models via transparent source attribution A randomized controlled evaluation in uro-oncology |
| 856 | 4 | 0 | |u https://doi.org/10.1016/j.ejca.2025.116168 |x Verlag |x Resolving-System |z kostenfrei |3 Volltext |7 0 |
| 856 | 4 | 0 | |u https://www.sciencedirect.com/science/article/pii/S0959804925010548 |x Verlag |z kostenfrei |3 Volltext |7 0 |
| 951 | |a AR | ||
| 992 | |a 20260216 | ||
| 993 | |a Article | ||
| 994 | |a 2026 | ||
| 998 | |g 1156309395 |a Brinker, Titus Josef |m 1156309395:Brinker, Titus Josef |d 50000 |e 50000PB1156309395 |k 0/50000/ |p 12 |y j | ||
| 998 | |g 1183323255 |a Wessels, Frederik |m 1183323255:Wessels, Frederik |d 60000 |d 63100 |d 60000 |e 60000PW1183323255 |e 63100PW1183323255 |e 60000PW1183323255 |k 0/60000/ |k 1/60000/63100/ |k 0/60000/ |p 11 | ||
| 998 | |g 121842568 |a Michel, Maurice Stephan |m 121842568:Michel, Maurice Stephan |d 60000 |d 63100 |e 60000PM121842568 |e 63100PM121842568 |k 0/60000/ |k 1/60000/63100/ |p 10 | ||
| 998 | |g 107322306X |a Westhoff, Niklas Christian |m 107322306X:Westhoff, Niklas Christian |d 60000 |d 63100 |e 60000PW107322306X |e 63100PW107322306X |k 0/60000/ |k 1/60000/63100/ |p 9 | ||
| 998 | |g 1028335679 |a Worst, Thomas |m 1028335679:Worst, Thomas |d 60000 |d 63100 |e 60000PW1028335679 |e 63100PW1028335679 |k 0/60000/ |k 1/60000/63100/ |p 8 | ||
| 998 | |g 1283585405 |a Maywald, Lasse |m 1283585405:Maywald, Lasse |d 910000 |d 950000 |d 950950 |e 910000PM1283585405 |e 950000PM1283585405 |e 950950PM1283585405 |k 0/910000/ |k 1/910000/950000/ |k 2/910000/950000/950950/ |p 7 | ||
| 998 | |g 1282415905 |a Mangold, Maurin Helen |m 1282415905:Mangold, Maurin Helen |d 60000 |e 60000PM1282415905 |k 0/60000/ |p 6 | ||
| 998 | |g 1371584729 |a Winterstein, Jana Therés |m 1371584729:Winterstein, Jana Therés |d 50000 |e 50000PW1371584729 |k 0/50000/ |p 5 | ||
| 998 | |g 1231946709 |a Haggenmüller, Sarah |m 1231946709:Haggenmüller, Sarah |d 910000 |d 950000 |d 950950 |e 910000PH1231946709 |e 950000PH1231946709 |e 950950PH1231946709 |k 0/910000/ |k 1/910000/950000/ |k 2/910000/950000/950950/ |p 4 | ||
| 998 | |g 1307730442 |a Wies, Christoph |m 1307730442:Wies, Christoph |d 50000 |e 50000PW1307730442 |k 0/50000/ |p 3 | ||
| 998 | |g 1380936772 |a Hetz, Martin Joachim |m 1380936772:Hetz, Martin Joachim |d 50000 |e 50000PH1380936772 |k 0/50000/ |p 2 | ||
| 998 | |g 1274470323 |a Carl, Nicolas |m 1274470323:Carl, Nicolas |d 60000 |e 60000PC1274470323 |k 0/60000/ |p 1 |x j | ||
| 999 | |a KXP-PPN1961118122 |e 4918154832 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"origin":[{"dateIssuedDisp":"17 January 2026","dateIssuedKey":"2026"}],"name":{"displayForm":["Nicolas Carl, Martin Joachim Hetz, Christoph Wies, Sarah Haggenmüller, Jana Theres Winterstein, Maurin Helen Mangold, Lasse Maywald, Thomas Stefan Worst, Niklas Westhoff, Maurice Stephan Michel, Frederik Wessels, Titus Josef Brinker"]},"id":{"eki":["1961118122"],"doi":["10.1016/j.ejca.2025.116168"]},"physDesc":[{"extent":"7 S."}],"type":{"bibl":"article-journal","media":"Online-Ressource"},"person":[{"role":"aut","given":"Nicolas","display":"Carl, Nicolas","family":"Carl"},{"family":"Hetz","display":"Hetz, Martin Joachim","given":"Martin Joachim","role":"aut"},{"family":"Wies","display":"Wies, Christoph","given":"Christoph","role":"aut"},{"role":"aut","given":"Sarah","display":"Haggenmüller, Sarah","family":"Haggenmüller"},{"family":"Winterstein","display":"Winterstein, Jana Therés","given":"Jana Therés","role":"aut"},{"family":"Mangold","display":"Mangold, Maurin Helen","given":"Maurin Helen","role":"aut"},{"role":"aut","given":"Lasse","display":"Maywald, Lasse","family":"Maywald"},{"display":"Worst, Thomas","family":"Worst","role":"aut","given":"Thomas"},{"display":"Westhoff, Niklas Christian","family":"Westhoff","role":"aut","given":"Niklas Christian"},{"display":"Michel, Maurice Stephan","family":"Michel","role":"aut","given":"Maurice Stephan"},{"display":"Wessels, Frederik","family":"Wessels","role":"aut","given":"Frederik"},{"family":"Brinker","display":"Brinker, Titus Josef","given":"Titus Josef","role":"aut"}],"language":["eng"],"title":[{"title":"Enhancing clinicians’ trust in large language models via transparent source attribution","subtitle":"A randomized controlled evaluation in uro-oncology","title_sort":"Enhancing clinicians’ trust in large language models via transparent source attribution"}],"note":["Online verfügbar: 11. Dezember 2025, Artikelversion: 15. Dezember 2025","Gesehen am 16.02.2026"],"recId":"1961118122","relHost":[{"disp":"Enhancing clinicians’ trust in large language models via transparent source attribution A randomized controlled evaluation in uro-oncologyEuropean journal of cancer","part":{"volume":"233","text":"233(2026), Artikel-ID 116168, Seite 1-7","year":"2026","extent":"7","pages":"1-7"},"titleAlt":[{"title":"EJC online"}],"pubHistory":["28.1992 -"],"recId":"266883400","note":["Gesehen am 21.03.24","Ungezählte Beil.: Supplement"],"title":[{"title":"European journal of cancer","title_sort":"European journal of cancer"}],"language":["eng"],"origin":[{"dateIssuedKey":"1992","publisher":"Elsevier ; Pergamon Press","dateIssuedDisp":"1992-","publisherPlace":"Amsterdam [u.a.] ; [Erscheinungsort nicht ermittelbar]"}],"type":{"bibl":"periodical","media":"Online-Ressource"},"id":{"zdb":["1468190-0"],"eki":["266883400"],"issn":["1879-0852"]},"corporate":[{"role":"isb","display":"European Organization for Research on Treatment of Cancer"},{"display":"European Association for Cancer Research","role":"isb"},{"role":"isb","display":"European School of Oncology"}]}]} | ||
| SRT | |a CARLNICOLAENHANCINGC1720 | ||