Medical transformer for multimodal survival prediction in intensive care: integration of imaging and non-imaging data
When clinicians assess the prognosis of patients in intensive care, they take imaging and non-imaging data into account. In contrast, many traditional machine learning models rely on only one of these modalities, limiting their potential in medical applications. This work proposes and evaluates a tr...
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
2023
|
| In: |
Scientific reports
Year: 2023, Jahrgang: 13, Pages: 1-11 |
| ISSN: | 2045-2322 |
| DOI: | 10.1038/s41598-023-37835-1 |
| Online-Zugang: | Verlag, kostenfrei, Volltext: https://doi.org/10.1038/s41598-023-37835-1 Verlag, kostenfrei, Volltext: https://www.nature.com/articles/s41598-023-37835-1 
|
| Verfasserangaben: | Firas Khader, Jakob Nikolas Kather, Gustav Müller-Franzes, Tianci Wang, Tianyu Han, Soroosh Tayebi Arasteh, Karim Hamesch, Keno Bressem, Christoph Haarburger, Johannes Stegmaier, Christiane Kuhl, Sven Nebelung & Daniel Truhn |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1858255724 | ||
| 003 | DE-627 | ||
| 005 | 20250113233244.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 230830s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1038/s41598-023-37835-1 |2 doi | |
| 035 | |a (DE-627)1858255724 | ||
| 035 | |a (DE-599)KXP1858255724 | ||
| 035 | |a (OCoLC)1425211010 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 33 |2 sdnb | ||
| 100 | 1 | |a Khader, Firas |d 1996- |e VerfasserIn |0 (DE-588)1298294681 |0 (DE-627)1854342223 |4 aut | |
| 245 | 1 | 0 | |a Medical transformer for multimodal survival prediction in intensive care |b integration of imaging and non-imaging data |c Firas Khader, Jakob Nikolas Kather, Gustav Müller-Franzes, Tianci Wang, Tianyu Han, Soroosh Tayebi Arasteh, Karim Hamesch, Keno Bressem, Christoph Haarburger, Johannes Stegmaier, Christiane Kuhl, Sven Nebelung & Daniel Truhn |
| 264 | 1 | |c 2023 | |
| 300 | |b Illustrationen | ||
| 300 | |a 11 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Veröffentlicht: 01. Juli 2023 | ||
| 500 | |a Gesehen am 30.08.2023 | ||
| 520 | |a When clinicians assess the prognosis of patients in intensive care, they take imaging and non-imaging data into account. In contrast, many traditional machine learning models rely on only one of these modalities, limiting their potential in medical applications. This work proposes and evaluates a transformer-based neural network as a novel AI architecture that integrates multimodal patient data, i.e., imaging data (chest radiographs) and non-imaging data (clinical data). We evaluate the performance of our model in a retrospective study with 6,125 patients in intensive care. We show that the combined model (area under the receiver operating characteristic curve [AUROC] of 0.863) is superior to the radiographs-only model (AUROC = 0.811, p < 0.001) and the clinical data-only model (AUROC = 0.785, p < 0.001) when tasked with predicting in-hospital survival per patient. Furthermore, we demonstrate that our proposed model is robust in cases where not all (clinical) data points are available. | ||
| 650 | 4 | |a Computer science | |
| 650 | 4 | |a Medical imaging | |
| 650 | 4 | |a Radiography | |
| 700 | 1 | |a Kather, Jakob Nikolas |d 1989- |e VerfasserIn |0 (DE-588)1064064914 |0 (DE-627)812897587 |0 (DE-576)423589091 |4 aut | |
| 700 | 1 | |a Müller-Franzes, Gustav |e VerfasserIn |4 aut | |
| 700 | 1 | |a Wang, Tianci |e VerfasserIn |4 aut | |
| 700 | 1 | |a Han, Tianyu |e VerfasserIn |4 aut | |
| 700 | 1 | |a Tayebi Arasteh, Soroosh |e VerfasserIn |4 aut | |
| 700 | 1 | |a Hamesch, Karim |e VerfasserIn |4 aut | |
| 700 | 1 | |a Bressem, Keno |e VerfasserIn |4 aut | |
| 700 | 1 | |a Haarburger, Christoph |e VerfasserIn |4 aut | |
| 700 | 1 | |8 1\p |a Stegmaier, Johannes |d 1985- |e VerfasserIn |0 (DE-588)1111064075 |0 (DE-627)865478260 |0 (DE-576)475912616 |4 aut | |
| 700 | 1 | |a Kuhl, Christiane |e VerfasserIn |4 aut | |
| 700 | 1 | |a Nebelung, Sven |e VerfasserIn |4 aut | |
| 700 | 1 | |a Truhn, Daniel |e VerfasserIn |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Scientific reports |d [London] : Springer Nature, 2011 |g 13(2023), Artikel-ID 10666, Seite 1-11 |h Online-Ressource |w (DE-627)663366712 |w (DE-600)2615211-3 |w (DE-576)346641179 |x 2045-2322 |7 nnas |a Medical transformer for multimodal survival prediction in intensive care integration of imaging and non-imaging data |
| 773 | 1 | 8 | |g volume:13 |g year:2023 |g elocationid:10666 |g pages:1-11 |g extent:11 |a Medical transformer for multimodal survival prediction in intensive care integration of imaging and non-imaging data |
| 856 | 4 | 0 | |u https://doi.org/10.1038/s41598-023-37835-1 |x Verlag |x Resolving-System |z kostenfrei |3 Volltext |
| 856 | 4 | 0 | |u https://www.nature.com/articles/s41598-023-37835-1 |x Verlag |z kostenfrei |3 Volltext |
| 883 | |8 1\p |a cgwrk |d 20241001 |q DE-101 |u https://d-nb.info/provenance/plan#cgwrk | ||
| 951 | |a AR | ||
| 992 | |a 20230830 | ||
| 993 | |a Article | ||
| 994 | |a 2023 | ||
| 998 | |g 1064064914 |a Kather, Jakob Nikolas |m 1064064914:Kather, Jakob Nikolas |d 910000 |d 910100 |e 910000PK1064064914 |e 910100PK1064064914 |k 0/910000/ |k 1/910000/910100/ |p 2 | ||
| 999 | |a KXP-PPN1858255724 |e 4371069355 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"name":{"displayForm":["Firas Khader, Jakob Nikolas Kather, Gustav Müller-Franzes, Tianci Wang, Tianyu Han, Soroosh Tayebi Arasteh, Karim Hamesch, Keno Bressem, Christoph Haarburger, Johannes Stegmaier, Christiane Kuhl, Sven Nebelung & Daniel Truhn"]},"id":{"eki":["1858255724"],"doi":["10.1038/s41598-023-37835-1"]},"recId":"1858255724","physDesc":[{"extent":"11 S.","noteIll":"Illustrationen"}],"origin":[{"dateIssuedKey":"2023","dateIssuedDisp":"2023"}],"person":[{"display":"Khader, Firas","role":"aut","given":"Firas","family":"Khader"},{"role":"aut","given":"Jakob Nikolas","family":"Kather","display":"Kather, Jakob Nikolas"},{"family":"Müller-Franzes","role":"aut","given":"Gustav","display":"Müller-Franzes, Gustav"},{"family":"Wang","given":"Tianci","role":"aut","display":"Wang, Tianci"},{"given":"Tianyu","role":"aut","family":"Han","display":"Han, Tianyu"},{"display":"Tayebi Arasteh, Soroosh","family":"Tayebi Arasteh","given":"Soroosh","role":"aut"},{"display":"Hamesch, Karim","given":"Karim","role":"aut","family":"Hamesch"},{"family":"Bressem","role":"aut","given":"Keno","display":"Bressem, Keno"},{"display":"Haarburger, Christoph","family":"Haarburger","given":"Christoph","role":"aut"},{"role":"aut","given":"Johannes","family":"Stegmaier","display":"Stegmaier, Johannes"},{"given":"Christiane","role":"aut","family":"Kuhl","display":"Kuhl, Christiane"},{"display":"Nebelung, Sven","family":"Nebelung","role":"aut","given":"Sven"},{"family":"Truhn","role":"aut","given":"Daniel","display":"Truhn, Daniel"}],"relHost":[{"id":{"zdb":["2615211-3"],"eki":["663366712"],"issn":["2045-2322"]},"part":{"pages":"1-11","year":"2023","text":"13(2023), Artikel-ID 10666, Seite 1-11","volume":"13","extent":"11"},"title":[{"title":"Scientific reports","title_sort":"Scientific reports"}],"type":{"bibl":"periodical","media":"Online-Ressource"},"language":["eng"],"note":["Gesehen am 12.07.24"],"origin":[{"publisher":"Springer Nature ; Nature Publishing Group","dateIssuedKey":"2011","publisherPlace":"[London] ; London","dateIssuedDisp":"2011-"}],"pubHistory":["1, article number 1 (2011)-"],"disp":"Medical transformer for multimodal survival prediction in intensive care integration of imaging and non-imaging dataScientific reports","recId":"663366712","physDesc":[{"extent":"Online-Ressource"}]}],"title":[{"title":"Medical transformer for multimodal survival prediction in intensive care","title_sort":"Medical transformer for multimodal survival prediction in intensive care","subtitle":"integration of imaging and non-imaging data"}],"type":{"media":"Online-Ressource","bibl":"article-journal"},"language":["eng"],"note":["Veröffentlicht: 01. Juli 2023","Gesehen am 30.08.2023"]} | ||
| SRT | |a KHADERFIRAMEDICALTRA2023 | ||