Cover Image

Medical transformer for multimodal survival prediction in intensive care: integration of imaging and non-imaging data

When clinicians assess the prognosis of patients in intensive care, they take imaging and non-imaging data into account. In contrast, many traditional machine learning models rely on only one of these modalities, limiting their potential in medical applications. This work proposes and evaluates a tr...

Full description

Saved in:
Bibliographic Details
Main Authors: Khader, Firas (Author) , Kather, Jakob Nikolas (Author) , Müller-Franzes, Gustav (Author) , Wang, Tianci (Author) , Han, Tianyu (Author) , Tayebi Arasteh, Soroosh (Author) , Hamesch, Karim (Author) , Bressem, Keno (Author) , Haarburger, Christoph (Author) , Stegmaier, Johannes (Author) , Kuhl, Christiane (Author) , Nebelung, Sven (Author) , Truhn, Daniel (Author)
Format: Article (Journal)
Language:English
Published: 2023
In: Scientific reports
Year: 2023, Volume: 13, Pages: 1-11
ISSN:2045-2322
DOI:10.1038/s41598-023-37835-1
Online Access:Verlag, kostenfrei, Volltext: https://doi.org/10.1038/s41598-023-37835-1
Verlag, kostenfrei, Volltext: https://www.nature.com/articles/s41598-023-37835-1
Get full text
Author Notes:Firas Khader, Jakob Nikolas Kather, Gustav Müller-Franzes, Tianci Wang, Tianyu Han, Soroosh Tayebi Arasteh, Karim Hamesch, Keno Bressem, Christoph Haarburger, Johannes Stegmaier, Christiane Kuhl, Sven Nebelung & Daniel Truhn
Description
Summary:When clinicians assess the prognosis of patients in intensive care, they take imaging and non-imaging data into account. In contrast, many traditional machine learning models rely on only one of these modalities, limiting their potential in medical applications. This work proposes and evaluates a transformer-based neural network as a novel AI architecture that integrates multimodal patient data, i.e., imaging data (chest radiographs) and non-imaging data (clinical data). We evaluate the performance of our model in a retrospective study with 6,125 patients in intensive care. We show that the combined model (area under the receiver operating characteristic curve [AUROC] of 0.863) is superior to the radiographs-only model (AUROC = 0.811, p < 0.001) and the clinical data-only model (AUROC = 0.785, p < 0.001) when tasked with predicting in-hospital survival per patient. Furthermore, we demonstrate that our proposed model is robust in cases where not all (clinical) data points are available.
Item Description:Veröffentlicht: 01. Juli 2023
Gesehen am 30.08.2023
Physical Description:Online Resource
ISSN:2045-2322
DOI:10.1038/s41598-023-37835-1

Similar Items