Automated ensemble multimodal machine learning for healthcare
The application of machine learning in medicine and healthcare has led to the creation of numerous diagnostic and prognostic models. However, despite their success, current approaches generally issue predictions using data from a single modality. This stands in stark contrast with clinician decision...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
June 2025
|
| In: |
IEEE journal of biomedical and health informatics
Year: 2025, Volume: 29, Issue: 6, Pages: 4213-4226 |
| ISSN: | 2168-2208 |
| DOI: | 10.1109/JBHI.2025.3530156 |
| Online Access: | Verlag, lizenzpflichtig, Volltext: https://doi.org/10.1109/JBHI.2025.3530156 Verlag, lizenzpflichtig, Volltext: https://ieeexplore.ieee.org/document/10842455/authors 
|
| Author Notes: | Fergus Imrie, Stefan Denner, Lucas S. Brunschwig, Klaus Maier-Hein, Mihaela van der Schaar |
| Summary: | The application of machine learning in medicine and healthcare has led to the creation of numerous diagnostic and prognostic models. However, despite their success, current approaches generally issue predictions using data from a single modality. This stands in stark contrast with clinician decision-making which employs diverse information from multiple sources. While several multimodal machine learning approaches exist, significant challenges in developing multimodal systems remain that are hindering clinical adoption. In this paper, we introduce a multimodal framework, AutoPrognosis-M, that enables the integration of structured clinical (tabular) data and medical imaging using automated machine learning. AutoPrognosis-M incorporates 17 imaging models, including convolutional neural networks and vision transformers, and three distinct multimodal fusion strategies. In an illustrative application using a multimodal skin lesion dataset, we highlight the importance of multimodal machine learning and the power of combining multiple fusion strategies using ensemble learning. We have open-sourced our framework as a tool for the community and hope it will accelerate the uptake of multimodal machine learning in healthcare and spur further innovation. |
|---|---|
| Item Description: | Veröffentlicht: 15. Januar 2025 Gesehen am 27.10.2025 |
| Physical Description: | Online Resource |
| ISSN: | 2168-2208 |
| DOI: | 10.1109/JBHI.2025.3530156 |