Medical slice transformer for improved diagnosis and explainability on 3D medical images with DINOv2
Abstract - - Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) are essential clinical cross-sectional imaging techniques for diagnosing complex conditions. However, large 3D datasets with annotations for deep learning are scarce. While methods like DINOv2 are e...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
04 July 2025
|
| In: |
Scientific reports
Year: 2025, Volume: 15, Pages: 1-12 |
| ISSN: | 2045-2322 |
| DOI: | 10.1038/s41598-025-09041-8 |
| Online Access: | Verlag, kostenfrei, Volltext: https://doi.org/10.1038/s41598-025-09041-8 Verlag, kostenfrei, Volltext: https://www.nature.com/articles/s41598-025-09041-8 
|
| Author Notes: | Gustav Müller-Franzes, Firas Khader, Robert Siepmann, Tianyu Han, Jakob Nikolas Kather, Sven Nebelung Nebelung & Daniel Truhn |
MARC
| LEADER | 00000naa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1946057282 | ||
| 003 | DE-627 | ||
| 005 | 20251215122216.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 251215s2025 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1038/s41598-025-09041-8 |2 doi | |
| 035 | |a (DE-627)1946057282 | ||
| 035 | |a (DE-599)KXP1946057282 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 33 |2 sdnb | ||
| 100 | 1 | |a Müller-Franzes, Gustav |e VerfasserIn |0 (DE-588)1328685586 |0 (DE-627)188810273X |4 aut | |
| 245 | 1 | 0 | |a Medical slice transformer for improved diagnosis and explainability on 3D medical images with DINOv2 |c Gustav Müller-Franzes, Firas Khader, Robert Siepmann, Tianyu Han, Jakob Nikolas Kather, Sven Nebelung Nebelung & Daniel Truhn |
| 264 | 1 | |c 04 July 2025 | |
| 300 | |a 12 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 15.12.2025 | ||
| 520 | |a Abstract - - Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) are essential clinical cross-sectional imaging techniques for diagnosing complex conditions. However, large 3D datasets with annotations for deep learning are scarce. While methods like DINOv2 are encouraging for 2D image analysis, these methods have not been applied to 3D medical images. Furthermore, deep learning models often lack explainability due to their “black-box” nature. This study aims to extend 2D self-supervised models, specifically DINOv2, to 3D medical imaging while evaluating their potential for explainable outcomes. We introduce the Medical Slice Transformer (MST) framework to adapt 2D self-supervised models for 3D medical image analysis. MST combines a Transformer architecture with a 2D feature extractor, i.e., DINOv2. We evaluate its diagnostic performance against a 3D convolutional neural network (3D ResNet) across three clinical datasets: breast MRI (651 patients), chest CT (722 patients), and knee MRI (1199 patients). Both methods were tested for diagnosing breast cancer, predicting lung nodule dignity, and detecting meniscus tears. Diagnostic performance was assessed by calculating the Area Under the Receiver Operating Characteristic Curve (AUC). Explainability was evaluated through a radiologist’s qualitative comparison of saliency maps based on slice and lesion correctness. P-values were calculated using Delong’s test. MST achieved higher AUC values compared to ResNet across all three datasets: breast (0.94 ± 0.01 vs. 0.91 ± 0.02, - P - = 0.02), chest (0.95 ± 0.01 vs. 0.92 ± 0.02, - P - = 0.13), and knee (0.85 ± 0.04 vs. 0.69 ± 0.05, - P - = 0.001). Saliency maps were consistently more precise and anatomically correct for MST than for ResNet. Self-supervised 2D models like DINOv2 can be effectively adapted for 3D medical imaging using MST, offering enhanced diagnostic accuracy and explainability compared to convolutional neural networks. | ||
| 700 | 1 | |a Khader, Firas |e VerfasserIn |4 aut | |
| 700 | 1 | |a Siepmann, Robert |e VerfasserIn |4 aut | |
| 700 | 1 | |a Han, Tianyu |e VerfasserIn |4 aut | |
| 700 | 1 | |a Kather, Jakob Nikolas |d 1989- |e VerfasserIn |0 (DE-588)1064064914 |0 (DE-627)812897587 |0 (DE-576)423589091 |4 aut | |
| 700 | 1 | |a Nebelung, Sven |e VerfasserIn |4 aut | |
| 700 | 1 | |a Truhn, Daniel |e VerfasserIn |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Scientific reports |d [London] : Springer Nature, 2011 |g 15(2025), Artikel-ID 23979, Seite 1-12 |h Online-Ressource |w (DE-627)663366712 |w (DE-600)2615211-3 |w (DE-576)346641179 |x 2045-2322 |7 nnas |a Medical slice transformer for improved diagnosis and explainability on 3D medical images with DINOv2 |
| 773 | 1 | 8 | |g volume:15 |g year:2025 |g elocationid:23979 |g pages:1-12 |g extent:12 |a Medical slice transformer for improved diagnosis and explainability on 3D medical images with DINOv2 |
| 856 | 4 | 0 | |u https://doi.org/10.1038/s41598-025-09041-8 |x Verlag |x Resolving-System |z kostenfrei |3 Volltext |7 0 |
| 856 | 4 | 0 | |u https://www.nature.com/articles/s41598-025-09041-8 |x Verlag |z kostenfrei |3 Volltext |7 0 |
| 951 | |a AR | ||
| 992 | |a 20251215 | ||
| 993 | |a Article | ||
| 994 | |a 2025 | ||
| 998 | |g 1064064914 |a Kather, Jakob Nikolas |m 1064064914:Kather, Jakob Nikolas |d 910000 |d 910100 |e 910000PK1064064914 |e 910100PK1064064914 |k 0/910000/ |k 1/910000/910100/ |p 5 | ||
| 999 | |a KXP-PPN1946057282 |e 4828059490 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"physDesc":[{"extent":"12 S."}],"recId":"1946057282","note":["Gesehen am 15.12.2025"],"language":["eng"],"type":{"media":"Online-Ressource","bibl":"article-journal"},"title":[{"title":"Medical slice transformer for improved diagnosis and explainability on 3D medical images with DINOv2","title_sort":"Medical slice transformer for improved diagnosis and explainability on 3D medical images with DINOv2"}],"relHost":[{"recId":"663366712","physDesc":[{"extent":"Online-Ressource"}],"disp":"Medical slice transformer for improved diagnosis and explainability on 3D medical images with DINOv2Scientific reports","origin":[{"dateIssuedKey":"2011","dateIssuedDisp":"2011-","publisherPlace":"[London] ; London","publisher":"Springer Nature ; Nature Publishing Group"}],"pubHistory":["1, article number 1 (2011)-"],"id":{"issn":["2045-2322"],"eki":["663366712"],"zdb":["2615211-3"]},"part":{"pages":"1-12","year":"2025","text":"15(2025), Artikel-ID 23979, Seite 1-12","volume":"15","extent":"12"},"title":[{"title":"Scientific reports","title_sort":"Scientific reports"}],"language":["eng"],"type":{"bibl":"periodical","media":"Online-Ressource"},"note":["Gesehen am 12.07.24"]}],"person":[{"family":"Müller-Franzes","given":"Gustav","role":"aut","display":"Müller-Franzes, Gustav"},{"family":"Khader","role":"aut","given":"Firas","display":"Khader, Firas"},{"family":"Siepmann","role":"aut","given":"Robert","display":"Siepmann, Robert"},{"display":"Han, Tianyu","given":"Tianyu","role":"aut","family":"Han"},{"display":"Kather, Jakob Nikolas","role":"aut","given":"Jakob Nikolas","family":"Kather"},{"family":"Nebelung","given":"Sven","role":"aut","display":"Nebelung, Sven"},{"given":"Daniel","role":"aut","family":"Truhn","display":"Truhn, Daniel"}],"origin":[{"dateIssuedKey":"2025","dateIssuedDisp":"04 July 2025"}],"id":{"doi":["10.1038/s41598-025-09041-8"],"eki":["1946057282"]},"name":{"displayForm":["Gustav Müller-Franzes, Firas Khader, Robert Siepmann, Tianyu Han, Jakob Nikolas Kather, Sven Nebelung Nebelung & Daniel Truhn"]}} | ||
| SRT | |a MUELLERFRAMEDICALSLI0420 | ||