Semantic hyperspectral image synthesis for cross-modality knowledge transfer in surgical data science
Purpose Hyperspectral imaging (HSI) is a promising intraoperative imaging modality, with potential applications ranging from tissue classification and discrimination to perfusion monitoring and cancer detection. However, surgical HSI datasets are scarce, hindering the development of robust data-driv...
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
24 April 2025
|
| In: |
International journal of computer assisted radiology and surgery
Year: 2025, Volume: 20, Issue: 6, Pages: 1205-1213 |
| ISSN: | 1861-6429 |
| DOI: | 10.1007/s11548-025-03364-7 |
| Online Access: | Verlag, lizenzpflichtig, Volltext: https://doi.org/10.1007/s11548-025-03364-7 Verlag, lizenzpflichtig, Volltext: http://link.springer.com/article/10.1007/s11548-025-03364-7 
|
| Author Notes: | Viet Tran Ba, Marco Hübner, Ahmad Bin Qasim, Maike Rees, Jan Sellner, Silvia Seidlitz, Evangelia Christodoulou, Berkin Özdemir, Alexander Studier-Fischer, Felix Nickel, Leonardo Ayala, Lena Maier-Hein |
MARC
| LEADER | 00000caa a22000002c 4500 | ||
|---|---|---|---|
| 001 | 1927313112 | ||
| 003 | DE-627 | ||
| 005 | 20250903140626.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 250603s2025 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1007/s11548-025-03364-7 |2 doi | |
| 035 | |a (DE-627)1927313112 | ||
| 035 | |a (DE-599)KXP1927313112 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 33 |2 sdnb | ||
| 100 | 1 | |a Tran Ba, Viet |d 2000- |e VerfasserIn |0 (DE-588)1367605512 |0 (DE-627)1927315158 |4 aut | |
| 245 | 1 | 0 | |a Semantic hyperspectral image synthesis for cross-modality knowledge transfer in surgical data science |c Viet Tran Ba, Marco Hübner, Ahmad Bin Qasim, Maike Rees, Jan Sellner, Silvia Seidlitz, Evangelia Christodoulou, Berkin Özdemir, Alexander Studier-Fischer, Felix Nickel, Leonardo Ayala, Lena Maier-Hein |
| 264 | 1 | |c 24 April 2025 | |
| 300 | |b Illustrationen, Diagramme | ||
| 300 | |a 9 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 03.06.2025 | ||
| 520 | |a Purpose Hyperspectral imaging (HSI) is a promising intraoperative imaging modality, with potential applications ranging from tissue classification and discrimination to perfusion monitoring and cancer detection. However, surgical HSI datasets are scarce, hindering the development of robust data-driven algorithms. The purpose of this work was to address this critical bottleneck with a novel approach to knowledge transfer across modalities. Methods We propose the use of generative modeling to leverage imaging data across optical imaging modalities. The core of the method is a latent diffusion model (LDM) capable of converting a semantic segmentation mask obtained from any modality into a realistic hyperspectral image, such that geometry information can be learned across modalities. The value of the approach was assessed both qualitatively and quantitatively using surgical scene segmentation as a downstream task. Results Our study with more than 13,000 hyperspectral images, partially annotated with a total of 37 tissue and object classes, suggests that LDMs are well-suited for the synthesis of realistic high-resolution hyperspectral images even when trained on few samples or applied to annotations from different modalities and geometric out-of-distribution annotations. Using our approach for generative augmentation yielded a performance boost of up to 35% in the Dice similarity coefficient for the task of semantic hyperspectral image segmentation. Conclusion As our method is capable of augmenting HSI datasets in a manner agnostic to the modality of the leveraged data, it could serve as a blueprint for addressing the data bottleneck encountered for novel imaging modalities. | ||
| 700 | 1 | |a Hübner, Marco |e VerfasserIn |0 (DE-588)125394556X |0 (DE-627)1796368660 |4 aut | |
| 700 | 1 | |a Qasim, Ahmad Bin |d 1994- |e VerfasserIn |0 (DE-588)1337181854 |0 (DE-627)1896765548 |4 aut | |
| 700 | 1 | |a Rees, Maike |e VerfasserIn |0 (DE-588)1333964293 |0 (DE-627)1892141671 |4 aut | |
| 700 | 1 | |a Sellner, Jan |d 1990- |e VerfasserIn |0 (DE-588)1333964773 |0 (DE-627)1892148137 |4 aut | |
| 700 | 1 | |a Seidlitz, Silvia |e VerfasserIn |0 (DE-588)1264853750 |0 (DE-627)1813743282 |4 aut | |
| 700 | 1 | |a Christodoulou, Evangelia |e VerfasserIn |0 (DE-588)134790574X |0 (DE-627)1908166002 |4 aut | |
| 700 | 1 | |a Özdemir, Berkin |d 1997- |e VerfasserIn |0 (DE-588)1252285809 |0 (DE-627)1793432740 |4 aut | |
| 700 | 1 | |a Studier-Fischer, Alexander |d 1995- |e VerfasserIn |0 (DE-588)1196041105 |0 (DE-627)1677916958 |4 aut | |
| 700 | 1 | |a Nickel, Felix |d 1982- |e VerfasserIn |0 (DE-588)1067980059 |0 (DE-627)819414875 |0 (DE-576)427122619 |4 aut | |
| 700 | 1 | |a Ayala, Leonardo |d 1994- |e VerfasserIn |0 (DE-588)1254842977 |0 (DE-627)1797630938 |4 aut | |
| 700 | 1 | |a Maier-Hein, Lena |d 1980- |e VerfasserIn |0 (DE-588)1075029252 |0 (DE-627)832869899 |0 (DE-576)190090804 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t International journal of computer assisted radiology and surgery |d Berlin : Springer, 2006 |g 20(2025), 6, Seite 1205-1213 |h Online-Ressource |w (DE-627)512299250 |w (DE-600)2235881-X |w (DE-576)283119322 |x 1861-6429 |7 nnas |a Semantic hyperspectral image synthesis for cross-modality knowledge transfer in surgical data science |
| 773 | 1 | 8 | |g volume:20 |g year:2025 |g number:6 |g pages:1205-1213 |g extent:9 |a Semantic hyperspectral image synthesis for cross-modality knowledge transfer in surgical data science |
| 856 | 4 | 0 | |u https://doi.org/10.1007/s11548-025-03364-7 |x Verlag |x Resolving-System |z lizenzpflichtig |3 Volltext |
| 856 | 4 | 0 | |u http://link.springer.com/article/10.1007/s11548-025-03364-7 |x Verlag |z lizenzpflichtig |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20250603 | ||
| 993 | |a Article | ||
| 994 | |a 2025 | ||
| 998 | |g 1075029252 |a Maier-Hein, Lena |m 1075029252:Maier-Hein, Lena |d 910000 |d 910200 |e 910000PM1075029252 |e 910200PM1075029252 |k 0/910000/ |k 1/910000/910200/ |p 12 |y j | ||
| 998 | |g 1067980059 |a Nickel, Felix |m 1067980059:Nickel, Felix |d 50000 |e 50000PN1067980059 |k 0/50000/ |p 10 | ||
| 998 | |g 1196041105 |a Studier-Fischer, Alexander |m 1196041105:Studier-Fischer, Alexander |d 60000 |d 63100 |d 60000 |e 60000PS1196041105 |e 63100PS1196041105 |e 60000PS1196041105 |k 0/60000/ |k 1/60000/63100/ |k 0/60000/ |p 9 | ||
| 998 | |g 1252285809 |a Özdemir, Berkin |m 1252285809:Özdemir, Berkin |d 50000 |e 50000PO1252285809 |k 0/50000/ |p 8 | ||
| 998 | |g 1264853750 |a Seidlitz, Silvia |m 1264853750:Seidlitz, Silvia |d 110000 |e 110000PS1264853750 |k 0/110000/ |p 6 | ||
| 998 | |g 1333964773 |a Sellner, Jan |m 1333964773:Sellner, Jan |p 5 | ||
| 998 | |g 1333964293 |a Rees, Maike |m 1333964293:Rees, Maike |d 110000 |e 110000PR1333964293 |k 0/110000/ |p 4 | ||
| 998 | |g 1337181854 |a Qasim, Ahmad Bin |m 1337181854:Qasim, Ahmad Bin |d 110000 |e 110000PQ1337181854 |k 0/110000/ |p 3 | ||
| 998 | |g 125394556X |a Hübner, Marco |m 125394556X:Hübner, Marco |d 110000 |e 110000PH125394556X |k 0/110000/ |p 2 | ||
| 998 | |g 1367605512 |a Tran Ba, Viet |m 1367605512:Tran Ba, Viet |d 110000 |e 110000PT1367605512 |k 0/110000/ |p 1 |x j | ||
| 999 | |a KXP-PPN1927313112 |e 473018257X | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"relHost":[{"language":["eng"],"note":["Gesehen am 26.02.14","Fortsetzung der Druck-Ausgabe"],"pubHistory":["1.2006/07 -"],"type":{"media":"Online-Ressource","bibl":"periodical"},"recId":"512299250","id":{"issn":["1861-6429"],"zdb":["2235881-X"],"eki":["512299250"]},"origin":[{"dateIssuedKey":"2006","publisherPlace":"Berlin ; Heidelberg [u.a.]","dateIssuedDisp":"2006-","publisher":"Springer"}],"disp":"Semantic hyperspectral image synthesis for cross-modality knowledge transfer in surgical data scienceInternational journal of computer assisted radiology and surgery","title":[{"title":"International journal of computer assisted radiology and surgery","subtitle":"a journal for interdisciplinary research, development and applications of image guided diagnosis and therapy","title_sort":"International journal of computer assisted radiology and surgery"}],"physDesc":[{"extent":"Online-Ressource"}],"part":{"volume":"20","text":"20(2025), 6, Seite 1205-1213","issue":"6","pages":"1205-1213","year":"2025","extent":"9"}}],"title":[{"title_sort":"Semantic hyperspectral image synthesis for cross-modality knowledge transfer in surgical data science","title":"Semantic hyperspectral image synthesis for cross-modality knowledge transfer in surgical data science"}],"physDesc":[{"extent":"9 S.","noteIll":"Illustrationen, Diagramme"}],"language":["eng"],"note":["Gesehen am 03.06.2025"],"person":[{"display":"Tran Ba, Viet","family":"Tran Ba","role":"aut","given":"Viet"},{"given":"Marco","role":"aut","family":"Hübner","display":"Hübner, Marco"},{"display":"Qasim, Ahmad Bin","role":"aut","family":"Qasim","given":"Ahmad Bin"},{"given":"Maike","role":"aut","family":"Rees","display":"Rees, Maike"},{"family":"Sellner","role":"aut","display":"Sellner, Jan","given":"Jan"},{"display":"Seidlitz, Silvia","family":"Seidlitz","role":"aut","given":"Silvia"},{"display":"Christodoulou, Evangelia","family":"Christodoulou","role":"aut","given":"Evangelia"},{"given":"Berkin","display":"Özdemir, Berkin","family":"Özdemir","role":"aut"},{"given":"Alexander","family":"Studier-Fischer","role":"aut","display":"Studier-Fischer, Alexander"},{"family":"Nickel","role":"aut","display":"Nickel, Felix","given":"Felix"},{"given":"Leonardo","display":"Ayala, Leonardo","family":"Ayala","role":"aut"},{"given":"Lena","display":"Maier-Hein, Lena","family":"Maier-Hein","role":"aut"}],"type":{"bibl":"article-journal","media":"Online-Ressource"},"recId":"1927313112","id":{"doi":["10.1007/s11548-025-03364-7"],"eki":["1927313112"]},"name":{"displayForm":["Viet Tran Ba, Marco Hübner, Ahmad Bin Qasim, Maike Rees, Jan Sellner, Silvia Seidlitz, Evangelia Christodoulou, Berkin Özdemir, Alexander Studier-Fischer, Felix Nickel, Leonardo Ayala, Lena Maier-Hein"]},"origin":[{"dateIssuedDisp":"24 April 2025","dateIssuedKey":"2025"}]} | ||
| SRT | |a TRANBAVIETSEMANTICHY2420 | ||