Prompt injection attacks on vision language models in oncology
Vision-language artificial intelligence models (VLMs) possess medical knowledge and can be employed in healthcare in numerous ways, including as image interpreters, virtual scribes, and general decision support systems. However, here, we demonstrate that current VLMs applied to medical tasks exhibit...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
01 February 2025
|
| In: |
Nature Communications
Year: 2025, Volume: 16, Pages: 1-9 |
| ISSN: | 2041-1723 |
| DOI: | 10.1038/s41467-024-55631-x |
| Online Access: | Verlag, kostenfrei, Volltext: https://doi.org/10.1038/s41467-024-55631-x Verlag, kostenfrei, Volltext: http://www.nature.com/articles/s41467-024-55631-x 
|
| Author Notes: | Jan Clusmann, Dyke Ferber, Isabella C. Wiest, Carolin V. Schneider, Titus J. Brinker, Sebastian Foersch, Daniel Truhn and Jakob Nikolas Kather |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1929937695 | ||
| 003 | DE-627 | ||
| 005 | 20250709095810.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 250708s2025 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1038/s41467-024-55631-x |2 doi | |
| 035 | |a (DE-627)1929937695 | ||
| 035 | |a (DE-599)KXP1929937695 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 33 |2 sdnb | ||
| 100 | 1 | |a Clusmann, Jan Niklas |e VerfasserIn |0 (DE-588)1286278783 |0 (DE-627)1842721984 |4 aut | |
| 245 | 1 | 0 | |a Prompt injection attacks on vision language models in oncology |c Jan Clusmann, Dyke Ferber, Isabella C. Wiest, Carolin V. Schneider, Titus J. Brinker, Sebastian Foersch, Daniel Truhn and Jakob Nikolas Kather |
| 264 | 1 | |c 01 February 2025 | |
| 300 | |b Illustrationen, Diagramme | ||
| 300 | |a 9 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 08.07.2025 | ||
| 520 | |a Vision-language artificial intelligence models (VLMs) possess medical knowledge and can be employed in healthcare in numerous ways, including as image interpreters, virtual scribes, and general decision support systems. However, here, we demonstrate that current VLMs applied to medical tasks exhibit a fundamental security flaw: they can be compromised by prompt injection attacks. These can be used to output harmful information just by interacting with the VLM, without any access to its parameters. We perform a quantitative study to evaluate the vulnerabilities to these attacks in four state of the art VLMs: Claude-3 Opus, Claude-3.5 Sonnet, Reka Core, and GPT-4o. Using a set of N = 594 attacks, we show that all of these models are susceptible. Specifically, we show that embedding sub-visual prompts in manifold medical imaging data can cause the model to provide harmful output, and that these prompts are non-obvious to human observers. Thus, our study demonstrates a key vulnerability in medical VLMs which should be mitigated before widespread clinical adoption. | ||
| 650 | 4 | |a Cancer imaging | |
| 650 | 4 | |a Computational science | |
| 650 | 4 | |a Machine learning | |
| 650 | 4 | |a Medical imaging | |
| 700 | 1 | |a Ferber, Dyke |e VerfasserIn |0 (DE-588)1171467079 |0 (DE-627)1040545629 |0 (DE-576)513746056 |4 aut | |
| 700 | 1 | |a Wiest, Isabella |d 1992- |e VerfasserIn |0 (DE-588)1198882956 |0 (DE-627)168103638X |4 aut | |
| 700 | 1 | |a Schneider, Carolin Victoria |d 1995- |e VerfasserIn |0 (DE-588)121656907X |0 (DE-627)1727792238 |4 aut | |
| 700 | 1 | |a Brinker, Titus Josef |d 1990- |e VerfasserIn |0 (DE-588)1156309395 |0 (DE-627)1018860487 |0 (DE-576)502097434 |4 aut | |
| 700 | 1 | |a Försch, Sebastian |d 1985- |e VerfasserIn |0 (DE-588)1018553894 |0 (DE-627)682860832 |0 (DE-576)356024814 |4 aut | |
| 700 | 1 | |a Truhn, Daniel |e VerfasserIn |0 (DE-588)1047348306 |0 (DE-627)778145913 |0 (DE-576)400927314 |4 aut | |
| 700 | 1 | |a Kather, Jakob Nikolas |d 1989- |e VerfasserIn |0 (DE-588)1064064914 |0 (DE-627)812897587 |0 (DE-576)423589091 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Nature Communications |d [London] : Springer Nature, 2010 |g 16(2025), Artikel-ID 1239, Seite 1-9 |h Online-Ressource |w (DE-627)626457688 |w (DE-600)2553671-0 |w (DE-576)331555905 |x 2041-1723 |7 nnas |a Prompt injection attacks on vision language models in oncology |
| 773 | 1 | 8 | |g volume:16 |g year:2025 |g elocationid:1239 |g pages:1-9 |g extent:9 |a Prompt injection attacks on vision language models in oncology |
| 856 | 4 | 0 | |u https://doi.org/10.1038/s41467-024-55631-x |x Verlag |x Resolving-System |z kostenfrei |3 Volltext |
| 856 | 4 | 0 | |u http://www.nature.com/articles/s41467-024-55631-x |x Verlag |z kostenfrei |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20250708 | ||
| 993 | |a Article | ||
| 994 | |a 2025 | ||
| 998 | |g 1064064914 |a Kather, Jakob Nikolas |m 1064064914:Kather, Jakob Nikolas |d 910000 |d 910100 |e 910000PK1064064914 |e 910100PK1064064914 |k 0/910000/ |k 1/910000/910100/ |p 8 |y j | ||
| 998 | |g 1156309395 |a Brinker, Titus Josef |m 1156309395:Brinker, Titus Josef |d 50000 |e 50000PB1156309395 |k 0/50000/ |p 5 | ||
| 998 | |g 1198882956 |a Wiest, Isabella |m 1198882956:Wiest, Isabella |d 60000 |d 61100 |e 60000PW1198882956 |e 61100PW1198882956 |k 0/60000/ |k 1/60000/61100/ |p 3 | ||
| 998 | |g 1171467079 |a Ferber, Dyke |m 1171467079:Ferber, Dyke |d 910000 |d 910100 |e 910000PF1171467079 |e 910100PF1171467079 |k 0/910000/ |k 1/910000/910100/ |p 2 | ||
| 999 | |a KXP-PPN1929937695 |e 4743765277 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"person":[{"family":"Clusmann","roleDisplay":"VerfasserIn","display":"Clusmann, Jan Niklas","given":"Jan Niklas","role":"aut"},{"roleDisplay":"VerfasserIn","display":"Ferber, Dyke","given":"Dyke","family":"Ferber","role":"aut"},{"role":"aut","family":"Wiest","roleDisplay":"VerfasserIn","given":"Isabella","display":"Wiest, Isabella"},{"display":"Schneider, Carolin Victoria","roleDisplay":"VerfasserIn","given":"Carolin Victoria","family":"Schneider","role":"aut"},{"roleDisplay":"VerfasserIn","display":"Brinker, Titus Josef","given":"Titus Josef","family":"Brinker","role":"aut"},{"given":"Sebastian","roleDisplay":"VerfasserIn","display":"Försch, Sebastian","family":"Försch","role":"aut"},{"role":"aut","family":"Truhn","display":"Truhn, Daniel","roleDisplay":"VerfasserIn","given":"Daniel"},{"role":"aut","given":"Jakob Nikolas","roleDisplay":"VerfasserIn","display":"Kather, Jakob Nikolas","family":"Kather"}],"type":{"media":"Online-Ressource","bibl":"article-journal"},"title":[{"title":"Prompt injection attacks on vision language models in oncology","title_sort":"Prompt injection attacks on vision language models in oncology"}],"recId":"1929937695","id":{"doi":["10.1038/s41467-024-55631-x"],"eki":["1929937695"]},"name":{"displayForm":["Jan Clusmann, Dyke Ferber, Isabella C. Wiest, Carolin V. Schneider, Titus J. Brinker, Sebastian Foersch, Daniel Truhn and Jakob Nikolas Kather"]},"note":["Gesehen am 08.07.2025"],"physDesc":[{"noteIll":"Illustrationen, Diagramme","extent":"9 S."}],"relHost":[{"type":{"media":"Online-Ressource","bibl":"periodical"},"physDesc":[{"extent":"Online-Ressource"}],"part":{"text":"16(2025), Artikel-ID 1239, Seite 1-9","extent":"9","volume":"16","pages":"1-9","year":"2025"},"note":["Gesehen am 13.06.24"],"language":["eng"],"disp":"Prompt injection attacks on vision language models in oncologyNature Communications","title":[{"title":"Nature Communications","title_sort":"Nature Communications"}],"id":{"issn":["2041-1723"],"eki":["626457688"],"zdb":["2553671-0"]},"recId":"626457688","pubHistory":["2010-"],"origin":[{"dateIssuedDisp":"[2010]-","publisher":"Springer Nature ; Nature Publishing Group UK","publisherPlace":"[London] ; [London]"}]}],"language":["eng"],"origin":[{"dateIssuedKey":"2025","dateIssuedDisp":"01 February 2025"}]} | ||
| SRT | |a CLUSMANNJAPROMPTINJE0120 | ||