MOOD 2020: a public benchmark for out-of-distribution detection and localization on medical images
Detecting Out-of-Distribution (OoD) data is one of the greatest challenges in safe and robust deployment of machine learning algorithms in medicine. When the algorithms encounter cases that deviate from the distribution of the training data, they often produce incorrect and over-confident prediction...
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
[October 2022]
|
| In: |
IEEE transactions on medical imaging
Year: 2022, Jahrgang: 41, Heft: 10, Pages: 2728-2738 |
| ISSN: | 1558-254X |
| DOI: | 10.1109/TMI.2022.3170077 |
| Online-Zugang: | Verlag, Volltext: http://dx.doi.org/10.1109/TMI.2022.3170077
|
| Verfasserangaben: | David Zimmerer, Peter M. Full, Fabian Isensee, Paul Jäger, Tim Adler, Jens Petersen, Gregor Köhler, Tobias Ross, Annika Reinke, Antanas Kascenas, Bjørn Sand Jensen, Alison Q. O’Neil, Jeremy Tan, Benjamin Hou, James Batten, Huaqi Qiu, Bernhard Kainz, Nina Shvetsova, Irina Fedulova, Dmitry V. Dylov, Baolun Yu, Jianyang Zhai, Jingtao Hu, Runxuan Si, Sihang Zhou, Siqi Wang, Xinyang Li, Xuerun Chen, Yang Zhao, Sergio Naval Marimont, Giacomo Tarroni, Victor Saase, Lena Maier-Hein, and Klaus Maier-Hein |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1831412330 | ||
| 003 | DE-627 | ||
| 005 | 20230707002427.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 230118s2022 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1109/TMI.2022.3170077 |2 doi | |
| 035 | |a (DE-627)1831412330 | ||
| 035 | |a (DE-599)KXP1831412330 | ||
| 035 | |a (OCoLC)1389536373 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 33 |2 sdnb | ||
| 100 | 1 | |a Zimmerer, David |d 1994- |e VerfasserIn |0 (DE-588)1276158033 |0 (DE-627)1828116300 |4 aut | |
| 245 | 1 | 0 | |a MOOD 2020 |b a public benchmark for out-of-distribution detection and localization on medical images |c David Zimmerer, Peter M. Full, Fabian Isensee, Paul Jäger, Tim Adler, Jens Petersen, Gregor Köhler, Tobias Ross, Annika Reinke, Antanas Kascenas, Bjørn Sand Jensen, Alison Q. O’Neil, Jeremy Tan, Benjamin Hou, James Batten, Huaqi Qiu, Bernhard Kainz, Nina Shvetsova, Irina Fedulova, Dmitry V. Dylov, Baolun Yu, Jianyang Zhai, Jingtao Hu, Runxuan Si, Sihang Zhou, Siqi Wang, Xinyang Li, Xuerun Chen, Yang Zhao, Sergio Naval Marimont, Giacomo Tarroni, Victor Saase, Lena Maier-Hein, and Klaus Maier-Hein |
| 264 | 1 | |c [October 2022] | |
| 300 | |b Illustrationen | ||
| 300 | |a 11 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date of current version 30 September 2022 | ||
| 500 | |a Gesehen am 18.01.2023 | ||
| 520 | |a Detecting Out-of-Distribution (OoD) data is one of the greatest challenges in safe and robust deployment of machine learning algorithms in medicine. When the algorithms encounter cases that deviate from the distribution of the training data, they often produce incorrect and over-confident predictions. OoD detection algorithms aim to catch erroneous predictions in advance by analysing the data distribution and detecting potential instances of failure. Moreover, flagging OoD cases may support human readers in identifying incidental findings. Due to the increased interest in OoD algorithms, benchmarks for different domains have recently been established. In the medical imaging domain, for which reliable predictions are often essential, an open benchmark has been missing. We introduce the Medical-Out-Of-Distribution-Analysis-Challenge (MOOD) as an open, fair, and unbiased benchmark for OoD methods in the medical imaging domain. The analysis of the submitted algorithms shows that performance has a strong positive correlation with the perceived difficulty, and that all algorithms show a high variance for different anomalies, making it yet hard to recommend them for clinical practice. We also see a strong correlation between challenge ranking and performance on a simple toy test set, indicating that this might be a valuable addition as a proxy dataset during anomaly detection algorithm development. | ||
| 650 | 4 | |a Annotations | |
| 650 | 4 | |a Anomaly detection | |
| 650 | 4 | |a anomaly localization | |
| 650 | 4 | |a Benchmark testing | |
| 650 | 4 | |a biomedical challenge | |
| 650 | 4 | |a Biomedical imaging | |
| 650 | 4 | |a out-of-distribution analysis | |
| 650 | 4 | |a Prediction algorithms | |
| 650 | 4 | |a Task analysis | |
| 650 | 4 | |a Training | |
| 700 | 1 | |a Full, Peter M. |e VerfasserIn |0 (DE-588)1219695777 |0 (DE-627)1735698784 |4 aut | |
| 700 | 1 | |a Isensee, Fabian |d 1990- |e VerfasserIn |0 (DE-588)1207568430 |0 (DE-627)1694044998 |4 aut | |
| 700 | 1 | |a Jaeger, Paul F. |e VerfasserIn |0 (DE-588)1278497218 |0 (DE-627)1831414333 |4 aut | |
| 700 | 1 | |a Adler, Tim |d 1991- |e VerfasserIn |0 (DE-588)1194987672 |0 (DE-627)1677181680 |4 aut | |
| 700 | 1 | |a Petersen, Jens |d 1988- |e VerfasserIn |0 (DE-588)1197162348 |0 (DE-627)1678966665 |4 aut | |
| 700 | 1 | |a Köhler, Gregor |e VerfasserIn |0 (DE-588)1237383773 |0 (DE-627)1764084845 |4 aut | |
| 700 | 1 | |a Roß, Tobias |d 1990- |e VerfasserIn |0 (DE-588)118364955X |0 (DE-627)1663271143 |4 aut | |
| 700 | 1 | |a Reinke, Annika |d 1993- |e VerfasserIn |0 (DE-588)1219684252 |0 (DE-627)1735662143 |4 aut | |
| 700 | 1 | |a Kascenas, Antanas |e VerfasserIn |4 aut | |
| 700 | 1 | |a Jensen, Bjørn Sand |e VerfasserIn |4 aut | |
| 700 | 1 | |a O’Neil, Alison Q. |e VerfasserIn |4 aut | |
| 700 | 1 | |a Tan, Jeremy |e VerfasserIn |4 aut | |
| 700 | 1 | |a Hou, Benjamin |e VerfasserIn |4 aut | |
| 700 | 1 | |a Batten, James |e VerfasserIn |4 aut | |
| 700 | 1 | |a Qiu, Huaqi |e VerfasserIn |4 aut | |
| 700 | 1 | |a Kainz, Bernhard |e VerfasserIn |4 aut | |
| 700 | 1 | |a Shvetsova, Nina |e VerfasserIn |4 aut | |
| 700 | 1 | |a Fedulova, Irina |e VerfasserIn |4 aut | |
| 700 | 1 | |a Dylov, Dmitry V. |e VerfasserIn |4 aut | |
| 700 | 1 | |a Yu, Baolun |e VerfasserIn |4 aut | |
| 700 | 1 | |a Zhai, Jianyang |e VerfasserIn |4 aut | |
| 700 | 1 | |a Hu, Jingtao |e VerfasserIn |4 aut | |
| 700 | 1 | |a Si, Runxuan |e VerfasserIn |4 aut | |
| 700 | 1 | |a Zhou, Sihang |e VerfasserIn |4 aut | |
| 700 | 1 | |a Wang, Siqi |e VerfasserIn |4 aut | |
| 700 | 1 | |a Li, Xinyang |e VerfasserIn |4 aut | |
| 700 | 1 | |a Chen, Xuerun |e VerfasserIn |4 aut | |
| 700 | 1 | |a Zhao, Yang |e VerfasserIn |4 aut | |
| 700 | 1 | |a Marimont, Sergio Naval |e VerfasserIn |4 aut | |
| 700 | 1 | |a Tarroni, Giacomo |e VerfasserIn |4 aut | |
| 700 | 1 | |a Saase, Victor |d 1985- |e VerfasserIn |0 (DE-588)1214458068 |0 (DE-627)172550457X |4 aut | |
| 700 | 1 | |a Maier-Hein, Lena |d 1980- |e VerfasserIn |0 (DE-588)1075029252 |0 (DE-627)832869899 |0 (DE-576)190090804 |4 aut | |
| 700 | 1 | |a Maier-Hein, Klaus H. |d 1980- |e VerfasserIn |0 (DE-588)1100551875 |0 (DE-627)85946461X |0 (DE-576)333771222 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |a Institute of Electrical and Electronics Engineers |t IEEE transactions on medical imaging |d New York, NY : Institute of Electrical and Electronics Engineers,, 1982 |g 41(2022), 10 vom: Okt., Seite 2728-2738 |h Online-Ressource |w (DE-627)341354759 |w (DE-600)2068206-2 |w (DE-576)105283061 |x 1558-254X |7 nnas |
| 773 | 1 | 8 | |g volume:41 |g year:2022 |g number:10 |g month:10 |g pages:2728-2738 |g extent:11 |a MOOD 2020 a public benchmark for out-of-distribution detection and localization on medical images |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1109/TMI.2022.3170077 |x Verlag |x Resolving-System |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20230118 | ||
| 993 | |a Article | ||
| 994 | |a 2022 | ||
| 998 | |g 1100551875 |a Maier-Hein, Klaus H. |m 1100551875:Maier-Hein, Klaus H. |d 910000 |d 911400 |e 910000PM1100551875 |e 911400PM1100551875 |k 0/910000/ |k 1/910000/911400/ |p 34 |y j | ||
| 998 | |g 1075029252 |a Maier-Hein, Lena |m 1075029252:Maier-Hein, Lena |d 110000 |e 110000PM1075029252 |k 0/110000/ |p 33 | ||
| 998 | |g 1214458068 |a Saase, Victor |m 1214458068:Saase, Victor |d 60000 |e 60000PS1214458068 |k 0/60000/ |p 32 | ||
| 998 | |g 1219684252 |a Reinke, Annika |m 1219684252:Reinke, Annika |d 110000 |e 110000PR1219684252 |k 0/110000/ |p 9 | ||
| 998 | |g 1194987672 |a Adler, Tim |m 1194987672:Adler, Tim |d 110000 |e 110000PA1194987672 |k 0/110000/ |p 5 | ||
| 998 | |g 1219695777 |a Full, Peter M. |m 1219695777:Full, Peter M. |d 50000 |e 50000PF1219695777 |k 0/50000/ |p 2 | ||
| 998 | |g 1276158033 |a Zimmerer, David |m 1276158033:Zimmerer, David |d 110000 |e 110000PZ1276158033 |k 0/110000/ |p 1 |x j | ||
| 999 | |a KXP-PPN1831412330 |e 424951546X | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"recId":"1831412330","physDesc":[{"noteIll":"Illustrationen","extent":"11 S."}],"origin":[{"dateIssuedDisp":"[October 2022]","dateIssuedKey":"2022"}],"relHost":[{"physDesc":[{"extent":"Online-Ressource"}],"recId":"341354759","disp":"Institute of Electrical and Electronics EngineersIEEE transactions on medical imaging","origin":[{"publisherPlace":"New York, NY ; New York, NY","dateIssuedKey":"1982","dateIssuedDisp":"1982-","publisher":"Institute of Electrical and Electronics Engineers, ; IEEE"}],"type":{"bibl":"periodical","media":"Online-Ressource"},"language":["eng"],"note":["Gesehen am 13.01.11"],"title":[{"subtitle":"a publication of the IEEE Engineering in Medicine and Biology Society ...","title":"IEEE transactions on medical imaging","title_sort":"IEEE transactions on medical imaging"}],"corporate":[{"display":"Institute of Electrical and Electronics Engineers","role":"aut"}],"titleAlt":[{"title":"Transactions on medical imaging"}],"pubHistory":["1.1982(July) -"],"id":{"zdb":["2068206-2"],"issn":["1558-254X"],"eki":["341354759"]},"part":{"pages":"2728-2738","text":"41(2022), 10 vom: Okt., Seite 2728-2738","extent":"11","issue":"10","year":"2022","volume":"41"}}],"person":[{"role":"aut","given":"David","family":"Zimmerer","display":"Zimmerer, David"},{"given":"Peter M.","role":"aut","family":"Full","display":"Full, Peter M."},{"display":"Isensee, Fabian","family":"Isensee","given":"Fabian","role":"aut"},{"display":"Jaeger, Paul F.","family":"Jaeger","given":"Paul F.","role":"aut"},{"family":"Adler","given":"Tim","role":"aut","display":"Adler, Tim"},{"family":"Petersen","role":"aut","given":"Jens","display":"Petersen, Jens"},{"display":"Köhler, Gregor","given":"Gregor","role":"aut","family":"Köhler"},{"display":"Roß, Tobias","given":"Tobias","role":"aut","family":"Roß"},{"display":"Reinke, Annika","family":"Reinke","given":"Annika","role":"aut"},{"role":"aut","given":"Antanas","family":"Kascenas","display":"Kascenas, Antanas"},{"display":"Jensen, Bjørn Sand","family":"Jensen","given":"Bjørn Sand","role":"aut"},{"given":"Alison Q.","role":"aut","family":"O’Neil","display":"O’Neil, Alison Q."},{"family":"Tan","given":"Jeremy","role":"aut","display":"Tan, Jeremy"},{"family":"Hou","given":"Benjamin","role":"aut","display":"Hou, Benjamin"},{"family":"Batten","role":"aut","given":"James","display":"Batten, James"},{"role":"aut","given":"Huaqi","family":"Qiu","display":"Qiu, Huaqi"},{"family":"Kainz","role":"aut","given":"Bernhard","display":"Kainz, Bernhard"},{"given":"Nina","role":"aut","family":"Shvetsova","display":"Shvetsova, Nina"},{"family":"Fedulova","given":"Irina","role":"aut","display":"Fedulova, Irina"},{"family":"Dylov","given":"Dmitry V.","role":"aut","display":"Dylov, Dmitry V."},{"family":"Yu","given":"Baolun","role":"aut","display":"Yu, Baolun"},{"display":"Zhai, Jianyang","family":"Zhai","role":"aut","given":"Jianyang"},{"display":"Hu, Jingtao","family":"Hu","given":"Jingtao","role":"aut"},{"display":"Si, Runxuan","family":"Si","given":"Runxuan","role":"aut"},{"family":"Zhou","role":"aut","given":"Sihang","display":"Zhou, Sihang"},{"display":"Wang, Siqi","family":"Wang","role":"aut","given":"Siqi"},{"display":"Li, Xinyang","family":"Li","role":"aut","given":"Xinyang"},{"display":"Chen, Xuerun","role":"aut","given":"Xuerun","family":"Chen"},{"family":"Zhao","given":"Yang","role":"aut","display":"Zhao, Yang"},{"family":"Marimont","role":"aut","given":"Sergio Naval","display":"Marimont, Sergio Naval"},{"role":"aut","given":"Giacomo","family":"Tarroni","display":"Tarroni, Giacomo"},{"family":"Saase","given":"Victor","role":"aut","display":"Saase, Victor"},{"display":"Maier-Hein, Lena","family":"Maier-Hein","role":"aut","given":"Lena"},{"display":"Maier-Hein, Klaus H.","family":"Maier-Hein","given":"Klaus H.","role":"aut"}],"title":[{"title_sort":"MOOD 2020","title":"MOOD 2020","subtitle":"a public benchmark for out-of-distribution detection and localization on medical images"}],"language":["eng"],"type":{"bibl":"article-journal","media":"Online-Ressource"},"note":["Date of current version 30 September 2022","Gesehen am 18.01.2023"],"name":{"displayForm":["David Zimmerer, Peter M. Full, Fabian Isensee, Paul Jäger, Tim Adler, Jens Petersen, Gregor Köhler, Tobias Ross, Annika Reinke, Antanas Kascenas, Bjørn Sand Jensen, Alison Q. O’Neil, Jeremy Tan, Benjamin Hou, James Batten, Huaqi Qiu, Bernhard Kainz, Nina Shvetsova, Irina Fedulova, Dmitry V. Dylov, Baolun Yu, Jianyang Zhai, Jingtao Hu, Runxuan Si, Sihang Zhou, Siqi Wang, Xinyang Li, Xuerun Chen, Yang Zhao, Sergio Naval Marimont, Giacomo Tarroni, Victor Saase, Lena Maier-Hein, and Klaus Maier-Hein"]},"id":{"eki":["1831412330"],"doi":["10.1109/TMI.2022.3170077"]}} | ||
| SRT | |a ZIMMERERDAMOOD20202022 | ||