A benchmark for neural network robustness in skin cancer classification
Background - One prominent application for deep learning-based classifiers is skin cancer classification on dermoscopic images. However, classifier evaluation is often limited to holdout data which can mask common shortcomings such as susceptibility to confounding factors. To increase clinical appli...
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
11 August 2021
|
| In: |
European journal of cancer
Year: 2021, Jahrgang: 155, Pages: 191-199 |
| ISSN: | 1879-0852 |
| DOI: | 10.1016/j.ejca.2021.06.047 |
| Online-Zugang: | Verlag, lizenzpflichtig, Volltext: https://doi.org/10.1016/j.ejca.2021.06.047 Verlag, lizenzpflichtig, Volltext: https://www.sciencedirect.com/science/article/pii/S0959804921004421 
|
| Verfasserangaben: | Roman C. Maron, Justin G. Schlager, Sarah Haggenmüller, Christof von Kalle, Jochen S. Utikal, Friedegund Meier, Frank F. Gellrich, Sarah Hobelsberger, Axel Hauschild, Lars French, Lucie Heinzerling, Max Schlaak, Kamran Ghoreschi, Franz J. Hilke, Gabriela Poch, Markus V. Heppt, Carola Berking, Sebastian Haferkamp, Wiebke Sondermann, Dirk Schadendorf, Bastian Schilling, Matthias Goebeler, Eva Krieghoff-Henning, Achim Hekler, Stefan Fröhling, Daniel B. Lipka, Jakob N. Kather, Titus J. Brinker |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1775329305 | ||
| 003 | DE-627 | ||
| 005 | 20240414193525.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 211026s2021 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1016/j.ejca.2021.06.047 |2 doi | |
| 035 | |a (DE-627)1775329305 | ||
| 035 | |a (DE-599)KXP1775329305 | ||
| 035 | |a (OCoLC)1341422276 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 33 |2 sdnb | ||
| 100 | 1 | |a Maron, Roman C. |e VerfasserIn |0 (DE-588)1198959851 |0 (DE-627)1681173867 |4 aut | |
| 245 | 1 | 2 | |a A benchmark for neural network robustness in skin cancer classification |c Roman C. Maron, Justin G. Schlager, Sarah Haggenmüller, Christof von Kalle, Jochen S. Utikal, Friedegund Meier, Frank F. Gellrich, Sarah Hobelsberger, Axel Hauschild, Lars French, Lucie Heinzerling, Max Schlaak, Kamran Ghoreschi, Franz J. Hilke, Gabriela Poch, Markus V. Heppt, Carola Berking, Sebastian Haferkamp, Wiebke Sondermann, Dirk Schadendorf, Bastian Schilling, Matthias Goebeler, Eva Krieghoff-Henning, Achim Hekler, Stefan Fröhling, Daniel B. Lipka, Jakob N. Kather, Titus J. Brinker |
| 264 | 1 | |c 11 August 2021 | |
| 300 | |a 9 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 26.10.2021 | ||
| 520 | |a Background - One prominent application for deep learning-based classifiers is skin cancer classification on dermoscopic images. However, classifier evaluation is often limited to holdout data which can mask common shortcomings such as susceptibility to confounding factors. To increase clinical applicability, it is necessary to thoroughly evaluate such classifiers on out-of-distribution (OOD) data. - Objective - The objective of the study was to establish a dermoscopic skin cancer benchmark in which classifier robustness to OOD data can be measured. - Methods - Using a proprietary dermoscopic image database and a set of image transformations, we create an OOD robustness benchmark and evaluate the robustness of four different convolutional neural network (CNN) architectures on it. - Results - The benchmark contains three data sets—Skin Archive Munich (SAM), SAM-corrupted (SAM-C) and SAM-perturbed (SAM-P)—and is publicly available for download. To maintain the benchmark's OOD status, ground truth labels are not provided and test results should be sent to us for assessment. The SAM data set contains 319 unmodified and biopsy-verified dermoscopic melanoma (n = 194) and nevus (n = 125) images. SAM-C and SAM-P contain images from SAM which were artificially modified to test a classifier against low-quality inputs and to measure its prediction stability over small image changes, respectively. All four CNNs showed susceptibility to corruptions and perturbations. - Conclusions - This benchmark provides three data sets which allow for OOD testing of binary skin cancer classifiers. Our classifier performance confirms the shortcomings of CNNs and provides a frame of reference. Altogether, this benchmark should facilitate a more thorough evaluation process and thereby enable the development of more robust skin cancer classifiers. | ||
| 650 | 4 | |a Artificial intelligence | |
| 650 | 4 | |a Benchmarking | |
| 650 | 4 | |a Deep learning | |
| 650 | 4 | |a Dermatology | |
| 650 | 4 | |a Melanoma | |
| 650 | 4 | |a Nevus | |
| 650 | 4 | |a Skin neoplasms | |
| 700 | 1 | |a Schlager, Justin G. |e VerfasserIn |4 aut | |
| 700 | 1 | |a Haggenmüller, Sarah |d 1995- |e VerfasserIn |0 (DE-588)1231946709 |0 (DE-627)1755618042 |4 aut | |
| 700 | 1 | |a Kalle, Christof von |d 1962- |e VerfasserIn |0 (DE-588)1036481115 |0 (DE-627)75107926X |0 (DE-576)168957396 |4 aut | |
| 700 | 1 | |a Utikal, Jochen |d 1974- |e VerfasserIn |0 (DE-588)1026463750 |0 (DE-627)726765015 |0 (DE-576)371816580 |4 aut | |
| 700 | 1 | |a Meier, Friedegund |e VerfasserIn |4 aut | |
| 700 | 1 | |a Gellrich, Frank F. |e VerfasserIn |4 aut | |
| 700 | 1 | |a Hobelsberger, Sarah |e VerfasserIn |4 aut | |
| 700 | 1 | |a Hauschild, Axel |e VerfasserIn |4 aut | |
| 700 | 1 | |a French, Lars |e VerfasserIn |4 aut | |
| 700 | 1 | |a Heinzerling, Lucie |e VerfasserIn |4 aut | |
| 700 | 1 | |a Schlaak, Max Simon |d 1977- |e VerfasserIn |0 (DE-588)124559166 |0 (DE-627)363420835 |0 (DE-576)294231625 |4 aut | |
| 700 | 1 | |a Ghoreschi, Kamran |d 1970- |e VerfasserIn |0 (DE-588)124466079 |0 (DE-627)363367357 |0 (DE-576)294184988 |4 aut | |
| 700 | 1 | |a Hilke, Franz |e VerfasserIn |0 (DE-588)1230558411 |0 (DE-627)1753030498 |4 aut | |
| 700 | 1 | |a Poch, Gabriela |e VerfasserIn |4 aut | |
| 700 | 1 | |a Heppt, Markus V. |d 1987- |e VerfasserIn |0 (DE-588)1072242346 |0 (DE-627)827081111 |0 (DE-576)43371767X |4 aut | |
| 700 | 1 | |a Berking, Carola |d 1971- |e VerfasserIn |0 (DE-588)115659714 |0 (DE-627)077390121 |0 (DE-576)290004942 |4 aut | |
| 700 | 1 | |a Haferkamp, Sebastian |d 1978- |e VerfasserIn |0 (DE-588)132018330 |0 (DE-627)51684296X |0 (DE-576)298896044 |4 aut | |
| 700 | 1 | |a Sondermann, Wiebke |e VerfasserIn |0 (DE-588)1198953756 |0 (DE-627)1681162121 |4 aut | |
| 700 | 1 | |a Schadendorf, Dirk |d 1960- |e VerfasserIn |0 (DE-588)11142576X |0 (DE-627)499566076 |0 (DE-576)289702275 |4 aut | |
| 700 | 1 | |a Schilling, Bastian |d 1979- |e VerfasserIn |0 (DE-588)142513563 |0 (DE-627)704247887 |0 (DE-576)331358913 |4 aut | |
| 700 | 1 | |a Goebeler, Matthias |d 1963- |e VerfasserIn |0 (DE-588)11206468X |0 (DE-627)671503510 |0 (DE-576)352522062 |4 aut | |
| 700 | 1 | |a Krieghoff-Henning, Eva |d 1976- |e VerfasserIn |0 (DE-588)132407914 |0 (DE-627)52267786X |0 (DE-576)299126706 |4 aut | |
| 700 | 1 | |a Hekler, Achim |e VerfasserIn |0 (DE-588)1196829314 |0 (DE-627)1678721344 |4 aut | |
| 700 | 1 | |a Fröhling, Stefan |d 1969- |e VerfasserIn |0 (DE-588)120890046 |0 (DE-627)080950302 |0 (DE-576)188733930 |4 aut | |
| 700 | 1 | |a Lipka, Daniel |d 1976- |e VerfasserIn |0 (DE-588)131915312 |0 (DE-627)516076426 |0 (DE-576)298833328 |4 aut | |
| 700 | 1 | |a Kather, Jakob Nikolas |d 1989- |e VerfasserIn |0 (DE-588)1064064914 |0 (DE-627)812897587 |0 (DE-576)423589091 |4 aut | |
| 700 | 1 | |a Brinker, Titus Josef |d 1990- |e VerfasserIn |0 (DE-588)1156309395 |0 (DE-627)1018860487 |0 (DE-576)502097434 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t European journal of cancer |d Amsterdam [u.a.] : Elsevier, 1992 |g 155(2021), Seite 191-199 |w (DE-627)266883400 |w (DE-600)1468190-0 |w (DE-576)090954173 |x 1879-0852 |7 nnas |a A benchmark for neural network robustness in skin cancer classification |
| 773 | 1 | 8 | |g volume:155 |g year:2021 |g pages:191-199 |g extent:9 |a A benchmark for neural network robustness in skin cancer classification |
| 856 | 4 | 0 | |u https://doi.org/10.1016/j.ejca.2021.06.047 |x Verlag |x Resolving-System |z lizenzpflichtig |3 Volltext |
| 856 | 4 | 0 | |u https://www.sciencedirect.com/science/article/pii/S0959804921004421 |x Verlag |z lizenzpflichtig |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20211026 | ||
| 993 | |a Article | ||
| 994 | |a 2021 | ||
| 998 | |g 1156309395 |a Brinker, Titus Josef |m 1156309395:Brinker, Titus Josef |d 50000 |e 50000PB1156309395 |k 0/50000/ |p 28 |y j | ||
| 998 | |g 1064064914 |a Kather, Jakob Nikolas |m 1064064914:Kather, Jakob Nikolas |d 910000 |d 910100 |e 910000PK1064064914 |e 910100PK1064064914 |k 0/910000/ |k 1/910000/910100/ |p 27 | ||
| 998 | |g 131915312 |a Lipka, Daniel |m 131915312:Lipka, Daniel |d 140000 |e 140000PL131915312 |k 0/140000/ |p 26 | ||
| 998 | |g 120890046 |a Fröhling, Stefan |m 120890046:Fröhling, Stefan |d 50000 |e 50000PF120890046 |k 0/50000/ |p 25 | ||
| 998 | |g 11142576X |a Schadendorf, Dirk |m 11142576X:Schadendorf, Dirk |d 50000 |e 50000PS11142576X |k 0/50000/ |p 20 | ||
| 998 | |g 1026463750 |a Utikal, Jochen |m 1026463750:Utikal, Jochen |d 60000 |e 60000PU1026463750 |k 0/60000/ |p 5 | ||
| 998 | |g 1036481115 |a Kalle, Christof von |m 1036481115:Kalle, Christof von |d 50000 |e 50000PK1036481115 |k 0/50000/ |p 4 | ||
| 998 | |g 1231946709 |a Haggenmüller, Sarah |m 1231946709:Haggenmüller, Sarah |d 60000 |e 60000PH1231946709 |k 0/60000/ |p 3 | ||
| 999 | |a KXP-PPN1775329305 |e 3996299311 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"name":{"displayForm":["Roman C. Maron, Justin G. Schlager, Sarah Haggenmüller, Christof von Kalle, Jochen S. Utikal, Friedegund Meier, Frank F. Gellrich, Sarah Hobelsberger, Axel Hauschild, Lars French, Lucie Heinzerling, Max Schlaak, Kamran Ghoreschi, Franz J. Hilke, Gabriela Poch, Markus V. Heppt, Carola Berking, Sebastian Haferkamp, Wiebke Sondermann, Dirk Schadendorf, Bastian Schilling, Matthias Goebeler, Eva Krieghoff-Henning, Achim Hekler, Stefan Fröhling, Daniel B. Lipka, Jakob N. Kather, Titus J. Brinker"]},"id":{"doi":["10.1016/j.ejca.2021.06.047"],"eki":["1775329305"]},"recId":"1775329305","physDesc":[{"extent":"9 S."}],"relHost":[{"recId":"266883400","disp":"A benchmark for neural network robustness in skin cancer classificationEuropean journal of cancer","origin":[{"publisher":"Elsevier ; Pergamon Press","dateIssuedKey":"1992","publisherPlace":"Amsterdam [u.a.] ; [Erscheinungsort nicht ermittelbar]","dateIssuedDisp":"1992-"}],"language":["eng"],"type":{"bibl":"periodical","media":"Online-Ressource"},"note":["Gesehen am 21.03.24","Ungezählte Beil.: Supplement"],"title":[{"title":"European journal of cancer","title_sort":"European journal of cancer"}],"corporate":[{"role":"isb","display":"European Organization for Research on Treatment of Cancer"},{"display":"European Association for Cancer Research","role":"isb"},{"display":"European School of Oncology","role":"isb"}],"pubHistory":["28.1992 -"],"titleAlt":[{"title":"EJC online"}],"id":{"eki":["266883400"],"issn":["1879-0852"],"zdb":["1468190-0"]},"part":{"pages":"191-199","year":"2021","volume":"155","text":"155(2021), Seite 191-199","extent":"9"}}],"person":[{"display":"Maron, Roman C.","given":"Roman C.","role":"aut","family":"Maron"},{"family":"Schlager","given":"Justin G.","role":"aut","display":"Schlager, Justin G."},{"given":"Sarah","role":"aut","family":"Haggenmüller","display":"Haggenmüller, Sarah"},{"family":"Kalle","role":"aut","given":"Christof von","display":"Kalle, Christof von"},{"display":"Utikal, Jochen","given":"Jochen","role":"aut","family":"Utikal"},{"family":"Meier","role":"aut","given":"Friedegund","display":"Meier, Friedegund"},{"family":"Gellrich","role":"aut","given":"Frank F.","display":"Gellrich, Frank F."},{"display":"Hobelsberger, Sarah","role":"aut","given":"Sarah","family":"Hobelsberger"},{"display":"Hauschild, Axel","given":"Axel","role":"aut","family":"Hauschild"},{"family":"French","role":"aut","given":"Lars","display":"French, Lars"},{"display":"Heinzerling, Lucie","role":"aut","given":"Lucie","family":"Heinzerling"},{"family":"Schlaak","given":"Max Simon","role":"aut","display":"Schlaak, Max Simon"},{"family":"Ghoreschi","given":"Kamran","role":"aut","display":"Ghoreschi, Kamran"},{"role":"aut","given":"Franz","family":"Hilke","display":"Hilke, Franz"},{"display":"Poch, Gabriela","family":"Poch","role":"aut","given":"Gabriela"},{"display":"Heppt, Markus V.","role":"aut","given":"Markus V.","family":"Heppt"},{"display":"Berking, Carola","family":"Berking","given":"Carola","role":"aut"},{"display":"Haferkamp, Sebastian","role":"aut","given":"Sebastian","family":"Haferkamp"},{"family":"Sondermann","given":"Wiebke","role":"aut","display":"Sondermann, Wiebke"},{"role":"aut","given":"Dirk","family":"Schadendorf","display":"Schadendorf, Dirk"},{"display":"Schilling, Bastian","given":"Bastian","role":"aut","family":"Schilling"},{"family":"Goebeler","given":"Matthias","role":"aut","display":"Goebeler, Matthias"},{"display":"Krieghoff-Henning, Eva","given":"Eva","role":"aut","family":"Krieghoff-Henning"},{"family":"Hekler","role":"aut","given":"Achim","display":"Hekler, Achim"},{"display":"Fröhling, Stefan","given":"Stefan","role":"aut","family":"Fröhling"},{"display":"Lipka, Daniel","family":"Lipka","given":"Daniel","role":"aut"},{"family":"Kather","role":"aut","given":"Jakob Nikolas","display":"Kather, Jakob Nikolas"},{"role":"aut","given":"Titus Josef","family":"Brinker","display":"Brinker, Titus Josef"}],"origin":[{"dateIssuedDisp":"11 August 2021","dateIssuedKey":"2021"}],"title":[{"title_sort":"benchmark for neural network robustness in skin cancer classification","title":"A benchmark for neural network robustness in skin cancer classification"}],"note":["Gesehen am 26.10.2021"],"type":{"bibl":"article-journal","media":"Online-Ressource"},"language":["eng"]} | ||
| SRT | |a MARONROMANBENCHMARKF1120 | ||