Effects of label noise on deep learning-based skin cancer classification
Recent studies have shown that deep learning is capable of classifying dermatoscopic images at least as well as dermatologists. However, many studies in skin cancer classification utilize non-biopsy-verified training images. This imperfect ground truth introduces a systematic error, but the effects...
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
06 May 2020
|
| In: |
Frontiers in medicine
Year: 2020, Jahrgang: 7, Pages: 1-7 |
| ISSN: | 2296-858X |
| DOI: | 10.3389/fmed.2020.00177 |
| Online-Zugang: | Resolving-System, kostenfrei: http://dx.doi.org/10.3389/fmed.2020.00177 Verlag, lizenzpflichtig, Volltext: https://doi.org/10.3389/fmed.2020.00177 Verlag, lizenzpflichtig, Volltext: https://www.frontiersin.org/articles/10.3389/fmed.2020.00177/full 
|
| Verfasserangaben: | Achim Hekler, Jakob N. Kather, Eva Krieghoff-Henning, Jochen S. Utikal, Friedegund Meier, Frank F. Gellrich, Julius Upmeier zu Belzen, Lars French, Justin G. Schlager, Kamran Ghoreschi, Tabea Wilhelm, Heinz Kutzner, Carola Berking, Markus V. Heppt, Sebastian Haferkamp, Wiebke Sondermann, Dirk Schadendorf, Bastian Schilling, Benjamin Izar, Roman Maron, Max Schmitt, Stefan Fröhling, Daniel B. Lipka and Titus J. Brinker |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1738651126 | ||
| 003 | DE-627 | ||
| 005 | 20251021115850.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 201113s2020 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.3389/fmed.2020.00177 |2 doi | |
| 035 | |a (DE-627)1738651126 | ||
| 035 | |a (DE-599)KXP1738651126 | ||
| 035 | |a (OCoLC)1341376548 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 33 |2 sdnb | ||
| 100 | 1 | |a Hekler, Achim |e VerfasserIn |0 (DE-588)1196829314 |0 (DE-627)1678721344 |4 aut | |
| 245 | 1 | 0 | |a Effects of label noise on deep learning-based skin cancer classification |c Achim Hekler, Jakob N. Kather, Eva Krieghoff-Henning, Jochen S. Utikal, Friedegund Meier, Frank F. Gellrich, Julius Upmeier zu Belzen, Lars French, Justin G. Schlager, Kamran Ghoreschi, Tabea Wilhelm, Heinz Kutzner, Carola Berking, Markus V. Heppt, Sebastian Haferkamp, Wiebke Sondermann, Dirk Schadendorf, Bastian Schilling, Benjamin Izar, Roman Maron, Max Schmitt, Stefan Fröhling, Daniel B. Lipka and Titus J. Brinker |
| 264 | 1 | |c 06 May 2020 | |
| 300 | |a 7 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 520 | |a Recent studies have shown that deep learning is capable of classifying dermatoscopic images at least as well as dermatologists. However, many studies in skin cancer classification utilize non-biopsy-verified training images. This imperfect ground truth introduces a systematic error, but the effects on classifier performance are currently unknown. Here, we systematically examine the effects of label noise by training and evaluating convolutional neural networks (CNN) with 804 images of melanoma and nevi labeled either by dermatologists or by biopsy. The CNNs are evaluated on a test set of 384 images by means of 4-fold cross validation comparing the outputs with either the corresponding dermatological or the biopsy-verified diagnosis. With identical ground truths of training and test labels, high accuracies with 75.03% (95% CI: 74.39-75.66%) for dermatological and 73.80% (95% CI: 73.10-74.51%) for biopsy-verified labels can be achieved. However, if the CNN is trained and tested with different ground truths, accuracy drops significantly to 64.53% (95% CI: 63.12-65.94%, p<0.01) on a non-biopsy-verified and to 64.24% (95% CI: 62.66-65.83%, p<0.01) on a biopsy-verified test set. In conclusion, deep learning methods for skin cancer classification are highly sensitive to label noise and future work should use biopsy-verified training images to mitigate this problem. | ||
| 650 | 4 | |a artificial intelligence | |
| 650 | 4 | |a Dermatology | |
| 650 | 4 | |a Label noise | |
| 650 | 4 | |a machine learning | |
| 650 | 4 | |a Melanoma | |
| 650 | 4 | |a Skin Cancer | |
| 700 | 1 | |a Kather, Jakob Nikolas |d 1989- |e VerfasserIn |0 (DE-588)1064064914 |0 (DE-627)812897587 |0 (DE-576)423589091 |4 aut | |
| 700 | 1 | |a Krieghoff-Henning, Eva |d 1976- |e VerfasserIn |0 (DE-588)132407914 |0 (DE-627)52267786X |0 (DE-576)299126706 |4 aut | |
| 700 | 1 | |a Utikal, Jochen |d 1974- |e VerfasserIn |0 (DE-588)1026463750 |0 (DE-627)726765015 |0 (DE-576)371816580 |4 aut | |
| 700 | 1 | |a Meier, Friedegund |d 1960- |e VerfasserIn |0 (DE-588)124970184 |0 (DE-627)369238672 |0 (DE-576)294594825 |4 aut | |
| 700 | 1 | |a Gellrich, Frank Friedrich |d 1990- |e VerfasserIn |0 (DE-588)123548162X |0 (DE-627)1760486523 |4 aut | |
| 700 | 1 | |a Upmeier zu Belzen, Julius |e VerfasserIn |4 aut | |
| 700 | 1 | |a French, Lars E. |d 1963- |e VerfasserIn |0 (DE-588)141166835 |0 (DE-627)70389966X |0 (DE-576)322518792 |4 aut | |
| 700 | 1 | |a Schlager, Justin Gabriel |d 1987- |e VerfasserIn |0 (DE-588)1119171520 |0 (DE-627)872683915 |0 (DE-576)479749930 |4 aut | |
| 700 | 1 | |a Ghoreschi, Kamran |d 1970- |e VerfasserIn |0 (DE-588)124466079 |0 (DE-627)363367357 |0 (DE-576)294184988 |4 aut | |
| 700 | 1 | |a Wilhelm, Tabea |e VerfasserIn |4 aut | |
| 700 | 1 | |a Kutzner, Heinz |e VerfasserIn |0 (DE-588)1207300756 |0 (DE-627)1693511363 |4 aut | |
| 700 | 1 | |a Berking, Carola |d 1971- |e VerfasserIn |0 (DE-588)115659714 |0 (DE-627)077390121 |0 (DE-576)290004942 |4 aut | |
| 700 | 1 | |a Heppt, Markus V. |d 1987- |e VerfasserIn |0 (DE-588)1072242346 |0 (DE-627)827081111 |0 (DE-576)43371767X |4 aut | |
| 700 | 1 | |a Haferkamp, Sebastian |d 1978- |e VerfasserIn |0 (DE-588)132018330 |0 (DE-627)51684296X |0 (DE-576)298896044 |4 aut | |
| 700 | 1 | |a Sondermann, Wiebke |e VerfasserIn |0 (DE-588)1198953756 |0 (DE-627)1681162121 |4 aut | |
| 700 | 1 | |a Schadendorf, Dirk |d 1960- |e VerfasserIn |0 (DE-588)11142576X |0 (DE-627)499566076 |0 (DE-576)289702275 |4 aut | |
| 700 | 1 | |a Schilling, Bastian |d 1979- |e VerfasserIn |0 (DE-588)142513563 |0 (DE-627)704247887 |0 (DE-576)331358913 |4 aut | |
| 700 | 1 | |a Izar, Benjamin |e VerfasserIn |4 aut | |
| 700 | 1 | |a Maron, Roman C. |e VerfasserIn |0 (DE-588)1198959851 |0 (DE-627)1681173867 |4 aut | |
| 700 | 1 | |a Schmitt, Max |e VerfasserIn |0 (DE-588)1236577469 |0 (DE-627)1761961586 |4 aut | |
| 700 | 1 | |a Fröhling, Stefan |d 1969- |e VerfasserIn |0 (DE-588)120890046 |0 (DE-627)080950302 |0 (DE-576)188733930 |4 aut | |
| 700 | 1 | |a Lipka, Daniel |d 1976- |e VerfasserIn |0 (DE-588)131915312 |0 (DE-627)516076426 |0 (DE-576)298833328 |4 aut | |
| 700 | 1 | |a Brinker, Titus Josef |d 1990- |e VerfasserIn |0 (DE-588)1156309395 |0 (DE-627)1018860487 |0 (DE-576)502097434 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Frontiers in medicine |d Lausanne : Frontiers Media, 2014 |g 7(2020), Artikel-ID 177, Seite 1-7 |h Online-Ressource |w (DE-627)789482991 |w (DE-600)2775999-4 |w (DE-576)408729597 |x 2296-858X |7 nnas |a Effects of label noise on deep learning-based skin cancer classification |
| 773 | 1 | 8 | |g volume:7 |g year:2020 |g elocationid:177 |g pages:1-7 |g extent:7 |a Effects of label noise on deep learning-based skin cancer classification |
| 856 | 4 | 0 | |u http://dx.doi.org/10.3389/fmed.2020.00177 |x Resolving-System |z kostenfrei |
| 856 | 4 | 0 | |u https://doi.org/10.3389/fmed.2020.00177 |x Verlag |x Resolving-System |z lizenzpflichtig |3 Volltext |
| 856 | 4 | 0 | |u https://www.frontiersin.org/articles/10.3389/fmed.2020.00177/full |x Verlag |z lizenzpflichtig |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20210412 | ||
| 993 | |a Article | ||
| 994 | |a 2020 | ||
| 998 | |g 1156309395 |a Brinker, Titus Josef |m 1156309395:Brinker, Titus Josef |d 910000 |d 911300 |e 910000PB1156309395 |e 911300PB1156309395 |k 0/910000/ |k 1/910000/911300/ |p 24 |y j | ||
| 998 | |g 131915312 |a Lipka, Daniel |m 131915312:Lipka, Daniel |d 140000 |e 140000PL131915312 |k 0/140000/ |p 23 | ||
| 998 | |g 120890046 |a Fröhling, Stefan |m 120890046:Fröhling, Stefan |d 50000 |e 50000PF120890046 |k 0/50000/ |p 22 | ||
| 998 | |g 11142576X |a Schadendorf, Dirk |m 11142576X:Schadendorf, Dirk |d 50000 |e 50000PS11142576X |k 0/50000/ |p 17 | ||
| 998 | |g 1026463750 |a Utikal, Jochen |m 1026463750:Utikal, Jochen |d 60000 |e 60000PU1026463750 |k 0/60000/ |p 4 | ||
| 998 | |g 1064064914 |a Kather, Jakob Nikolas |m 1064064914:Kather, Jakob Nikolas |d 910000 |d 910100 |e 910000PK1064064914 |e 910100PK1064064914 |k 0/910000/ |k 1/910000/910100/ |p 2 | ||
| 999 | |a KXP-PPN1738651126 |e 3906073661 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"physDesc":[{"extent":"7 S."}],"recId":"1738651126","type":{"bibl":"article-journal","media":"Online-Ressource"},"language":["eng"],"id":{"eki":["1738651126"],"doi":["10.3389/fmed.2020.00177"]},"title":[{"title_sort":"Effects of label noise on deep learning-based skin cancer classification","title":"Effects of label noise on deep learning-based skin cancer classification"}],"name":{"displayForm":["Achim Hekler, Jakob N. Kather, Eva Krieghoff-Henning, Jochen S. Utikal, Friedegund Meier, Frank F. Gellrich, Julius Upmeier zu Belzen, Lars French, Justin G. Schlager, Kamran Ghoreschi, Tabea Wilhelm, Heinz Kutzner, Carola Berking, Markus V. Heppt, Sebastian Haferkamp, Wiebke Sondermann, Dirk Schadendorf, Bastian Schilling, Benjamin Izar, Roman Maron, Max Schmitt, Stefan Fröhling, Daniel B. Lipka and Titus J. Brinker"]},"origin":[{"dateIssuedKey":"2020","dateIssuedDisp":"06 May 2020"}],"person":[{"family":"Hekler","role":"aut","given":"Achim","display":"Hekler, Achim"},{"display":"Kather, Jakob Nikolas","family":"Kather","given":"Jakob Nikolas","role":"aut"},{"given":"Eva","role":"aut","family":"Krieghoff-Henning","display":"Krieghoff-Henning, Eva"},{"role":"aut","given":"Jochen","family":"Utikal","display":"Utikal, Jochen"},{"display":"Meier, Friedegund","family":"Meier","given":"Friedegund","role":"aut"},{"given":"Frank Friedrich","role":"aut","family":"Gellrich","display":"Gellrich, Frank Friedrich"},{"family":"Upmeier zu Belzen","given":"Julius","role":"aut","display":"Upmeier zu Belzen, Julius"},{"family":"French","role":"aut","given":"Lars E.","display":"French, Lars E."},{"display":"Schlager, Justin Gabriel","family":"Schlager","role":"aut","given":"Justin Gabriel"},{"display":"Ghoreschi, Kamran","role":"aut","given":"Kamran","family":"Ghoreschi"},{"display":"Wilhelm, Tabea","family":"Wilhelm","given":"Tabea","role":"aut"},{"family":"Kutzner","role":"aut","given":"Heinz","display":"Kutzner, Heinz"},{"display":"Berking, Carola","given":"Carola","role":"aut","family":"Berking"},{"family":"Heppt","role":"aut","given":"Markus V.","display":"Heppt, Markus V."},{"display":"Haferkamp, Sebastian","family":"Haferkamp","given":"Sebastian","role":"aut"},{"family":"Sondermann","given":"Wiebke","role":"aut","display":"Sondermann, Wiebke"},{"family":"Schadendorf","given":"Dirk","role":"aut","display":"Schadendorf, Dirk"},{"display":"Schilling, Bastian","given":"Bastian","role":"aut","family":"Schilling"},{"display":"Izar, Benjamin","role":"aut","given":"Benjamin","family":"Izar"},{"family":"Maron","given":"Roman C.","role":"aut","display":"Maron, Roman C."},{"display":"Schmitt, Max","family":"Schmitt","role":"aut","given":"Max"},{"given":"Stefan","role":"aut","family":"Fröhling","display":"Fröhling, Stefan"},{"display":"Lipka, Daniel","family":"Lipka","role":"aut","given":"Daniel"},{"role":"aut","given":"Titus Josef","family":"Brinker","display":"Brinker, Titus Josef"}],"relHost":[{"titleAlt":[{"title":"FMED"},{"title":"Front. Med."}],"pubHistory":["2014 -"],"id":{"issn":["2296-858X"],"eki":["789482991"],"zdb":["2775999-4"]},"part":{"pages":"1-7","year":"2020","text":"7(2020), Artikel-ID 177, Seite 1-7","volume":"7","extent":"7"},"origin":[{"publisher":"Frontiers Media","dateIssuedKey":"2014","publisherPlace":"Lausanne","dateIssuedDisp":"2014-"}],"title":[{"title":"Frontiers in medicine","title_sort":"Frontiers in medicine"}],"type":{"media":"Online-Ressource","bibl":"periodical"},"language":["eng"],"note":["Gesehen am 09.02.2021"],"recId":"789482991","physDesc":[{"extent":"Online-Ressource"}],"disp":"Effects of label noise on deep learning-based skin cancer classificationFrontiers in medicine"}]} | ||
| SRT | |a HEKLERACHIEFFECTSOFL0620 | ||