Why rankings of biomedical image analysis competitions should be interpreted with care

Biomedical image analysis challenges have increased in the last ten years, but common practices have not been established yet. Here the authors analyze 150 recent challenges and demonstrate that outcome varies based on the metrics used and that limited information reporting hampers reproducibility.

Saved in:
Bibliographic Details
Main Authors: Maier-Hein, Lena (Author) , Eisenmann, Matthias (Author) , Reinke, Annika (Author) , Onogur, Sinan (Author) , Stankovic, Marko (Author) , Godau, Patrick (Author) , Arbel, Tal (Author) , Hrvoje, Bogunovic (Author) , Bradley, Andrew P. (Author) , Carass, Aaron (Author) , Feldmann, Carolin (Author) , Frangi, Alejandro F. (Author) , Full, Peter M. (Author) , van Ginneken, Bram (Author) , Hanbury, Allan (Author) , Honauer, Katrin (Author) , Kozubek, Michal (Author) , Landman, Bennett A. (Author) , März, Keno (Author) , Maier, Oskar (Author) , Maier-Hein, Klaus H. (Author) , Menze, Bjoern (Author) , Müller, Henning (Author) , Neher, Peter (Author) , Niessen, Wiro (Author) , Rajppot, Nasir (Author) , Sharp, Gregory C. (Author) , Sirinukunwattana, Korsuk (Author) , Speidel, Stafanie (Author) , Stock, Christian (Author) , Stoyanov, Danail (Author) , Taha, Abdel Aziz (Author) , van der Sommen, Fons (Author) , Wang, Ching-Wei (Author) , Weber, Marc-André (Author) , Zheng, Guoyan (Author) , Jannin, Pierre (Author) , Kopp-Schneider, Annette (Author)
Format: Article (Journal)
Language:English
Published: 06 December 2018
In: Nature Communications
Year: 2018, Volume: 9, Pages: 5217
ISSN:2041-1723
DOI:10.1038/s41467-018-07619-7
Online Access:Verlag, Volltext: http://dx.doi.org/10.1038/s41467-018-07619-7
Verlag, Volltext: https://www.nature.com/articles/s41467-018-07619-7
Get full text
Author Notes:Lena Maier-Hein et al.#
Description
Summary:Biomedical image analysis challenges have increased in the last ten years, but common practices have not been established yet. Here the authors analyze 150 recent challenges and demonstrate that outcome varies based on the metrics used and that limited information reporting hampers reproducibility.
Item Description:#Lena Maier-Henn, Matthias Eisenmann, Annika Reinke, Sinan Onogur, Marko Stankovic, Patrick Scholz, Tal Arbel, Hrvoje Bogunovic, Andrew P. Bradley, Aaron Carass, Carolin Feldmann, Alejandro F. Frangi, Peter M. Full, Bram van Ginneken, Allan Hanbury, Katrin Honauer, Michal Kozubek, Bennett A. Landman, Keno März, Oskar Maier, Klaus Maier-Hein, Bjoern H. Menze, Henning Müller, Peter F. Neher, Wiro Niessen, Nasir Rajpoot, Gregory C. Sharp, Korsuk Sirinukunwattana, Stefanie Speidel, Christian Stock, Danail Stoyanov, Abdel Aziz Taha, Fons van der Sommen, Ching-Wei Wang, Marc-André Weber, Guoyan Zheng, Pierre Jannin & Annette Kopp-Schneider
Gesehen am 27.02.2019
Physical Description:Online Resource
ISSN:2041-1723
DOI:10.1038/s41467-018-07619-7