Why rankings of biomedical image analysis competitions should be interpreted with care - Maier-Hein, Lena; Eisenmann, Matthias; Reinke, Annika; Onogur, Sinan; Stankovic, Marko; Godau, Patrick; Arbel, Tal; Hrvoje, Bogunovic; Bradley, Andrew P.; Carass, Aaron; Feldmann, Carolin; Frangi, Alejandro F.; et al. (Nature Communications, 2018)
Biomedical image analysis challenges have increased in the last ten years, but common practices have not been established yet. Here the authors analyze 150 recent challenges and demonstrate that outcome varies based on the metrics used and that limited information reporting hampers reproducibility.