From classification accuracy to proper scoring rules: elicitability of probabilistic top list predictions
In the face of uncertainty, the need for probabilistic assessments has long been recognized in the literature on forecasting. In classification, however, comparative evaluation of classifiers often focuses on predictions specifying a single class through the use of simple accuracy measures, which di...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article (Journal) Chapter/Article |
| Language: | English |
| Published: |
27 Jan 2023
|
| In: |
Arxiv
Year: 2023, Pages: 1-21 |
| DOI: | 10.48550/arXiv.2301.11797 |
| Online Access: | Verlag, kostenfrei, Volltext: https://doi.org/10.48550/arXiv.2301.11797 Verlag, kostenfrei, Volltext: http://arxiv.org/abs/2301.11797 
|
| Author Notes: | Johannes Resin |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1878838245 | ||
| 003 | DE-627 | ||
| 005 | 20240307030215.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240124s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.48550/arXiv.2301.11797 |2 doi | |
| 035 | |a (DE-627)1878838245 | ||
| 035 | |a (DE-599)KXP1878838245 | ||
| 035 | |a (OCoLC)1425207741 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 17 |2 sdnb | ||
| 100 | 1 | |a Resin, Johannes |e VerfasserIn |0 (DE-588)1286759269 |0 (DE-627)1843351706 |4 aut | |
| 245 | 1 | 0 | |a From classification accuracy to proper scoring rules |b elicitability of probabilistic top list predictions |c Johannes Resin |
| 264 | 1 | |c 27 Jan 2023 | |
| 300 | |a 21 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 24.01.2024 | ||
| 520 | |a In the face of uncertainty, the need for probabilistic assessments has long been recognized in the literature on forecasting. In classification, however, comparative evaluation of classifiers often focuses on predictions specifying a single class through the use of simple accuracy measures, which disregard any probabilistic uncertainty quantification. I propose probabilistic top lists as a novel type of prediction in classification, which bridges the gap between single-class predictions and predictive distributions. The probabilistic top list functional is elicitable through the use of strictly consistent evaluation metrics. The proposed evaluation metrics are based on symmetric proper scoring rules and admit comparison of various types of predictions ranging from single-class point predictions to fully specified predictive distributions. The Brier score yields a metric that is particularly well suited for this kind of comparison. | ||
| 650 | 4 | |a Statistics - Machine Learning | |
| 650 | 4 | |a Statistics - Methodology | |
| 773 | 0 | 8 | |i Enthalten in |t Arxiv |d Ithaca, NY : Cornell University, 1991 |g (2023), Artikel-ID 2301.11797, Seite 1-21 |h Online-Ressource |w (DE-627)509006531 |w (DE-600)2225896-6 |w (DE-576)28130436X |7 nnas |a From classification accuracy to proper scoring rules elicitability of probabilistic top list predictions |
| 773 | 1 | 8 | |g year:2023 |g elocationid:2301.11797 |g pages:1-21 |g extent:21 |a From classification accuracy to proper scoring rules elicitability of probabilistic top list predictions |
| 856 | 4 | 0 | |u https://doi.org/10.48550/arXiv.2301.11797 |x Verlag |x Resolving-System |z kostenfrei |3 Volltext |
| 856 | 4 | 0 | |u http://arxiv.org/abs/2301.11797 |x Verlag |z kostenfrei |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20240124 | ||
| 993 | |a Article | ||
| 994 | |a 2023 | ||
| 998 | |g 1286759269 |a Resin, Johannes |m 1286759269:Resin, Johannes |d 180000 |d 181000 |e 180000PR1286759269 |e 181000PR1286759269 |k 0/180000/ |k 1/180000/181000/ |p 1 |x j |y j | ||
| 999 | |a KXP-PPN1878838245 |e 4467207651 | ||
| BIB | |a Y | ||
| JSO | |a {"title":[{"title_sort":"From classification accuracy to proper scoring rules","subtitle":"elicitability of probabilistic top list predictions","title":"From classification accuracy to proper scoring rules"}],"person":[{"family":"Resin","given":"Johannes","display":"Resin, Johannes","roleDisplay":"VerfasserIn","role":"aut"}],"note":["Gesehen am 24.01.2024"],"type":{"media":"Online-Ressource","bibl":"chapter"},"language":["eng"],"recId":"1878838245","origin":[{"dateIssuedDisp":"27 Jan 2023","dateIssuedKey":"2023"}],"id":{"doi":["10.48550/arXiv.2301.11797"],"eki":["1878838245"]},"name":{"displayForm":["Johannes Resin"]},"physDesc":[{"extent":"21 S."}],"relHost":[{"language":["eng"],"recId":"509006531","disp":"From classification accuracy to proper scoring rules elicitability of probabilistic top list predictionsArxiv","type":{"media":"Online-Ressource","bibl":"edited-book"},"note":["Gesehen am 28.05.2024"],"titleAlt":[{"title":"Arxiv.org"},{"title":"Arxiv.org e-print archive"},{"title":"Arxiv e-print archive"},{"title":"De.arxiv.org"}],"part":{"pages":"1-21","year":"2023","extent":"21","text":"(2023), Artikel-ID 2301.11797, Seite 1-21"},"pubHistory":["1991 -"],"title":[{"title":"Arxiv","title_sort":"Arxiv"}],"physDesc":[{"extent":"Online-Ressource"}],"id":{"zdb":["2225896-6"],"eki":["509006531"]},"origin":[{"publisherPlace":"Ithaca, NY ; [Erscheinungsort nicht ermittelbar]","dateIssuedDisp":"1991-","dateIssuedKey":"1991","publisher":"Cornell University ; Arxiv.org"}]}]} | ||
| SRT | |a RESINJOHANFROMCLASSI2720 | ||