Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination
A common principle in model diagnostics and forecast evaluation is that fitted or predicted distributions ought to be reliable, ideally in the sense of auto-calibration, where the outcome is a random draw from the posited distribution. For binary responses, auto-calibration is the universal concept...
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
2023
|
| In: |
Electronic journal of statistics
Year: 2023, Jahrgang: 17, Heft: 2, Pages: 3226-3286 |
| ISSN: | 1935-7524 |
| DOI: | 10.1214/23-EJS2180 |
| Online-Zugang: | Verlag, kostenfrei, Volltext: https://doi.org/10.1214/23-EJS2180 Verlag, kostenfrei, Volltext: https://projecteuclid.org/journals/electronic-journal-of-statistics/volume-17/issue-2/Regression-diagnostics-meets-forecast-evaluation--conditional-calibration-reliability-diagrams/10.1214/23-EJS2180.full 
|
| Verfasserangaben: | Tilmann Gneiting and Johannes Resin |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1911756133 | ||
| 003 | DE-627 | ||
| 005 | 20250716214928.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 241210s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1214/23-EJS2180 |2 doi | |
| 035 | |a (DE-627)1911756133 | ||
| 035 | |a (DE-599)KXP1911756133 | ||
| 035 | |a (OCoLC)1528014491 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 17 |2 sdnb | ||
| 100 | 1 | |a Gneiting, Tilmann |e VerfasserIn |0 (DE-588)1019627484 |0 (DE-627)690974809 |0 (DE-576)358470323 |4 aut | |
| 245 | 1 | 0 | |a Regression diagnostics meets forecast evaluation |b conditional calibration, reliability diagrams, and coefficient of determination |c Tilmann Gneiting and Johannes Resin |
| 264 | 1 | |c 2023 | |
| 300 | |b Illustrationen | ||
| 300 | |a 61 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 10.12.2024 | ||
| 520 | |a A common principle in model diagnostics and forecast evaluation is that fitted or predicted distributions ought to be reliable, ideally in the sense of auto-calibration, where the outcome is a random draw from the posited distribution. For binary responses, auto-calibration is the universal concept of reliability. For real-valued outcomes, a general theory of calibration has been elusive, despite a recent surge of interest in distributional regression and machine learning. We develop a framework rooted in probability theory, which gives rise to hierarchies of calibration, and applies to both predictive distributions and stand-alone point forecasts. In a nutshell, a prediction is conditionally T-calibrated if it can be taken at face value in terms of an identifiable functional T. We introduce population versions of T-reliability diagrams and revisit a score decomposition into measures of miscalibration, discrimination, and uncertainty. In empirical settings, stable and efficient estimators of T-reliability diagrams and score components arise via nonparametric isotonic regression and the pool-adjacent-violators algorithm. For in-sample model diagnostics, we propose a universal coefficient of determination that nests and reinterprets the classical R2 in least squares regression and its natural analog R1 in quantile regression, yet applies to T-regression in general. | ||
| 650 | 4 | |a 62G99 | |
| 650 | 4 | |a 62J20 | |
| 650 | 4 | |a Calibration test | |
| 650 | 4 | |a canonical loss | |
| 650 | 4 | |a consistent scoring function | |
| 650 | 4 | |a model diagnostics | |
| 650 | 4 | |a nonparametric isotonic regression | |
| 650 | 4 | |a prequential principle | |
| 650 | 4 | |a score decomposition | |
| 650 | 4 | |a skill score | |
| 700 | 1 | |a Resin, Johannes |e VerfasserIn |0 (DE-588)1286759269 |0 (DE-627)1843351706 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Electronic journal of statistics |d Ithaca, NY : Cornell University Library, 2007 |g 17(2023), 2, Seite 3226-3286 |h Online-Ressource |w (DE-627)538998830 |w (DE-600)2381001-4 |w (DE-576)28134714X |x 1935-7524 |7 nnas |a Regression diagnostics meets forecast evaluation conditional calibration, reliability diagrams, and coefficient of determination |
| 773 | 1 | 8 | |g volume:17 |g year:2023 |g number:2 |g pages:3226-3286 |g extent:61 |a Regression diagnostics meets forecast evaluation conditional calibration, reliability diagrams, and coefficient of determination |
| 856 | 4 | 0 | |u https://doi.org/10.1214/23-EJS2180 |x Verlag |x Resolving-System |z kostenfrei |3 Volltext |
| 856 | 4 | 0 | |u https://projecteuclid.org/journals/electronic-journal-of-statistics/volume-17/issue-2/Regression-diagnostics-meets-forecast-evaluation--conditional-calibration-reliability-diagrams/10.1214/23-EJS2180.full |x Verlag |z kostenfrei |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20241210 | ||
| 993 | |a Article | ||
| 994 | |a 2023 | ||
| 998 | |g 1286759269 |a Resin, Johannes |m 1286759269:Resin, Johannes |d 180000 |d 181000 |e 180000PR1286759269 |e 181000PR1286759269 |k 0/180000/ |k 1/180000/181000/ |p 2 |y j | ||
| 999 | |a KXP-PPN1911756133 |e 4634248980 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"id":{"doi":["10.1214/23-EJS2180"],"eki":["1911756133"]},"origin":[{"dateIssuedDisp":"2023","dateIssuedKey":"2023"}],"name":{"displayForm":["Tilmann Gneiting and Johannes Resin"]},"relHost":[{"title":[{"title":"Electronic journal of statistics","subtitle":"EJS","title_sort":"Electronic journal of statistics"}],"type":{"bibl":"periodical","media":"Online-Ressource"},"disp":"Regression diagnostics meets forecast evaluation conditional calibration, reliability diagrams, and coefficient of determinationElectronic journal of statistics","language":["eng"],"recId":"538998830","pubHistory":["1.2007 -"],"part":{"extent":"61","text":"17(2023), 2, Seite 3226-3286","volume":"17","issue":"2","pages":"3226-3286","year":"2023"},"titleAlt":[{"title":"EJS"}],"origin":[{"publisherPlace":"Ithaca, NY","publisher":"Cornell University Library","dateIssuedKey":"2007","dateIssuedDisp":"2007-"}],"id":{"issn":["1935-7524"],"zdb":["2381001-4"],"eki":["538998830"]},"physDesc":[{"extent":"Online-Ressource"}]}],"physDesc":[{"noteIll":"Illustrationen","extent":"61 S."}],"title":[{"title_sort":"Regression diagnostics meets forecast evaluation","subtitle":"conditional calibration, reliability diagrams, and coefficient of determination","title":"Regression diagnostics meets forecast evaluation"}],"person":[{"given":"Tilmann","family":"Gneiting","role":"aut","roleDisplay":"VerfasserIn","display":"Gneiting, Tilmann"},{"role":"aut","roleDisplay":"VerfasserIn","display":"Resin, Johannes","given":"Johannes","family":"Resin"}],"language":["eng"],"recId":"1911756133","note":["Gesehen am 10.12.2024"],"type":{"bibl":"article-journal","media":"Online-Ressource"}} | ||
| SRT | |a GNEITINGTIREGRESSION2023 | ||