In EDS ansehen

Decoding the Atmosphere: Optimising Probabilistic Forecasts with Information Gain.

Gespeichert in:
Bibliographische Detailangaben
Titel: Decoding the Atmosphere: Optimising Probabilistic Forecasts with Information Gain.
Autoren: Lawson, John R., Potvin, Corey K., Nelson, Kenric
Quelle: Meteorology; Jun2024, Vol. 3 Issue 2, p212-231, 20p
Schlagwörter: WEATHER forecasting, METEOROLOGICAL databases, EXTREME weather, MACHINE learning, PROBABILISTIC databases
Abstract: Probabilistic prediction models exist to reduce surprise about future events. This paper explores the evaluation of such forecasts when the event of interest is rare. We review how the family of Brier-type scores may be ill-suited to evaluate predictions of rare events, and we offer an alternative to information-theoretical scores such as Ignorance. The reduction in surprise provided by a set of forecasts is represented as information gain, a frequent loss function in machine learning training, meaning the reduction in ignorance over a baseline having received a new forecast. We evaluate predictions of a synthetic dataset of rare events and demonstrate the differences in interpretation of the same datasets depending on whether the Brier or Ignorance score is used. While the two types of scores are broadly similar, there are substantial differences in interpretation at extreme probabilities. Information gain is measured in units of bits, an irreducible unit of information, that allows forecasts of different variables to be comparatively evaluated fairly. Further insight from information-based scores is gained via a similar reliability–discrimination decomposition as found in Brier-type scores. We conclude by crystallising multiple concepts to better equip forecast-system developers and decision-makers with tools to navigate complex trade-offs and uncertainties that characterise meteorological forecasting. To this end, we also provide computer code to reproduce data and figures herein. [ABSTRACT FROM AUTHOR]
Copyright of Meteorology is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Datenbank: Complementary Index
Beschreibung
Abstract:Probabilistic prediction models exist to reduce surprise about future events. This paper explores the evaluation of such forecasts when the event of interest is rare. We review how the family of Brier-type scores may be ill-suited to evaluate predictions of rare events, and we offer an alternative to information-theoretical scores such as Ignorance. The reduction in surprise provided by a set of forecasts is represented as information gain, a frequent loss function in machine learning training, meaning the reduction in ignorance over a baseline having received a new forecast. We evaluate predictions of a synthetic dataset of rare events and demonstrate the differences in interpretation of the same datasets depending on whether the Brier or Ignorance score is used. While the two types of scores are broadly similar, there are substantial differences in interpretation at extreme probabilities. Information gain is measured in units of bits, an irreducible unit of information, that allows forecasts of different variables to be comparatively evaluated fairly. Further insight from information-based scores is gained via a similar reliability–discrimination decomposition as found in Brier-type scores. We conclude by crystallising multiple concepts to better equip forecast-system developers and decision-makers with tools to navigate complex trade-offs and uncertainties that characterise meteorological forecasting. To this end, we also provide computer code to reproduce data and figures herein. [ABSTRACT FROM AUTHOR]
ISSN:26740494
DOI:10.3390/meteorology3020010