A refined index of model performance: a rejoinder

Willmott et al. [Willmott CJ, Robeson SM, Matsuura K. 2012. A refined index of model performance. International Journal of Climatology, forthcoming. DOI:10.1002/joc.2419.] recently suggest a refined index of model performance (dr) that they purport to be superior to other methods. Their refined inde...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:International journal of climatology Ročník 33; číslo 4; s. 1053 - 1056
Hlavní autori: Legates, David R., McCabe, Gregory J.
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Chichester, UK John Wiley & Sons, Ltd 30.03.2013
Wiley
Wiley Subscription Services, Inc
Predmet:
accuracy indices
coefficient of efficiency
Earth, ocean, space
Exact sciences and technology
External geophysics
Geophysics. Techniques, methods, instrumentation and models
goodness‐of‐fit
Meteorology
model evaluation
model‐performance statistics
Other topics in atmospheric geophysics
Performance evaluation
Climatology
models
efficiency
Proxy
accuracy indices
model evaluation
coefficient of efficiency
model-performance statistics
goodness-of-fit
statistical analysis
ISSN:0899-8418, 1097-0088
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Willmott et al. [Willmott CJ, Robeson SM, Matsuura K. 2012. A refined index of model performance. International Journal of Climatology, forthcoming. DOI:10.1002/joc.2419.] recently suggest a refined index of model performance (dr) that they purport to be superior to other methods. Their refined index ranges from − 1.0 to 1.0 to resemble a correlation coefficient, but it is merely a linear rescaling of our modified coefficient of efficiency (E1) over the positive portion of the domain of dr. We disagree with Willmott et al. (2012) that dr provides a better interpretation; rather, E1 is more easily interpreted such that a value of E1 = 1.0 indicates a perfect model (no errors) while E1 = 0.0 indicates a model that is no better than the baseline comparison (usually the observed mean). Negative values of E1 (and, for that matter, dr < 0.5) indicate a substantially flawed model as they simply describe a ‘level of inefficacy’ for a model that is worse than the comparison baseline. Moreover, while dr is piecewise continuous, it is not continuous through the second and higher derivatives. We explain why the coefficient of efficiency (E or E2) and its modified form (E1) are superior and preferable to many other statistics, including dr, because of intuitive interpretability and because these indices have a fundamental meaning at zero. We also expand on the discussion begun by Garrick et al. [Garrick M, Cunnane C, Nash JE. 1978. A criterion of efficiency for rainfall‐runoff models. Journal of Hydrology 36: 375‐381.] and continued by Legates and McCabe [Legates DR, McCabe GJ. 1999. Evaluating the use of “goodness‐of‐fit” measures in hydrologic and hydroclimatic model validation. Water Resources Research 35(1): 233‐241.] and Schaefli and Gupta [Schaefli B, Gupta HV. 2007. Do Nash values have value? Hydrological Processes 21: 2075‐2080. DOI: 10.1002/hyp.6825.]. This important discussion focuses on the appropriate baseline comparison to use, and why the observed mean often may be an inadequate choice for model evaluation and development. Copyright © 2012 Royal Meteorological Society
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
content type line 14
ObjectType-Feature-2
content type line 23
ISSN:0899-8418
1097-0088
DOI:10.1002/joc.3487