Comparing accuracy assessments to infer superiority of image classification methods

The z-test based on the Kappa statistic is commonly used to infer superiority of one map production method over another. Typically the same reference data set is used to calculate and next compare the Kappa's of the two maps. This data structure easily leads to dependence between the two error-...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:International journal of remote sensing Jg. 27; H. 1; S. 223 - 232
Hauptverfasser: de Leeuw, J., Jia, H., Yang, L., Liu, X., Schmidt, K., Skidmore, A. K.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Abingdon Taylor & Francis 10.01.2006
Taylor and Francis
Schlagworte:
Animal, plant and microbial ecology
Applied geophysics
Biological and medical sciences
Earth sciences
Earth, ocean, space
Exact sciences and technology
Fundamental and applied biological sciences. Psychology
General aspects. Techniques
Internal geophysics
system
Teledetection and vegetation maps
Netherlands
Western Europe
brackish-water environment
maps
Europe
accuracy
vegetation
Classifier
classification
evaluation
Randomization
salt marshes
imagery
Alternative method
statistical analysis
Performance
Data structure
errors
ISSN:0143-1161, 1366-5901
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The z-test based on the Kappa statistic is commonly used to infer superiority of one map production method over another. Typically the same reference data set is used to calculate and next compare the Kappa's of the two maps. This data structure easily leads to dependence between the two error-matrices. This may result in overly large variance estimates and too conservative inference about the difference in accuracy between the two methods. Tests considering the dependency between the error matrices would be more sensitive in such case. In this article we compare the performance of two such tests, a randomization and McNemar's test, with the traditional z-test. We compared 16 alternative methods to classify salt marsh vegetation in The Netherlands. The error matrices were positively associated in all 120 possible comparisons of pairs of classification methods. This suggests that dependency between pairs of error matrices used in classifier comparison is a common phenomenon. Both the randomization and McNemar test gave lower p values and rejected the null hypothesis of equal performance more frequently than the z-test. We therefore recommend considering their use.
Bibliographie:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 23
ISSN:0143-1161
1366-5901
DOI:10.1080/01431160500275762