Measuring and Reporting Intercoder Reliability in Plan Quality Evaluation Research

Plan quality evaluation researchers typically evaluate plans in relation to whether they contain certain desirable features. Best practice dictates that plans be evaluated by at least two readers and that researchers report a measure of the extent to which the readers agree on whether the plans cont...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of planning education and research Jg. 34; H. 1; S. 77 - 93
Hauptverfasser: Stevens, Mark R., Lyles, Ward, Berke, Philip R.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Los Angeles, CA SAGE Publications 01.03.2014
Sage Publications Ltd
Schlagworte:
Agreements
Alternative approaches
Best practice
Criticism
Data quality
Education
Evaluation research
Planning
Readers
Reliability
Reporting
Researchers
percent agreement
Krippendorff’s alpha
intercoder reliability
content analysis
plan quality
plan evaluation
ISSN:0739-456X, 1552-6577
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Plan quality evaluation researchers typically evaluate plans in relation to whether they contain certain desirable features. Best practice dictates that plans be evaluated by at least two readers and that researchers report a measure of the extent to which the readers agree on whether the plans contain the desirable features. Established practice for assessing this agreement has been subject to criticism. We summarize this criticism, discuss an alternative approach to assessing agreement, and provide recommendations for plan quality evaluation researchers to follow to improve the quality of their data and the manner in which they assess and report that quality.
Bibliographie:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0739-456X
1552-6577
DOI:10.1177/0739456X13513614